The Deloitte AI Blunder: A Lesson in Why Human Oversight Still Matters

Oct 29

The Basics

In December 2024 the Department of Employment and Workplace Relations (DEWR) in Australia commissioned Deloitte to conduct an “independent assurance review” of its targeted compliance framework and related IT systems, which relate to the welfare/unemployment system. The fee for the report was around A$439,000 (US$290,000) and was published in July 2025.

What Went Wrong

After publication, academics and media flagged multiple errors in the report, including:

References to academic papers or court cases that did not exist. For example, citations attributed to fictitious scholars and a legal case that could not be verified.
Footnotes and references that were incorrect, misleading or entirely made-up.
A lack of traceability between the system rules being reviewed and the legislation they were supposed to implement.

Deloitte later admitted that part of the report was produced using a generative AI tool: specifically the “Azure OpenAI GPT-4o based tool chain licensed by DEWR and hosted on DEWR’s Azure tenancy”.
However, Deloitte stopped short of saying that AI caused the errors, instead emphasising that the core findings and recommendations remained unchanged.

The Outcome

Deloitte agreed to repay the final instalment under the contract (effectively a refund) following the error revelations.
The revised version of the report was republished after corrections: more than a dozen false footnotes/references were fixed, and the appendix added disclosure about the AI-tool use.
The DEWR maintained that despite the corrections, the substantive recommendations of the review did not change.

What This Means for the Advisory & Accounting Space

For firms like ours (full-service accounting + advisory for limited companies), the incident highlights several lessons:

AI doesn’t replace subject-matter rigour: Generative tools can produce plausible narrative and citations, but as this case shows, they may fabricate or hallucinate references if not correctly supervised.
Quality control is essential: Large consulting firms are under scrutiny: the regulator in the UK has warned about audit-quality risks from automated tools.
Transparency with clients and stakeholders: Disclosing when and how AI is used in deliverables matters. In the Deloitte case, the initial version lacked that disclosure.
The risk of reputational damage: For high-stake engagements (e.g., government, regulated environment), errors undermine credibility.
For smaller businesses, it’s a cautionary tale: If you engage advisory or consulting help, it’s wise to ask: what role did AI play in the deliverable? What checks were done?

Why It Doesn’t Fully Sink the Report

Interestingly, despite the errors, many commentators accept that the core conclusions of the review were likely valid (i.e., the broad structural issues identified in the welfare-compliance system).

So the episode isn’t simply “AI = useless”. Rather it’s a stark reminder: AI outputs still need human verification, domain expertise and strong governance.

Key Take-Away

For us at J‑Benn Finance Ltd:

When leveraging AI (e.g., for data-analysis, forecasting, reporting) maintain clarity about its role; ensure outputs are validated by human experts.
If we outsource work or purchase advisory output, check how the deliverable was produced: by humans, by AI, or hybrid – and what quality checks were applied.
Communicate to clients the value of having trusted human oversight in their financial advisory. It’s a differentiator: AI + human = better.
More broadly, this story reinforces our value-proposition: we are not just “number crunchers”, but trusted advisors who interpret, verify and guide. That matters especially when others might cut corners with flashy tech but weak controls.

Disclosure….this blog item was suggested by AI, summarised by AI, but check by a human that it actually happened!

James Bennett https://www.j-benn.co.uk