#4 - AI hallucinations in a report on AI adoption and handling inappropriate kid trends

The recent withdrawal of an AI adoption report by KPMG shows why "human-in-the-loop" protocols fail. Let's see how organisations and individuals can protect their reputations from AI pollution - and also discuss how algorithms come from screens to playgrounds and what parents/educators can do.

Share
#4 - AI hallucinations in a report on AI adoption and handling inappropriate kid trends
"Air Apparition": Hallucination (Pencil), 1922. Artist: Hans Prinzhorn. Source

KPMG published a report about AI adoption, independent researchers found fabricated examples inside it, and the report was withdrawn.

Is it just about a single consulting firm's embarrassment?

No - because every organisation using GenAI is (or should be!) asking now:

Could this happen to us? Do we even know?

For this issue's deep dive, we're examining the KPMG case from several angles - what it means for AI governance, for deployers integrating AI into workflows, and for users consuming AI-generated information, including those who may not even realise how much of what they read is AI-generated.

In the second part of the issue, we bring the conversation home to look at how viral algorithms come from screens to playgrounds, and share a "secret weapon" phrase parents can use to strip inappropriate internet memes of their cool factor.

Let's get into it.

What the KPMG report scandal reveals about AI governance, and why it concerns everyone

On June 14, KPMG - one of the world's "Big Four" accounting and consulting firms - quietly withdrew a published report on AI adoption ("Redefining excellence in the age of agentic AI") in business after independent researchers discovered it contained fabricated case studies and unverifiable claims.

Organisations cited as successful AI adopters either didn't exist or had no public record of implementing the described systems.

The evidence points to the report being at least partially generated by an AI model whose hallucinations were never caught by human reviewers before publication. Only 5 citations - out of 45! - were fully accurate.

💡
New term alert: VIBE CITING (when an AI hallucinates references, footnotes, or statistics that look incredibly realistic on the surface but are completely made up)

What makes this extraordinary is not the fact of AI-generated falsehood - we have seen it all before. What IS extraordinary is who failed: a firm whose entire commercial identity rests on accuracy, verification, and trust.

One of the comments under the FT article
...and a fair concern by another FT reader - information poisoning is real

🧠 Implications for AI governance

The core problem: this case is a textbook failure of governance-as-designed versus governance-as-practiced. Every responsible AI framework in existence, from the NIST AI Risk Management Framework to the EU AI Act's requirements - includes some version of "human oversight" or "human-in-the-loop." But this case proves that adding the principle to your AI use policy is meaningless without the operational mechanism.

The failures cascade across multiple governance layers:

  • Policy: do we allow AI use in client-facing publications without human verification?
  • Process: is there a mandatory fact-checking workflow for AI-generated claims?
  • Technical: do we have any AI-generated content detection tools or watermarking practice in internal documents?
  • Accountability: do we have a named owner responsible for the accuracy of AI-assisted work product?

The "human-in-the-loop" standard is insufficient without this specificity.

"A human reviews the output" means nothing unless you define:

  • What qualifications/training does that human need?
  • What specific checks must they perform?
  • Under what incentives?

A junior employee clicking "looks good" on an AI-generated paragraph is technically a "human in the loop" but may be useless.

If you are an AI deployer - run this checklist now

  1. Inventory all AI-generated content that leaves your organisation. Reports, client communications, marketing materials, support
    responses - everything. Do you know which of these have been checked by a qualified human reviewer? If not, assume some are wrong.
  2. Create a tiered verification system. Not all content needs the same level of scrutiny. A chatbot greeting a customer needs less checking than a consulting report. Define tiers - for example:
    • No human review (internal drafts, brainstorming)
    • Spot-check review (low-stakes customer interactions)
    • Full verification (published reports, financial advice, medical information, legal documents)
    • Expert validation (anything with compliance or legal implications)
  3. Implement technical guardrails. There are now AI governance platforms - tools that can:
    • Flag AI-generated content for mandatory review
    • Detect statistical patterns of hallucination risk
    • Maintain immutable audit trails of who reviewed what and when
    • Block publication of AI-generated claims without a verified source
  4. Train your people on AI skepticism. The most dangerous approach in AI deployment is "AI said it". Every employee who touches AI tools needs to understand that AI generates plausible-sounding text, not verified truth. This is not about discouraging AI use - it's about creating a culture of verification.
  5. Establish escalation pathways. When an employee suspects that AI-generated content contains errors, there must be a clear, non-punitive
    process for flagging it. The employee who caught KPMG's errors was an external researcher. Imagine what internal processes could have caught them before publication.

The reputational calculus is brutal: such errors can damage your brand. The cost of a few hours of human verification is trivial compared to the cost of a withdrawn report, a scandal, and the question "can we trust anything from you?"

If you want to learn more about non-monetary costs of AI errors, check out my recent article:

NIST AI RMF Map 3.2: What is the “human cost” of your AI system?
Today, I move to a sub-category that often separates the “compliant” from the truly “responsible”: Map 3.2.

If you use AI tools daily,

you are on the front lines of this problem.

You need a personal verification protocol. Before you forward, publish, or act on AI-generated content, ask yourself:

  • Would I stake my professional reputation on this being true?
  • Can I find an independent source for each specific claim?
  • If I am wrong, who gets hurt?

Document your verification process. If you're using AI to draft something important, keep a record of your fact-checking steps.

Watch for signs of AI pollution:

  • References that sound plausible but are hard to verify ("a 2023 study by a major European university found...").
  • Specific numbers that feel "too round" or too perfect.
  • Case studies that describe generic situations without identifiable details

The burden is unfairly on you. None of this is your fault. The systems should be better, the processes should be in place, the verification should happen automatically. But in the current environment, the last line of defense is the human who reads it before it goes out - so let's take measures to protect ourselves accordingly.

Defeating the playground algorithms

My LinkedIn followers may already know that when it comes to social media/scrolling/algorithms and young kids, I have a strong opinion: my 8-year-old has no access to TikTok or YouTube, unless we watch something educational (and, occasionally, just fun) together.

But lately, I have noticed during playdates that my kid’s friends dropping highly specific, repetitive sounds of an unmistakably adult character into their daily games, which I suspect to be some sort of memes from viral internet trends.

I found it deeply uncomfortable - and, of course, highly frustrating: we try so hard to keep this stuff out of our house, yet it walked through the front door anyway. And I found I am not the only one:

Reddit post on kids making moaning noises and sexualised movements

So I did some research to figure out how to react to all this. And these are the recommendations I have found - I hope you will find them useful too.

We need to understand that elementary school-age kids are likely making these sounds has absolutely zero adult intent. To them, it’s just a high-status sound effect that gets a laugh from their peers. But it also forces us into a unique modern parenting role: we are not just managing devices, we have to build media literacy, too.

When this happens, our job is to calmly strip the sound of its "cool" mystery.

I had a brief conversation with my kid.

"I know your friends think that’s a funny gaming meme/sound/move, but it’s actually something from adult videos. They may not know what it means, but now you do. When you guys are doing it, it makes you look silly - like babies who don't understand what they are doing”.

Yes, apparently, this last line is the secret weapon!

At this age, telling a kid they are making adults uncomfortable can accidentally make the trend feel powerful and rebellious. But telling them it makes them look silly? That completely kills the cool factor.

Obviously, we cannot build a perfect digital bubble around our kids but we can give them a dose of age-appropriate critical thinking and help them build that internal filter.

How are you handling the clash between your home's digital boundaries and peer-group trends? Let’s swap strategies!

Off-duty

Four issues in, this section is still my favorite. Here, I look at life beyond work - it's a curated mix of hobbies, books, movies, podcast and music recommendations and reviews.

What I am reading

I’m officially halfway through Isabel Allende's Violeta in the original Spanish, and the experience has completely transformed. What started as a challenge has become genuinely enjoyable because I can feel my vocabulary growing and my comprehension locking into place with every chapter.

It has been very interesting to observe this change in my perceptions. If you want to try reading in a foreign language, remember - the first 20–30% is the hardest. Every author has a specific "lexical fingerprint" - it is a set of favorite words, idioms, and sentence structures they use repeatedly. Once your brain adapts to their specific voice in the first few chapters, the cognitive load drops drastically.

True enjoyment happens when you stop translating words in your head. At some point, your brain starts linking foreign language words directly to the images and concepts they represent, and you start experiencing the narrative.

Another secret to my vocabulary growth is that I am not looking up every word - I let context clues do its work, I read through minor ambiguities, and I let my brain naturally deduce the meanings of unfamiliar words based on the surrounding. This creates much stronger neurological pathways than looking a word up in a dictionary ever could.

What I am watching

I just wrapped up Season 2 of Paradise - the post-apocalyptic political thriller that layers in some fascinating sci-fi and physics elements. I won't share any spoilers, but it left me itching to read up on the real-world physics, quantum theories, and emerging tech concepts that inspired the plot. Anyone else find that the best sci-fi is not necessarily the most accurate, but the kind that forces you to open a dozen browser tabs to understand the actual science behind it?

What should I watch next?


Once again, thank you for being a part of this journey and reading along! What did you think of today's notes? Just reply to this email to share your take.

  • Let's connect: if we aren't LinkedIn official yet, let's change that!
  • If you liked this issue, please forward it to a friend! It’s the best way to help this newsletter grow.
  • If you were forwarded this email, you can subscribe to the newsletter here.

Best,

Katya