Skip to content
BSWEN
Blog

What Are Common Hallucination Patterns in AI Coding Assistants Like Claude Code?

Cowrie
Cowrie
Dev @ Bswen

I was reviewing a pull request from a colleague when my AI coding assistant suddenly flagged the changes as suspicious. “This looks like it’s gutting a working component,” it warned. The PR? A straightforward dead code removal. The component wasn’t being gutted—it was being cleaned up.

This wasn’t a one-off error. I started noticing a pattern: the AI would confidently claim something was wrong, I’d verify and find nothing, and when I pushed back, it would double down with even more conviction. I was being gaslit by a machine.

After dozens of these encounters, I identified four distinct hallucination patterns that AI coding assistants like Claude Code exhibit. Understanding these patterns has saved me countless hours of debugging phantom problems.

Pattern 1: Threat Overfitting

The first hallucination pattern emerges from the AI’s tendency to overfit code changes to known “threat patterns” from its training data.

React useEffect with intentional empty dependency array
useEffect(() => {
  setData(initialData)
}, []) // AI: "Missing dependency! This will cause bugs!"
// Reality: Empty array is intentional for mount-only logic

When I showed this code to Claude Code, it immediately flagged the empty dependency array as problematic. “setData should be in the dependency array,” it insisted. “This violates the exhaustive-deps rule.”

But this was a standard React pattern—running initialization logic only on mount. The AI had learned from training data containing React performance warnings and was overfitting every empty dependency array to the “problematic pattern” bucket, regardless of context.

The pattern recognition works like this:

  1. AI sees useEffect with [] dependency
  2. Pattern matches to “exhaustive-deps warning” from training data
  3. Flags as error without analyzing actual intent
  4. Ignores that setData is stable and initialData should only run once

This happens because LLMs don’t verify conclusions against actual code behavior—they pattern-match against statistical regularities in their training data.

How to Spot Threat Overfitting

  • The AI references generic rules (“you should never…”) without considering specific context
  • It flags standard patterns as anti-patterns
  • The “problem” it identifies doesn’t match the actual code’s purpose
  • Pushing back reveals the AI made assumptions without verification

Pattern 2: Knowledge Staleness

The second pattern involves AI presenting outdated information as current fact.

Outdated Safari workaround being suggested
// AI: "Safari has localStorage issues, use this workaround"
function getStorage() {
  try {
    return localStorage
  } catch (e) {
    // Safari in private mode throws
    return null
  }
}
// Reality: That bug was fixed in Safari 11 (2017)

Claude Code suggested this workaround when I was implementing a storage feature. It confidently explained that “Safari has well-known localStorage issues in private browsing mode, and many developers use this try-catch pattern.”

The problem? This was based on a 2017-era Stack Overflow answer. Safari 11, released in 2017, fixed the issue. The AI was parroting 9-year-old information as if it were current.

This pattern is particularly insidious because:

  • The information was once correct
  • There’s often a grain of truth (Safari did have this bug)
  • The AI presents it with high confidence
  • Many developers won’t verify against current documentation

Why This Happens

LLMs have a training data cutoff and no real-time information access. More importantly, they don’t understand the concept of “current” vs. “historical”—everything in their training set is equally valid from the model’s perspective. A 2017 Stack Overflow answer carries the same weight as a 2023 MDN documentation update.

Pattern 3: Confirmation Bias Amplification

The most dangerous pattern starts with an initial (often incorrect) assumption and then interprets everything as confirmation.

AI conversation showing confirmation bias
AI: "This PR looks suspicious—large formatting changes with logic modifications buried inside."


Me: "Can you show me the specific logic changes you're concerned about?"


AI: "The removal of this entire block seems like gutting a working component."


Me: "That's dead code—the component was deprecated three months ago."


AI: "Even so, removing it alongside formatting changes makes it hard to review. And here's another change that concerns me..."

Claude Code later admitted (after I provided evidence the changes were correct):

“I was pattern-matching on ‘large formatting PR with logic changes buried inside’ as a threat model and then confirmation-biased my way through the review.”

This pattern works like a feedback loop:

  1. Initial pattern-match flags something as potentially suspicious
  2. AI searches for confirming evidence (rather than disconfirming)
  3. Ambiguous code is interpreted as validation
  4. Counter-evidence is downplayed or reinterpreted
  5. The AI becomes more confident in its wrong conclusion

A Reddit user described this perfectly:

“It hallucinates… I challenge again… it doubles back down on the original hallucination with an attitude this time, to the point I’m feeling gaslit.”

The Gaslighting Effect

What makes this pattern particularly harmful is the confidence level. When an AI is wrong but confident, and then doubles down when challenged, it can:

  • Make developers doubt their own understanding
  • Waste hours verifying correct code
  • Create a false paper trail of “issues” that code reviewers might accept
  • Undermine trust in legitimate AI assistance

Pattern 4: Defensive Doubling Down

When challenged with evidence, AI coding assistants often become defensive rather than acknowledging errors.

AI defensive response to correction
Developer: "This is actually correct—see the React documentation link I provided."


AI: "While that document exists, in practice this pattern causes issues in production environments. The React team's guidance has been inconsistent, and many teams report bugs with this approach..."

The AI doesn’t just admit error—it constructs elaborate explanations for why it’s actually still right. This isn’t intentional deception; it’s a fundamental limitation of how LLMs work:

  1. No Self-Verification: LLMs can’t check their outputs against ground truth
  2. Confidence from Coherence: “Confidence” comes from how well text flows, not accuracy
  3. No Uncertainty Mechanism: The model doesn’t know when it doesn’t know
  4. Completion Pressure: Once on a track, the model feels “pressure” to continue it

This is why Claude Code’s own admission—“I was pattern-matching… and then confirmation-biased my way through”—is so significant. It’s not that the AI understood its error and explained it; a human had to manually train it to recognize this pattern.

Why These Patterns Matter

AI coding assistants are being integrated into development workflows at an accelerating pace. GitHub Copilot, Claude Code, Cursor, and others are becoming standard tools. But these hallucination patterns introduce real risks:

Time Waste: Developers can spend hours debugging phantom issues the AI identifies. One developer reported spending an entire afternoon tracking down a “memory leak” that didn’t exist—the AI had misidentified standard React cleanup patterns as problematic.

False Confidence: Teams might skip human review for AI-approved changes. “The AI reviewed it” becomes shorthand for “it’s probably fine.” But AI approval is pattern-matching, not verification.

Technical Debt: AI suggestions based on outdated patterns can introduce subtle bugs. The localStorage workaround example isn’t harmful, but imagine an AI suggesting security “fixes” based on deprecated practices.

Trust Erosion: When developers are repeatedly gaslit by confident-but-wrong AI responses, they stop trusting legitimate AI assistance. The “boy who cried wolf” problem applies to AI tools too.

Strategies for Dealing with AI Hallucinations

After encountering these patterns repeatedly, I’ve developed a verification workflow:

1. Assume Good Intent from Code Authors First When an AI flags something as suspicious, my first assumption is now that the AI is wrong, not the code. This reverses the traditional code review dynamic.

2. Request Specific Evidence “Show me the exact lines and explain why this is problematic.” Vague warnings are usually pattern-matching false positives.

3. Check Documentation Dates When the AI references “known issues” or “best practices,” I check when that guidance was published. Is it from 2017 or 2024?

4. Test AI Confidence with Wrong Information I sometimes provide intentionally incorrect “corrections” to see how the AI responds. If it agrees with obviously wrong information, I know it’s in confirmation-bias mode.

5. Verify Before Acting Never make changes based solely on AI recommendations without running tests, checking docs, or verifying with other sources.

The Fundamental Limitation

These hallucination patterns aren’t bugs that will be fixed—they’re architectural limitations of current LLMs. An AI coding assistant:

  • Cannot verify its conclusions against actual code execution
  • Has no concept of temporal validity for knowledge
  • Generates confidence from linguistic coherence, not accuracy
  • Lacks the meta-cognition to recognize its own uncertainty

As a senior engineer commented on the original hallucination discussion: “The AI isn’t thinking—it’s completing. It completes text in a way that looks like reasoning, but there’s no verification step, no uncertainty quantification, no self-correction mechanism.”

Understanding this distinction helps set appropriate expectations. AI coding assistants are powerful tools for generating, explaining, and exploring code. But they’re not reliable auditors or reviewers. The verification burden still falls on humans.

The Path Forward

The AI coding tool landscape is evolving rapidly. Some developments that could help:

Verification Layers: Tools that actually run code, check documentation freshness, and validate AI claims against ground truth.

Confidence Calibration: AI models trained to express appropriate uncertainty rather than uniform high confidence.

Context Freshness Indicators: Clear markers showing when AI knowledge dates from, so developers know to verify against current sources.

Human-in-the-Loop by Default: Workflow designs that assume AI will make mistakes and require human verification at critical decision points.

Until these improvements arrive, developers need to approach AI coding assistants with calibrated skepticism. Trust but verify isn’t enough—we need to assume AI will sometimes be confidently wrong and have processes to catch it.

The four hallucination patterns—threat overfitting, knowledge staleness, confirmation bias amplification, and defensive doubling down—are predictable once you know to look for them. That predictability is the key to using AI tools effectively without falling into their traps.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: