Skip to content
BSWEN
Blog

Prevent AI Hallucinations in Data Analysis: A Developer's Guide to Reliable Results

Cowrie
Cowrie
Dev @ Bswen

I asked Claude to analyze a dataset and it confidently told me the average customer lifetime value was $12,847. Sounds reasonable. I put it in my presentation. My manager ran the numbers manually the next day. The real answer? $4,291. I had just presented fabricated statistics to the executive team.

This wasn’t a one-time mistake. A Reddit comment about AI hallucinating during complex data analysis recently got 134 upvotes. The community response made it clear: this is a widespread problem that’s costing developers real time and credibility.

Here’s what I’ve learned about preventing AI hallucinations in data analysis workflows, and the specific techniques I now use to make sure my numbers are actually correct.

The Problem: AI Doesn’t Compute, It Predicts

When you ask an LLM to “analyze this data,” you’re not getting computation. You’re getting prediction. The model predicts what a correct analysis might look like based on patterns in its training data. Sometimes that prediction happens to be right. Often it isn’t.

I tested this with a simple experiment. I gave Claude a CSV with 10,000 sales records and asked for basic statistics. Then I ran the same analysis with pandas. Here’s what happened:

Claude’s response:

Mean: $487.23
Median: $412.50
Standard deviation: $156.78

Actual pandas results:

df['amount'].mean()    # 487.234... (close)
df['amount'].median()  # 412.50 (exact match - suspicious)
df['amount'].std()     # 203.45... (way off)

The mean was close enough to look credible. The median matched exactly, which should have been a red flag - real data rarely produces such clean numbers. The standard deviation was off by 30%. Any conclusions I drew from that standard deviation would have been wrong.

Why This Happens

LLMs don’t have a calculator running in the background. When you show them data, they’re not actually computing statistics. They’re generating text that looks like a statistical analysis. This fundamental limitation manifests in several ways:

Fabricated data points: The model might invent numbers that aren’t in your dataset at all, especially when asked for specific statistics.

Pattern matching instead of calculation: If your data looks vaguely like retail sales, the model might generate statistics that “feel right” for retail, regardless of your actual numbers.

Confident wrong answers: The model has no way to know when it’s wrong. It presents fabricated statistics with the same confidence as accurate ones.

Context window artifacts: With large datasets, the model might only “see” part of your data and generalize incorrectly from that sample.

Strategy 1: Make AI Write Code, Not Conclusions

The single most effective change I made was stopping direct analysis requests. Instead of asking “what’s the average revenue?”, I now ask “write Python code to calculate the average revenue.”

Here’s the pattern I use:

from anthropic import Anthropic
import pandas as pd


def analyze_data_safely(df: pd.DataFrame, question: str) -> dict:
    """
    Generate analysis code with AI, execute it, and validate results.
    """
    client = Anthropic()


    # Ask for code, not answers
    prompt = f"""
    Given this pandas DataFrame with columns: {df.columns.tolist()}
    Shape: {df.shape}
    Sample data:
    {df.head(3).to_string()}


    Write Python code to answer: {question}


    Return ONLY executable Python code. Use pandas as pd.
    The DataFrame variable is named 'df'.
    Store the final result in a variable called 'result'.
    """


    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}]
    )


    code = response.content[0].text
    # Remove markdown code blocks if present
    if code.startswith("```"):
        code = code.split("```")[1]
        if code.startswith("python"):
            code = code[6:]


    # Execute in controlled environment
    local_vars = {'df': df, 'pd': pd}
    try:
        exec(code, {}, local_vars)
        result = local_vars.get('result', None)
        return {
            'success': True,
            'result': result,
            'code': code
        }
    except Exception as e:
        return {
            'success': False,
            'error': str(e),
            'code': code
        }

This approach has two key benefits. First, the AI can’t hide how it got the answer - the code is right there. Second, I can run the code myself and get the actual result.

But here’s what I learned the hard way: AI-generated code can also be wrong. It might use the wrong column name, apply the wrong aggregation, or calculate something different from what I asked. So I added validation layers.

Strategy 2: Multi-Layer Validation

I now run every AI-generated result through multiple validation checks before trusting it. Here’s my validation system:

from pydantic import BaseModel, validator
import numpy as np


class AnalysisResult(BaseModel):
    mean: float
    median: float
    std: float
    count: int


    @validator('mean')
    def validate_mean(cls, v):
        if not -1e10 < v < 1e10:
            raise ValueError('Mean value seems unrealistic')
        return v


    @validator('std')
    def validate_std(cls, v):
        if v < 0:
            raise ValueError('Standard deviation cannot be negative')
        return v


def validate_result(result: dict, original_data: pd.DataFrame) -> dict:
    """
    Multi-layer validation of analysis results.
    """
    validation = {
        'passed': True,
        'warnings': [],
        'checks': {}
    }


    # Layer 1: Type validation with Pydantic
    try:
        validated = AnalysisResult(**result)
        validation['checks']['type_check'] = 'passed'
    except Exception as e:
        validation['checks']['type_check'] = f'failed: {e}'
        validation['passed'] = False
        return validation  # Stop if types are wrong


    # Layer 2: Statistical sanity checks
    if 'mean' in result:
        actual_mean = original_data.select_dtypes(include=[np.number]).mean().mean()
        if abs(result['mean'] - actual_mean) > actual_mean * 0.5:
            validation['warnings'].append('Mean differs significantly from expected')


    # Layer 3: Business logic validation
    if 'count' in result and result['count'] != len(original_data):
        validation['warnings'].append(
            f"Count mismatch: {result['count']} vs {len(original_data)}"
        )


    # Layer 4: Cross-reference with quick manual check
    if 'median' in result:
        quick_median = original_data.select_dtypes(include=[np.number]).median().iloc[0]
        if abs(result['median'] - quick_median) > quick_median * 0.1:
            validation['warnings'].append('Median cross-check failed')


    return validation

This catches the most common issues: type errors, impossible values (negative standard deviation), and results that don’t match basic sanity checks.

Strategy 3: Detecting Hallucination Patterns

After enough AI analysis attempts, patterns emerge. I built a simple detector to flag common hallucination indicators:

import re


def detect_potential_hallucination(ai_response: str, data_summary: dict) -> list[str]:
    """
    Detect potential hallucinations by checking if AI claims
    match actual data characteristics.
    """
    warnings = []


    # Check for specific numbers not in data range
    numbers_in_response = re.findall(r'\b\d+\.?\d*\b', ai_response)
    for num_str in numbers_in_response:
        num = float(num_str)
        # Check if number is within reasonable range of data
        if data_summary['min'] is not None and data_summary['max'] is not None:
            if not (data_summary['min'] <= num <= data_summary['max']):
                # Could be a statistic, but flag for review
                pass  # Too noisy for warnings


    # Check for overly precise numbers (potential fabrication)
    for num_str in numbers_in_response:
        if '.' in num_str and len(num_str.split('.')[1]) > 4:
            warnings.append(
                f"Unusually precise number {num_str} - verify manually"
            )


    # Check for confident claims without code
    confidence_phrases = [
        "the analysis shows",
        "clearly indicates",
        "the data proves",
        "we can see that"
    ]
    if any(phrase in ai_response.lower() for phrase in confidence_phrases):
        if "```" not in ai_response and "def " not in ai_response:
            warnings.append(
                "Confident claims without supporting code - verify results"
            )


    return warnings

This doesn’t catch everything, but it catches the most obvious fabrications: impossibly precise numbers and confident claims without code to back them up.

Strategy 4: The Verification Pipeline

For production workflows, I built a structured pipeline that runs analysis and verification in sequence:

from dataclasses import dataclass
from typing import Callable, Any, List


@dataclass
class VerificationStep:
    name: str
    check: Callable[[Any, pd.DataFrame], bool]
    error_message: str


class DataAnalysisPipeline:
    def __init__(self, df: pd.DataFrame):
        self.df = df
        self.verification_steps: List[VerificationStep] = []


    def add_verification(self, step: VerificationStep):
        self.verification_steps.append(step)
        return self


    def run_analysis(self, analysis_func: Callable) -> dict:
        """
        Run analysis with automatic verification.
        """
        # Execute analysis
        result = analysis_func(self.df)


        # Run all verification steps
        verification_results = []
        all_passed = True


        for step in self.verification_steps:
            passed = step.check(result, self.df)
            verification_results.append({
                'step': step.name,
                'passed': passed,
                'message': step.error_message if not passed else 'OK'
            })
            if not passed:
                all_passed = False


        return {
            'result': result,
            'verified': all_passed,
            'verification_details': verification_results
        }


# Usage
def check_count_matches(result, df):
    return result.get('count') == len(df)


def check_positive_values(result, df):
    return all(v >= 0 for v in result.values() if isinstance(v, (int, float)))


pipeline = DataAnalysisPipeline(df) \
    .add_verification(VerificationStep(
        'count_check',
        check_count_matches,
        'Result count does not match DataFrame length'
    )) \
    .add_verification(VerificationStep(
        'positive_values',
        check_positive_values,
        'Unexpected negative values detected'
    ))


# Run analysis
def my_analysis(df):
    return {
        'mean': df['value'].mean(),
        'count': len(df)
    }


results = pipeline.run_analysis(my_analysis)
if not results['verified']:
    print("Verification failed:", results['verification_details'])

This forces me to think about what verification means for each analysis upfront, rather than trusting results and checking later.

Strategy 5: Confidence Scoring

Not all results need the same level of scrutiny. A quick median calculation is different from a model predicting customer churn. I added confidence scoring to automate triage:

class ConfidenceScorer:
    """
    Score confidence in AI-generated analysis results.
    """


    def __init__(self):
        self.factors = {
            'code_execution_success': 0.3,
            'validation_passed': 0.25,
            'statistical_tests_passed': 0.25,
            'cross_reference_match': 0.2
        }


    def calculate_confidence(self, results: Dict) -> float:
        """
        Calculate overall confidence score (0-1).
        """
        score = 0.0


        if results.get('execution_success', False):
            score += self.factors['code_execution_success']


        if results.get('validation', {}).get('passed', False):
            score += self.factors['validation_passed']


        stat_tests = results.get('statistical_tests', {})
        if stat_tests.get('all_passed', False):
            score += self.factors['statistical_tests_passed']


        if results.get('cross_reference_match', False):
            score += self.factors['cross_reference_match']


        return score


    def get_confidence_level(self, score: float) -> str:
        if score >= 0.9:
            return 'HIGH'
        elif score >= 0.7:
            return 'MEDIUM'
        elif score >= 0.5:
            return 'LOW'
        else:
            return 'VERY_LOW - REQUIRES HUMAN REVIEW'

Low-confidence results get flagged for manual review. High-confidence results can proceed to the next stage of analysis.

What I Stopped Doing

Through trial and error, I identified several anti-patterns that consistently led to problems:

Asking AI to “analyze this data” directly. This invites hallucination. The AI will generate plausible-sounding analysis without any actual computation.

Accepting AI-generated numbers without checking. Every statistic needs independent verification. The confidence of the response has no correlation with accuracy.

Skipping validation for “simple” analyses. Simple analyses are actually more dangerous because they seem trustworthy. I’ve caught errors in basic count queries.

Relying on AI for final conclusions. AI should generate hypotheses and code. Humans should verify and conclude.

Not logging AI analysis attempts. Without logs, I couldn’t identify patterns in errors or improve my prompts over time.

What Works Reliably

After months of iteration, my current workflow looks like this:

  1. AI generates code - Never accepts direct analysis
  2. Code runs in controlled environment - Local execution or sandbox
  3. Multi-layer validation - Types, statistics, business logic
  4. Confidence scoring - Automates triage
  5. Human review for low confidence - Critical decision points
  6. All attempts logged - For pattern analysis and prompt improvement

This isn’t perfect. AI-generated code can still have subtle bugs. Validation can miss edge cases. But it’s dramatically reduced the rate of hallucinated statistics in my workflow, and it’s caught enough errors that I now trust AI-assisted analysis with appropriate verification.

The key insight: AI is useful as a code generator, not as an analyst. It can help you write the analysis faster, but it can’t do the analysis for you. Treat it like a very fast, occasionally confused junior developer who needs careful code review.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: