Skip to content

Why Do Pandas Type Stubs Produce False Positives? (And How to Fix Them)

I added pandas-stubs to my project and ran mypy. Within seconds, my terminal filled with red errors. But when I checked each one, the code worked perfectly fine at runtime.

mypy output
error: Too many arguments for "__getitem__" of "DataFrame"
error: Item "None" of "Optional[Series]" has no attribute "loc"
error: Incompatible return value type (got "Series", expected "DataFrame")

I spent the next hour convinced my pandas code was broken. It wasn’t.

The Core Problem: Dynamic API Meets Static Types

Pandas operations are inherently dynamic. The return type of df.loc[...] depends on:

  • What you pass in (single label, list, slice, boolean mask)
  • What the DataFrame looks like at runtime
  • Whether the index is a simple Index or a MultiIndex

Type stubs must be conservative. When they can’t determine the exact return type, they choose the safest option—which often doesn’t match reality.

Here’s what triggered my investigation:

multiindex_example.py
import pandas as pd
# Create a MultiIndex DataFrame
df = pd.DataFrame(
{'value': [1, 2, 3, 4]},
index=pd.MultiIndex.from_tuples([
('A', 'x'), ('A', 'y'), ('B', 'x'), ('B', 'y')
])
)
# This works perfectly at runtime
result = df.loc[('A', 'x')] # Returns a Series
value = result['value'] # Returns 1

But mypy complains:

mypy errors
error: Too many arguments for "__getitem__" of "DataFrame"
error: Invalid index type "tuple[str, str]" for "DataFrame"

The code runs. The tests pass. The type checker screams.

Why This Happens

I dug into the pandas-stubs source and found the limitation. The loc indexer’s type signature looks roughly like this:

simplified_stubs.py
@overload
def __getitem__(self, key: str) -> Series: ...
@overload
def __getitem__(self, key: list[str]) -> DataFrame: ...
# No overload for tuple (MultiIndex access)

The stubs handle common cases but can’t express “this tuple key is valid when the DataFrame has a MultiIndex with matching levels.” That would require the type system to understand DataFrame schema at compile time—which Python’s type system cannot do.

A Reddit comment captured this frustration perfectly:

“Type-complete is one thing, unfortunately with pandas those stubs are rather useless in my experience, since they produce way too many false positives. Several use cases of pandas are just flat-out not supported by the stubs like loc on DataFrame with MultiIndex.”

The gap between “type-complete” (all public APIs have annotations) and “practically useful” (types that don’t generate noise) is real.

Strategy 1: Targeted Type Ignore

My first instinct was to add # type: ignore everywhere. Bad idea. I quickly lost track of which ignores were necessary versus which were hiding real bugs.

A better approach:

targeted_ignore.py
result = df.loc[('A', 'x')] # type: ignore[index]
value = result['value']

Be specific about what you’re ignoring (index vs a broad ignore). And add a comment explaining why:

documented_ignore.py
# MultiIndex tuple access not supported by pandas-stubs
# See: https://github.com/pandas-dev/pandas-stubs/issues/XXX
result = df.loc[('A', 'x')] # type: ignore[index]

Strategy 2: Runtime Validation with Type Assertions

I tried using cast() from typing:

runtime_check.py
from typing import cast
import pandas as pd
def get_value_safe(df: pd.DataFrame, idx: tuple) -> int:
"""Get value with runtime validation."""
result = df.loc[idx] # type: ignore[index]
if not isinstance(result, pd.Series):
raise ValueError(f"Expected Series, got {type(result)}")
return cast(int, result['value'])
value = get_value_safe(df, ('A', 'x'))

The cast() tells mypy “trust me, this is an int” while the runtime check catches actual errors. It’s verbose but safe.

Strategy 3: Pandera for Schema Validation

Then I discovered pandera. This is the solution I actually use in production:

pandera_schema.py
import pandera as pa
class MultiIndexSchema(pa.DataFrameModel):
value: int
class Config:
index = pa.MultiIndex([
pa.Index(str, name="level_0"),
pa.Index(str, name="level_1"),
])
@pa.check_types
def process_data(df: pa.typing.DataFrame[MultiIndexSchema]) -> int:
result = df.loc[('A', 'x')] # type: ignore[index]
return int(result['value'])
# This validates at runtime AND provides type hints
validated_df = MultiIndexSchema.validate(df)
result = process_data(validated_df)

Pandera gives you:

  1. Runtime validation - catches schema mismatches when your code runs
  2. Type hints - static checkers understand the schema
  3. Clear error messages - when validation fails, you know exactly why

The trade-off: you write more boilerplate. But in a data pipeline, that’s worth it.

Strategy 4: Use Better-Typed Alternatives

Sometimes the issue isn’t the stubs—it’s that I’m using the wrong API for type checking.

For scalar access, .at[] has more predictable typing:

scalar_access.py
# Instead of:
# result = df.loc[('A', 'x')]
# Use .at for scalar access:
value = df.at[('A', 'x'), 'value'] # Cleaner type signature

For cross-sections, .xs() expresses intent better:

xs_access.py
# Instead of complex loc:
# subset = df.loc['A']
# Use xs for cross-section:
subset = df.xs('A', level=0) # Returns DataFrame, clearer intent

These alternatives don’t solve every case, but they help where they apply.

Strategy 5: Gradual Typing for Pandas-Heavy Modules

For my ETL scripts that are 90% pandas operations, strict mypy was creating more noise than value. I created a py.typed configuration:

pyproject.toml excerpt
[tool.mypy]
python_version = "3.11"
strict = true
[[tool.mypy.overrides]]
module = "etl.*"
disable_error_code = ["index", "assignment"]
warn_return_any = false

This lets me keep strict typing for the rest of my codebase while being more permissive in data processing modules.

The Trade-off Matrix

Strategy comparison
┌─────────────────────┬───────────────┬───────────────┬─────────────────┐
│ Strategy │ Type Safety │ Runtime Safe │ Effort │
├─────────────────────┼───────────────┼───────────────┼─────────────────┤
│ type: ignore │ Low │ No │ Minimal │
│ Runtime + cast │ Medium │ Yes │ Medium │
│ Pandera │ High │ Yes │ High (initial) │
│ Better APIs │ Medium │ Yes │ Low │
│ Per-module config │ Variable │ No │ Low │
└─────────────────────┴───────────────┴───────────────┴─────────────────┘

Common Mistakes I Made

  1. Ignoring too broadly: # type: ignore on an entire function hides real issues. Be specific.

  2. Not documenting ignores: Three months later, I couldn’t remember why I added that ignore comment.

  3. Expecting type checkers to understand schema: A DataFrame’s schema is runtime data. No amount of type hints will make mypy know that column “value” contains integers.

  4. Using cast() without runtime checks: cast() is just a hint. If the data is wrong, you’ll get runtime errors anyway.

  5. Fighting the stubs: I tried to “fix” my code to make mypy happy. But the code was correct—the stubs just couldn’t express it.

When to Accept False Positives

Not every type error needs fixing. I now ask myself:

  • Is this code tested?
  • Does it run correctly in production?
  • Is the error clearly a limitation of the stubs?

If yes to all three, a targeted # type: ignore[index] with a comment is the pragmatic choice.

Why This Matters for Long-Term Maintainability

False positives create alert fatigue. When mypy reports 47 errors and 45 are false positives, I stop reading the output. Then I miss the 2 real errors.

The pandas team made a reasonable choice: type-complete stubs that are overly restrictive. The alternative—loose stubs that produce false negatives—would be worse. At least with false positives, I can add ignores where I know the code is correct.

Better tooling may emerge. Type narrowing based on runtime checks, schema inference from data, or IDE plugins that understand pandas semantics. Until then, a combination of targeted ignores, pandera validation, and realistic expectations keeps my code maintainable.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments