Skip to content
BSWEN
Blog

How to prevent context rot in long ChatGPT conversations

Cowrie
Cowrie
Dev @ Bswen

Problem

I’ve noticed something frustrating when working with ChatGPT on long coding tasks. The conversation starts well - the AI follows my instructions, uses the right patterns, and produces quality code. But after 50 or 100 messages, things start to degrade.

Here’s what I’ve seen:

[Message 75]
User: Remember to use the error handling pattern we established at message 20
AI: "I'll add error handling" [but doesn't actually follow the pattern]


[Message 82]
User: Why are you using camelCase? We agreed on snake_case at the start
AI: "Apologies, I'll fix that" [but forgets again 5 messages later]


[Message 90]
User: This is the third time you've forgotten about the rate limiting requirement
AI: "You're right, let me be more careful..." [but performance continues to degrade]

I found a Reddit discussion where others faced the same issue. One commenter put it well: “Never keep any session too long whichever method you are using you get context rot and will get worse and worse performance.”

The problem has a name: context rot.

What is context rot?

Context rot is the gradual degradation of AI performance as conversation history accumulates. The model loses track of:

  • Original instructions and constraints
  • Previously established context
  • Code or content it generated earlier
  • Specific formatting requirements

I’ve seen this happen when:

  • Conversations exceed 80-100 messages
  • I find myself repeating the same instructions
  • The AI starts “interpreting” instead of “following” explicit directions
  • Code quality becomes inconsistent

One user in the Reddit thread reported: “if the conversations get too long, it easily forgets what it is doing.” Another mentioned the AI “leaves stuff out” and starts “interpreting” code instead of preserving it accurately.

Why does this happen?

From what I’ve learned, LLMs have limited attention spans even within large context windows. Here’s what occurs:

  1. Early messages get diluted - As more tokens are added, the original instructions lose priority
  2. Recent context dominates - The model prioritizes what was just said over what was established 50 messages ago
  3. Complexity increases - Each new message adds to the complexity the model must track

The community identified there’s a “sweet spot” for context - not too much, not too little. But how do you find it?

How I prevent context rot

I use several strategies to maintain quality across long sessions. Let me walk through what works.

Strategy 1: Keep conversations focused

I try to keep conversations under 50-100 messages when possible. When I notice performance dropping, I start fresh rather than pushing through.

Here’s what I watch for:

  • The AI makes the same mistake 3+ times
  • I’m repeating instructions frequently
  • Code/content quality noticeably declines
  • I can’t remember what was agreed upon 20+ messages ago

Quality over quantity - shorter focused sessions beat long meandering ones.

Strategy 2: Chunking with validation checkpoints

This is the approach I use most often. I break large tasks into smaller, focused sessions.

At each checkpoint:

  1. Summarize what was accomplished
  2. Verify outputs match requirements
  3. Document key decisions and constraints
  4. Start fresh session with context summary

For example, when building a REST API:

Wrong approach: Single 200-message conversation

[Session 1, messages 1-200]
User: Build a complete REST API with auth, database, testing...
AI: [generates code over 200 messages, quality degrades]
User: Why did you forget the JWT middleware we added at message 50?
AI: I apologize, let me fix that... [introduces new bugs]

Correct approach: Chunked with checkpoints

[Session 1: Database Schema, messages 1-30]
User: Design the database schema for a user auth system
AI: [generates schema]
User: Great, summarize what we built.
AI: We created users, sessions, and refresh_tokens tables...


[Session 2: Auth Endpoints, messages 1-25]
User: Context: We're building auth API. Schema from Session 1: [paste summary]
User: Now build the login/register endpoints using that schema.
AI: [generates focused, high-quality code]

I’ve found that three focused 30-message conversations with proper context handoff will outperform one 150-message conversation every time.

Strategy 3: Context summarization

When I need to continue a long session, I create a concise summary before starting fresh:

## Context Summary for Next Session


**Goal:** [What are we building/solving?]


**Completed:**
- [ ] Task 1 - [brief description]
- [ ] Task 2 - [brief description]


**Critical Constraints:**
- Must use PostgreSQL (not MongoDB)
- JWT tokens expire in 15 minutes
- All endpoints require rate limiting


**Key Decisions:**
- Using bcrypt for password hashing
- Refresh tokens stored in database, not Redis


**Next Steps:**
- Build password reset flow
- Add email verification


**Files Created:**
- /src/auth/login.ts
- /src/database/schema.sql

Then I start a new conversation with this summary and ask the AI to confirm understanding before proceeding.

Strategy 4: Cross-model validation

For critical work, I use multiple AI models (ChatGPT + Gemini + Claude) to review each other’s outputs.

Here’s my process:

[Step 1: Generate with ChatGPT]
User: Write a function to validate email addresses using regex
ChatGPT: [provides regex solution]


[Step 2: Validate with Gemini]
User: Review this email validation regex for edge cases: [paste ChatGPT code]
Gemini: "This regex doesn't handle international domains with accented characters..."


[Step 3: Validate with Claude]
User: Check this email validation code for security issues: [paste improved version]
Claude: "Consider using a library like validator.js instead of regex..."


[Result: More robust, thoroughly-vetted solution]

This is particularly useful for:

  • Code reviews
  • Fact-checking
  • Technical documentation
  • Complex problem-solving

Strategy 5: Explicit context reminders

When I must continue long conversations, I periodically:

  • Restate key instructions
  • Quote back important constraints
  • Ask the AI to confirm it remembers the original goal
  • Use explicit references: “As we discussed earlier…”

Common mistakes I’ve made

I made these mistakes early on. Maybe you can avoid them.

Mistake 1: Pushing through degradation

I kept going despite obvious performance drops, assuming “more context = better results.” That was wrong. When I see repetitive mistakes or forgotten instructions, I should start fresh.

Mistake 2: No validation between sessions

I didn’t verify outputs before continuing, lost track of what was accomplished, and assumed the AI remembered everything from previous sessions. Now I validate at each checkpoint.

Mistake 3: Poor context handoff

I started new conversations without adequate summaries or omitted critical constraints. Now I use the context summary template I showed you earlier.

Mistake 4: Ignoring early warning signs

I dismissed repetitive mistakes as one-offs or didn’t notice when the AI started “interpreting” instead of “following.” Now I recognize the warning signs and act immediately.

Mistake 5: Single-model reliance

I depended only on ChatGPT for critical tasks and didn’t cross-check important outputs. Now I use multiple models for anything important.

When to start fresh vs. continuing

Here’s what I’ve learned about when to do what:

Start a new conversation when:

  • Conversation exceeds 80-100 messages
  • AI makes the same mistake 3+ times
  • You find yourself repeating instructions frequently
  • Code/content quality noticeably declines
  • AI starts “interpreting” instead of “following” explicit directions
  • You can’t remember what was agreed upon 20+ messages ago

Continue existing conversation when:

  • Under 50 messages and performance is still strong
  • Task is simple/linear (not complex multi-step)
  • You have a clear context summary ready
  • Critical constraints are fresh in mind
  • Working on a tight sub-task that doesn’t require full context

The reason

I think the key reason context rot occurs is that LLMs have finite attention within conversation windows. Early messages get diluted as tokens accumulate, and the model prioritizes recent context over earlier instructions.

The solution isn’t to avoid long conversations entirely - it’s to manage them intentionally. Chunking tasks, validating at checkpoints, and summarizing context between sessions keeps the AI focused on what matters.

Summary

In this post, I showed how to prevent context rot in long ChatGPT conversations using chunking with validation checkpoints, context summarization, and cross-model validation.

The key point is that quality beats quantity. I’ve found that maintaining focused sessions (under 50-100 messages) with proper context handoff will outperform long, degraded conversations every time.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: