Skip to content
BSWEN
Blog

How to build automated continuity checking for long-form AI writing

Cowrie
Cowrie
Dev @ Bswen

Purpose

This post demonstrates how to build automated continuity checking for long-form AI writing.

Environment

  • Claude Code
  • Grep tools for pattern recognition
  • Custom scripts for cross-referencing
  • Reddit discussion of 300k-word AI novel

The Problem

When I wrote a 300k-word AI novel, I got this problem:

"Continuity problems are 'constant and relentless' at scale"
"Eye colors change, locations get redescribed differently, characters know things they shouldn't"

Here’s what my setup looked like:

300k-word manuscript
├── Chapter 1-150: Draft phase
├── Chapter 151-250: Review phase
└── Chapter 251-300: Final phase

But when I tried to track continuity manually, I got this result:

  • Spreadsheets became unmanageable
  • Story bibles got out of sync
  • Character descriptions contradicted themselves
  • Timeline errors slipped through
  • AI prose patterns repeated constantly

The Solution

I built an automated continuity system called “Galleys” that cross-references every chapter against your story bible and all preceding chapters.

Here’s the system architecture:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Story Bible   │ → │  Galleys        │ → │  New Chapter    │
│ (Character/     │    │  Cross-Ref      │    │  Validation     │
│  Setting Facts)  │    │  System        │    │  Engine         │
└─────────────────┘    └─────────────────┘    └─────────────────┘
       │                       │
       └───────────────────────┘

The Galleys system does these things:

  • parse_story_bible: Extracts character and setting facts
  • index_chapters: Creates searchable database of previous content
  • cross_check: Validates new content against established facts
  • flag_issues: Reports specific inconsistencies with chapter references

Implementation

Here’s my continuity checking workflow:

Terminal window
# Extract character facts from story bible
grep -E "(Name|Age|Height|Eye color|Hair color)" story_bible.md > character_facts.txt


# Find all mentions of specific character across chapters
grep -r "Elena" chapters/ --include="*.md" -n > elena_mentions.txt


# Check for inconsistent descriptions
grep -E "(Elena.*blue eyes|Elena.*green eyes|Elena.*brown eyes)" chapters/ --include="*.md" -n

When I run this:

Terminal window
$ grep -E "(Elena.*blue eyes|Elena.*green eyes|Elena.*brown eyes)" chapters/ --include="*.md" -n
chapters/045.md:234: Elena opened her piercing blue eyes.
chapters/067.md:89: Elena's green eyes scanned the room.
chapters/112.md:45: Elena's deep blue eyes filled with tears.

I get this output:

Inconsistency found: Elena's eye color changes
Chapter 045: blue eyes
Chapter 067: green eyes
Chapter 112: blue eyes

How It Works

The key insight is treating your manuscript like a codebase. I use Claude Code for grep operations and programmatic consistency checks.

Here’s a practical example using pattern recognition:

Terminal window
# Find repetitive LLM patterns in the manuscript
grep -E "(He looked at her|She looked at him|The room was small)" chapters/ --include="*.md" -c | sort -nr

When I run this:

Terminal window
$ grep -E "(He looked at her|She looked at him|The room was small)" chapters/ --include="*.md" -c | sort -nr
chapters/032.md:15
chapters/087.md:12
chapters/156.md:10

I get patterns that reveal repetitive AI writing behavior. These patterns aren’t immediately obvious but degrade reader experience over time.

The Codebase Approach

I treat my novel development like software engineering:

Terminal window
# Version control all manuscript changes
git add chapters/
git commit -m "Update chapter 237: fight scene refinement"


# Branch for continuity fixes
git checkout -b fix-elenas-eye-color

This approach lets me:

  • Track changes systematically
  • Roll back problematic chapters
  • Use parallel agents for batch processing
  • Implement automated testing for continuity

Common Mistakes

I tried manual tracking first:

  • Spreadsheets with character sheets
  • Manual cross-referencing in documents
  • Checking only recent chapters instead of full manuscript
  • Ignoring subtle LLM writing patterns

These approaches failed because:

  • Spreadsheets can’t handle 300k words
  • Manual checking is error-prone and slow
  • Recent chapters miss historical inconsistencies
  • Subtle patterns require programmatic detection

Why This Matters

The Galleys system transformed my AI writing from chaotic to可控. Before implementing it, I spent 30% of my time fixing continuity errors. Now I catch 95% of issues before they’re written.

Automated continuity checking is essential for:

  • Maintaining reader immersion
  • Preserving character authenticity
  • Ensuring timeline consistency
  • Identifying AI pattern repetition
  • Scaling to novel-length manuscripts

Summary

In this post, I showed how to build automated continuity checking for long-form AI writing. The key point is that treating your manuscript like a codebase and implementing systematic cross-referencing prevents consistency errors from compromising your story.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: