How to Handle AI-Generated Pull Requests in Open Source Projects
Problem
I maintain an open source project, and recently I’ve been overwhelmed by a flood of pull requests. After looking at the numbers, I found something shocking: 136 PRs in just 15 days, and 97 of them (71%) were AI-generated “slop”.
These aren’t helpful contributions. They’re low-quality, copy-pasted code from ChatGPT or Claude, submitted by people who haven’t spent any time understanding the codebase.
Here’s what I’m seeing:
PR #1423: "Fix bug in auth" - 2 lines changed, no tests, no linked issuePR #1424: "Fix bug in auth" - identical to #1423, submitted 3 minutes laterPR #1425: "Fix bug in auth" - same fix again, different authorPR #1426: "Add feature X" - breaks existing tests, author doesn't respond...The result? I’m spending more time filtering noise than building features. It’s pulling me away from actually making the framework better.
Environment
- GitHub-hosted open source project
- Active contributor community
- Increasing AI-generated contributions in 2024-2025
- Maintainer burnout risk
What Happened?
The rise of AI coding assistants created an unintended consequence. Developers can now generate pull requests at unprecedented speed—but often without the deep understanding required for quality contributions.
I noticed these patterns:
- Generic fixes that miss project conventions - The code works, but doesn’t match how the project does things
- PRs within minutes of issue creation - Impossible for human review and testing
- Multiple duplicate PRs for the same issue - Contributors racing to be first
- Solutions for symptoms, not root causes - AI patches the error, doesn’t fix the problem
- Missing tests and documentation - The PR adds code but no verification
How to Solve It?
I looked at what other projects are doing and came up with a multi-layered defense strategy.
Layer 1: Update CONTRIBUTING.md
I added clear contribution guidelines based on the Ghostty project’s approach:
## Before You Contribute
1. **Check for existing issues** - Search open issues before creating new ones2. **Link your PR** - All PRs must reference an existing issue3. **Wait for triage** - Don't start work until maintainers approve the approach4. **Disclose AI assistance** - If you used AI tools, describe how in your PR
## PR Requirements
- [ ] Linked to an approved issue- [ ] Includes tests for new functionality- [ ] Updates documentation if needed- [ ] Passes all CI checks- [ ] No unrelated changesLayer 2: Require Issue Linkage
I created a GitHub Action that checks for issue references:
name: PR Validation
on: pull_request: types: [opened, edited, synchronize]
jobs: check-issue-link: runs-on: ubuntu-latest steps: - name: Check for issue reference env: PR_BODY: ${{ github.event.pull_request.body }} PR_TITLE: ${{ github.event.pull_request.title }} run: | # Check for issue reference pattern if ! echo "$PR_BODY $PR_TITLE" | grep -qE "(fixes|closes|references|ref) #[0-9]+"; then echo "ERROR: PR must reference an existing issue" echo "Please add 'Fixes #<issue-number>' to your PR description" exit 1 fiNow PRs without linked issues fail automatically. This single rule filtered out 40% of the AI slop.
Layer 3: Add AI Disclosure to CLA
I updated the Contributor License Agreement:
## AI Assistance Disclosure (Required)
[ ] I did NOT use AI coding assistants for this contribution[ ] I used AI assistance. Details: - Tool used: _______________ - How I verified the output: _______________ - What I modified from AI output: _______________
Note: AI-assisted contributions are welcome, but must be disclosed.This isn’t about banning AI. It’s about transparency. Contributors who disclose AI use honestly tend to submit better PRs because they’ve actually reviewed the output.
Layer 4: Automated Quality Gates
I added CI checks that catch common AI slop patterns:
name: Quality Gates
on: pull_request:
jobs: quality-checks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4
- name: Check for suspicious patterns run: | # Check for minimal changes (common in AI slop) CHANGED_FILES=$(git diff origin/main --name-only | wc -l) ADDED_LINES=$(git diff origin/main --shortstat | grep -oE '[0-9]+ insertion' | grep -oE '[0-9]+' || echo "0")
if [ "$CHANGED_FILES" -gt 10 ] && [ "$ADDED_LINES" -lt 20 ]; then echo "Warning: Many files changed with few additions" fi
- name: Run tests run: npm testYou can see that I succeeded in reducing the noise. PRs now come with context, tests, and accountability.
The Reason
The key reason AI-generated PRs are problematic is the time imbalance:
Contributor time: 5 minutes (prompt + copy-paste)Reviewer time: 30-60 minutes (understand + verify + feedback)Open source is volunteer-driven. When maintainers spend more time filtering noise than building value, projects die.
The solution isn’t to ban AI contributions. Some high-quality contributions use AI assistance appropriately. The goal is to ensure contributors invest meaningful time understanding the codebase before submitting changes.
Summary
In this post, I showed how to handle AI-generated pull requests using a layered defense strategy. The key points are:
- Clear guidelines set expectations upfront
- Issue linkage filters out drive-by PRs
- AI disclosure builds transparency without banning tools
- Automated gates catch common patterns
- Time investment should be balanced between contributor and reviewer
The goal is to reduce noise while preserving signal. Well-intentioned contributors using AI as a tool will happily comply. Those generating volume without understanding will self-select out.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
- 👨💻 Pydantic AI Reddit Discussion
- 👨💻 Ghostty Contributing Guidelines
- 👨💻 GitHub Actions Documentation
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments