5 mistakes AI coding agents make, and how to catch them before merge
AI coding agents write code fast. They also make the same handful of mistakes over and over. None of them are exotic, which is exactly why they slip through review: the diff looks plausible, the summary sounds confident, and the reviewer is busy.
Here's what to check every AI-assisted diff for before it merges.
1. Invented APIs
How to spot it: calls to functions, methods, flags or packages that don't exist in the versions you actually depend on. The name is usually reasonable, which is the problem.
What to do: search the codebase and the library's docs for every unfamiliar call. Delete or replace anything you can't find. If the tests "pass," check that they actually execute that line.
2. Swallowed errors
How to spot it: empty catch blocks, except: pass, or an error that's logged and then ignored so the function carries on with bad state.
What to do: handle the error, rethrow it, or write a comment explaining why ignoring it is safe. "It made the test pass" isn't a reason.
3. Fake tests
How to spot it: tests with no assertions, tests that only check a mock was called, or tests that mirror the implementation line by line so they can never fail.
What to do: rewrite them to assert behavior someone outside the function would notice: the return value, the saved record, the HTTP response.
4. Confident wrong summaries
How to spot it: "All tests pass" or "Fixed the bug," with no command output shown.
What to do: run the command yourself and read the output. Every time. An agent's summary of a test run isn't a test run.
5. Silent behavior changes inside "refactors"
How to spot it: a diff labeled as a refactor that also changes defaults, error messages, log lines or the order things happen in.
What to do: move behavior changes into their own commit, with a test that shows the new behavior on purpose.
The one-question test
Before merging, ask:
Would I be comfortable defending this diff in review if I had written every line myself?
If not, it isn't ready, no matter who or what wrote it.
Making it stick
A checklist only helps if it's used on every diff, not just the ones that look suspicious. Two things that help:
- Put the list in your pull request template, so the author ticks it before asking for review.
- If your agent supports it (Claude Code subagents, Cursor rules, Copilot instructions), give it the same list and have it check its own work before handing it over. It won't catch everything, but it catches the cheap mistakes before a human has to.
The full 12-point slop filter, the pre-merge checklist and a ready-to-use code-reviewer subagent for Claude Code are free on GitHub: lean-team-products/ai-code-review-gates. Copy them into your repo and you're set up in two minutes.