A practical way to judge AI-written code beyond a green build.
An AI coding agent can give you a working feature and a maintenance problem in the same commit. Both can pass the build.
Imagine asking Claude Code to add bookmarks to an Android app. Users should be able to save an item, remove it, and find it again after closing the app.
The agent produces a screen, a ViewModel, a repository, and some tests. The app opens, the save button works, and the tests turn green.
Looks done.
Then you notice the app already had a repository for that data. The agent has created another one, with slightly different rules.
Or you close the app, reopen it, and discover that your bookmarks have vanished.
These are imagined examples, but they expose the question I care about as a mobile engineer: can an AI agent add a feature without making the next feature harder to build?
Generating code is only part of that job.
Give the agent a map before asking it to build
A feature request describes what should change. It rarely explains everything that should stay consistent.
“Add bookmarks” doesn’t tell the agent which existing component owns saved data. It doesn’t explain where business rules belong or how the team handles errors.
Those details live in the codebase, team decisions, and sometimes one developer’s head.
A new engineer would need time to learn them. An agent needs access to them too.
Before making changes, I’d ask it to find the closest existing feature and explain how it works. Where is data stored? How does a change reach the screen? Which parts can be reused?
That small step gives me something useful to review before a large diff appears.
I’d also keep a short set of project rules:
- Check existing components before creating new ones.
- Keep business logic out of the UI.
- Follow the project’s current state and error-handling patterns.
- Explain any new dependency or change to module boundaries.
- Leave unrelated code alone.
The aim isn’t to write a rule for every possible mistake. It’s to make the important decisions visible.
A reusable Claude Code skill (a saved set of instructions the agent loads when needed) can package that process so it doesn’t have to be explained from scratch for each feature. But the value is in the questions it makes the agent answer, not the length of the instructions.
A hundred lines of vague advice won’t beat five clear rules tied to the actual app.
And written rules still need checks. An instruction to reuse existing code doesn’t prove that reuse happened.
Passing tests can protect the wrong behaviour
Here’s the failure that worries me most.
An agent decides to store bookmarks in an in-memory list instead of Room or DataStore. It then writes tests that add an item to that collection and check that the item appears.
The tests pass because the collection works.
But the user wanted bookmarks that survive an app restart. The tests never asked about that.
The code and the tests agree. They’re both wrong about the requirement.
This can happen without AI too. A developer gets attached to an implementation, then tests the choices they made instead of the outcome the user needs.
The way out is to define important behaviour before writing the feature.
For bookmarks, I’d start here:
Save an item. Terminate the app. Launch it again. The saved item must still appear.
Then I’d check removal:
Remove a saved item. Relaunch the app. The item must remain removed.
These checks are plain enough for a product manager to understand. They also make it harder to mistake a working in-memory list for durable storage.
Different tests answer different questions. A ViewModel test can check what state reaches the UI; it doesn’t, by itself, prove that the real database saves data correctly.
The evidence has to match the claim.
A separate reviewer may catch a missing scenario, whether that reviewer is a person or another agent. But adding agents doesn’t automatically solve the problem.
If every reviewer starts with the same wrong assumption, they may all approve the same mistake.
Give the reviewer the original requirement and ask what evidence is missing. Agreement alone isn’t evidence.
Measure the work until it’s ready to merge
A feature generated in five minutes sounds impressive. It sounds less impressive if someone spends two hours removing duplicate code and repairing the tests.
Both periods belong on the clock.
That’s why I want to compare two workflows using the same small Android feature.
In one, the agent gets the repository and a normal feature request. In the other, it also gets project guidance, a reusable implementation process, and explicit review checkpoints.
Both should start from the same commit, use the same model and requirements, and have the same time limit. I’d repeat the runs and keep the failures in the results.
This is an experiment I want to run, not a benchmark I’ve completed.
I’d judge each run by five things:
- Required behaviour: Does the feature do what the user asked?
- Architecture: Does it fit the app’s existing structure?
- Human corrections: How much repair work remains?
- Total time: How long until the feature is reviewed and verified?
- Cost: What did the model and tools cost to reach that point?
The structured approach might produce fewer mistakes. It might also add work without enough benefit.
Either result is worth knowing.
I’d be especially careful not to compare an unreviewed first draft with a fully checked implementation. They aren’t the same stage of work.
Try this on your next small feature
You don’t need a fleet of agents to start. Pick one contained change and write down three observable behaviours before handing it over.
Then give the agent this instruction:
Before editing, inspect the closest existing feature. Explain what you will reuse, what you need to change, and how you will verify each requirement.
After implementation, report which checks passed, which failed, and which behaviours remain unverified.
Read the plan. Review the diff. Check whether the report supports the claim that the feature is done.
“Unit tests passed; restart behaviour wasn’t checked on a device” is a useful report. It tells you exactly where confidence ends.
“Everything works” hides that boundary.
What’s the most convincing-looking mistake an AI coding agent has made in your project? Drop it in the comments, especially if the tests passed.
I still want AI tools to help me ship faster. But I want to count the time spent understanding, checking, and repairing what they produce.
The next developer inherits your code, not your demo.





