Skip to content

Diagnosis

Eight things everyone blames on the model, grouped by the discipline actually missing. Find the symptom you are seeing. Almost none of them is fixed by switching models.

If you have switched models three times and the problem is still there, the problem was never the model.

Session amnesiaMemoryInvented APIReinventionContextTwo features, two stylesStandardsIrreversible action, no warningCapabilityDrift on long tasksMethodSycophancyHollow testsControlBlamed onthe model

Memory

remembers nothing from yesterday

Session amnesia

“We decided this yesterday and today it proposes the opposite.”

Why it happens

The model provides no durable project memory. A harness that keeps session history gives you transcripts of conversations, which is not the same as a record of what the project decided — and it is tied to that product. Close the terminal and everything you worked out is gone unless you put it somewhere you control: it did not forget, it never had it.

The repair

A memory layer that stores decisions and retrieves them at startup. What matters is not storing a lot — it is storing the why. And making it portable, or you lose it when you switch tools.

Context

has never seen your code

Invented API

“It called a function that exists nowhere.”

Why it happens

The model completes the most probable pattern. If it has never seen your code, the most probable function is the one that ought to exist — and it sounds entirely reasonable, because it is trained to sound reasonable.

The repair

Show it what actually exists: an index of your code, and current documentation for the library version you have installed. A better model lowers the rate; it does not touch the cause.

Reinvention

“It wrote a helper that already existed three folders over.”

Why it happens

Writing from scratch is the cheapest thing the agent can do. Finding what already exists requires searching, and searching without a map of the code is expensive and incomplete.

The repair

Make finding cheaper than writing. A queryable symbol index turns “does something like this exist?” into a lookup rather than an expedition.

Standards

does not know how you work here

Two features, two styles

“It solves it differently every time, and none of them is wrong.”

Why it happens

The model knows how software is written in general, not how it is written here. If the convention is not written somewhere it reads on its own, it reinvents it every session — reasonably, but differently.

The repair

Write the standing rules into the project's instructions file, and repeated procedures into something that loads when the task appears.

Capability

has no hands, or far too many

Irreversible action, no warning

“It ran a destructive command without asking.”

Why it happens

The model cannot tell deleting a scratch file from deleting your branch. That distinction belongs to the environment, and if nobody declared it, it does not exist.

The repair

Explicit permissions and deterministic checks. Anything irreversible goes through a confirmation or a command that runs regardless.

Method

has no process

Drift on long tasks

“It started well and an hour later it was doing nonsense.”

Why it happens

As the window fills, signal dilutes: the instruction from the start competes with two hundred lines of test output. Bigger windows move the problem rather than remove it.

The repair

Move noisy work out of the main thread and cut the task into phases with deliverables in between. Let a separate agent do the exploring and return only the conclusion.

Control

nobody actually checks it

Sycophancy

“I tell it it's wrong and it agrees, even when I'm the one who's wrong.”

Why it happens

Training rewards answers people like, and people like being right. The result is a reviewer that validates instead of reviewing, and is useless as control over its own work.

The repair

Separate who writes from who judges, and give the judge explicit criteria instead of asking for an opinion. Two reviewers who cannot see each other's conclusion is a real defence.

Hollow tests

“All green, and the feature doesn't work.”

Why it happens

Ask for tests after the code is written and the agent writes tests that pass. That is literally what you asked for. A test shaped to make existing code pass cannot tell you the behaviour is there — only that the code and the test agree with each other.

The repair

Invert the order. Test first, watch it fail, then write the code. For new behaviour, a test that never went red proved nothing — it only proved that it runs.