A coding agent reads its instruction file, the CLAUDE.md or the AGENTS.md, at the start of every task. If a model wrote that file, every task costs over 20% more on average and the agent resolves about as many.
The agent file is written with what the agent cannot work out from the code. Everything else makes it take more steps and spend more tokens without helping it resolve the task. A good file comes more from what the team removes than from what it adds.
What the ETH Zurich study measured
Thibaud Gloaguen, Niels Mündler-Sasahara and Martin Vechev, from ETH Zurich, with Mark Niklas Müller and Veselin Raychev, from LogicStar.ai, measured what happens when an agent works with and without an instruction file. They tested Claude Code with Sonnet 4.5, Codex with GPT-5.2 and with GPT-5.1 mini, and Qwen Code. They used 300 tasks from SWE-bench Lite, from 11 repositories without an instruction file, and 138 from a benchmark of their own, drawn from 12 repositories that already had a developer-written file.
With a model-generated file, resolved tasks fell on average by 0.5% on SWE-bench Lite and by 2% on their own benchmark, and cost rose by 20% and 23%. With the developer-written file, on their own benchmark, resolved tasks rose by 2.4% on average and cost by up to 19%. None of those differences in success is statistically significant. The one that is: files written by people resolved 7% more on average than generated ones.
The agent obeys what it reads. When the file mentioned the uv tool, the agent used it 1.6 times per task; when it did not, almost never. With the file, it also ran more tests and searched and read more files. And the repository overview, present in eight of the twelve developer files, did not get it to the file it had to change any sooner. When all documentation was removed from the repository, the generated files did help: what they contributed was a copy of what was already written.
Limit These are benchmark tasks, all in Python, and the study only measures whether the task is resolved: it leaves code security and efficiency out. The differences in success are small. The most solid findings are the cost and the fact that the agent follows every line.
What stays in the file
The authors are specific: a file written by people should only include the instructions the agent needs that are not already in the README, such as specific conventions or non-functional requirements. With that, what stays falls into three groups:
- The repository's own commands, above all the unusual ones: what installs the dependencies, which tests run before each change.
- Conventions the code does not reveal: which time zone dates are stored in, which API version is frozen.
- Requirements you cannot see in the code: a performance limit, a table that does not accept queries without an index.
Out go the folder map, the product description, whatever the README already says and the style rules the linter already enforces. So do instructions for a single task: GitHub, in the example in its documentation, asks that repository instructions “must not be task specific” and stay within two pages. That same example asks for a summary of the repository and its structure, exactly what the study finds unhelpful. The authors say it plainly: their recommendation runs against what agent developers advise.
A third thing stays out too: rules that can never be skipped. If a rule carries risk, it goes into permissions or an automated check, for the reasons I set out in where each rule goes in Claude Code.
How to prune the file, line by line
One way to do it with the file your team already has:
- For each line, ask whether the agent could work it out from the code, the README or the linter configuration. If it could, delete it.
- If the line only matters for one task, move it to that task's specification.
- If it is a rule that can never be skipped, like not touching migrations or not reading secrets, move it to a permission or an automated check.
- Review what is left in a pull request, the proposed change the team reviews before merging it, like any other change to the repository.
- Run the same handful of tasks before and after, and compare steps and cost. The authors ask for any change to the file to be evaluated before it is taken as good.
This is how a file might look after pruning. It is a made-up example, from no real team:
# Commands
- Dependencies with uv: `uv sync`. Do not use pip.
- Before each commit: `make test-fast`. The full suite only runs in CI.
# Conventions that are not the usual ones
- Dates are stored in UTC and shown in Europe/Madrid.
- New endpoints go under /v2. /v1 is frozen.
# Requirements you cannot see in the code
- No new query on orders without an index: the table has 40 million rows.What came out of that file: three paragraphs about the product, the folder tree and twelve style rules the linter was already checking.
Where to start this week
Open your team's instruction file and count how many lines pass the first step. If few do, the file is working against the agent. To find out which line is missing, look at what your team corrects in the agent every day (what those corrections teach).
If you want to review it with someone from outside, the diagnostic is free. Before that, you can see how we work with development teams in AI-Accelerated Development.
Frequently asked questions
What is AGENTS.md and how is it different from CLAUDE.md?
Both are instruction files that a coding agent reads when it starts working in a repository. CLAUDE.md is the one Claude Code reads. AGENTS.md is a name several agents accept: GitHub Copilot, for example, reads AGENTS.md and accepts CLAUDE.md or GEMINI.md at the root as an alternative. What is worth putting inside is the same for all of them.
Should the agent itself generate the file?
No. In the ETH Zurich study, files generated by a model did not improve the tasks resolved and raised cost by over 20% on average. The authors recommend not using them for the time being and writing the file by hand, with only what is not already in the README.
What should a CLAUDE.md or AGENTS.md include?
What the agent cannot work out by reading the code or the README: for example, that dependencies are installed with uv, or a performance limit that cannot be seen in the code.
Does a repository overview in the file help?
In the study, it did not. Eight of the twelve developer-written files had a codebase overview, and with them the agent did not reach the file it had to change any sooner. That overview can go.
How long should the instruction file be?
The study gives no number. GitHub, in the example in its documentation, asks for instructions of no more than two pages that are not specific to one task. A practical test: every line has to answer something the agent cannot find out on its own.
Does the agent follow what the file says?
Yes. In the study, when the file mentioned the uv tool, the agent used it 1.6 times per task on average; when it did not, almost never. That is why every line counts, including the ones that should not be there. Anything that must always hold is better moved to permissions or automated checks.
Sources
- Thibaud Gloaguen, Niels Mündler-Sasahara, Mark Niklas Müller, Veselin Raychev and Martin Vechev, “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?”, arXiv:2602.11988, v3 of 29 September 2026 (v1 of 12 February 2026).
- GitHub Docs, “Adding repository custom instructions for GitHub Copilot” (undated; accessed 4 November 2026).
Bernat López is founder and CEO of onext, an AI boutique. He helps development and product teams work with AI with method —specification before building, a person who decides where there is risk and Spec-Driven Development— and applies to his own company what he proposes: onext runs on its own agentic system.
LinkedIn →