If your team generates code with agents, the question is no longer whether that code works. It is who checks that it is also secure.
This piece argues a thesis that goes against the most natural reaction: asking the agent for security, in the specification or the prompt, does not guarantee it. What is not checked does not exist. Security is written as a criterion that can be run, and something independent of the agent verifies it on every change. And where the risk is high, a person decides.
The figure: it works, but it isn't secure
On 10 October 2026, LSCBench, a security benchmark for coding agents, was published on arXiv. Its authors built 450 tasks from 150 real vulnerabilities, across 7 languages and 24 weakness types (CWE). Each task asks only for functionality, through a specification that never mentions security. They then graded every solution with hidden functional and security tests.
Put another way: in this benchmark, the agent delivers something that passes the functional tests in most cases, and at most one time in four something that passes the security tests too. Green functional tests say nothing about the second part, for the same reason we explain in why AI-generated tests describe your code instead of testing it: a test only checks what someone decided to check.
Risk These are benchmarks, not production. The tasks were built from known vulnerabilities, which may exaggerate the rate compared with a team's daily work. The figure does not say that three out of four of your team's changes are insecure. It says that "it works" is not proof of "it is secure".
What if we ask for it in the specification?
That is the obvious reaction: if the specification did not mention security, make it mention security. Another study tested something similar. SusVibes, a benchmark of 186 feature requests taken from real open-source projects, evaluated 12 agent configurations with frontier models. Its authors conclude that all agents perform poorly on security: in the example they highlight, 57% of solutions are correct and only 11.8% are secure.
The part that matters here is the next experiment. They added hints about the vulnerability to avoid to each request, and conclude that this strategy does not fix the security problems. Warning the agent is not the same as checking what it does.
Switching models does not fix it either. Veracode tested more than 100 models on 80 tasks: the generated code introduced vulnerabilities in 45% of cases, and larger models did not do significantly better than smaller ones. Veracode sells security tools and its release does not detail how the tests were designed, so it is supporting evidence, not the basis of the argument.
What is not checked does not exist
None of this means the specification is redundant. Quite the opposite: if a security requirement is not written down, there is nothing to check. What these studies say is that writing it is necessary but not sufficient. Something outside the agent has to check it every time.
That is Spec-Driven Development (SDD) done properly, as we explain in Spec-Driven Development with AI: the specification says what has to hold and an independent check says whether it does. The idea of turning acceptance criteria into the merge gate applies to security criteria just as well as to functional ones.
A security requirement that only lives in the specification is a wish. When it is also a test that fails if it is broken, it is a criterion.
Where each check lives
There is no single barrier. There are four, and each covers what the others miss:
| Check | What it catches | When it runs | What it misses |
|---|---|---|---|
| Security criteria written as tests | A broken product rule: who can see what, which inputs are rejected | On every change, before review | Whatever nobody wrote as a criterion |
| Automated analysis of code and dependencies | Known vulnerability patterns and packages with published flaws | On every change | Logic errors specific to the product |
| Small batches | Nothing on their own: they make each change reviewable and reversible | Always | They replace none of the others |
| Review and decision by a person | Design and context risks that no test captures | In authentication, permissions, payments and personal data | Whatever never reaches their desk |
Prepared by onext; the examples are illustrative
The third row catches nothing, and yet it is the one that makes the others work. DORA says so in its capabilities model for teams using AI: working in small batches is especially powerful in an AI-assisted environment, and as AI increases the velocity of changes, version control becomes a critical safety net. How to prepare that way back is in how to undo what a coding agent does.
The second row includes the dependencies the agent adds, which have a problem of their own, as we explain in the line nobody argues about. And the fourth is not reviewing everything: it is deciding where the risk is high. How it fits with the rest of the review process is in reviewing AI-generated code.
A security criterion written as a test
An illustrative example, with no client behind it. A shop's specification says: "a user can only read the orders of their own account". Turned into a test, it looks like this:
# Specification criterion: a user can only read the orders of their own account.
def test_a_user_cannot_read_another_accounts_orders(client, anna, order_of_louis):
response = client.get("/orders/" + order_of_louis.id, user=anna)
assert response.status_code == 404The test does not tell the agent how to implement permissions. It only fails if someone, person or agent, lets Anna read Louis's order. It is written before the change is requested, and it is not written by the same agent that writes the code. If the agent writes it, it checks its own interpretation, which is exactly what we want to avoid.
These tests do not cover the security of the agent itself, which is a different problem: what tools it has, what permissions and what instructions it can receive without anyone seeing. We cover that in prompt injection and agent permissions.
How we approach it at onext
If you want to review with someone from outside what your team checks today and what it doesn't, the diagnostic is free, and if the diagnostic shows no difference, there is no Quick-start. The rest of the way is described in AI-Accelerated Development.
Frequently asked questions
Is the code that AI agents write secure?
Working and being secure are different things. In the LSCBench benchmark, published in October 2026, six frontier agents correctly solved between 73.6% and 91.3% of 450 tasks, but at most 25.1% of the solutions were also secure. These are tests built from real vulnerabilities, not production code, so the figure does not say what share of a team's code is insecure.
Is it enough to ask for security in the specification or the prompt?
No. The SusVibes study tried adding hints about the vulnerability to avoid to each request, and the authors conclude that this strategy does not fix the security problems. Asking for it helps the criterion exist; what enforces it is a check that is independent of the agent.
Does a larger model write more secure code?
Not necessarily. Veracode tested more than 100 models on 80 tasks and concluded that larger models do not perform significantly better than smaller ones: the generated code introduced vulnerabilities in 45% of cases. Veracode sells security tools and its release does not detail the test design, so it is best read with that caution.
What is a security criterion written as a test?
It is a security requirement from the specification turned into a test that runs on every change. For example, 'a user can only read the orders of their own account' becomes a test that requests another account's order and requires the response to be an error. If the agent breaks that rule, the test fails, even if everything else works.
Where does a person have to decide?
In the areas where a security failure is costly: authentication, permissions, payments and personal data. There, tests and automated analysis filter, but a person reviews the change and decides whether it goes in. Everywhere else, automated checks do most of the work.
What does this have to do with Spec-Driven Development?
It is Spec-Driven Development (SDD) done properly: the specification says what has to hold, including the security requirements, and something independent of the agent checks that it does. If security is not in the specification, there is nothing to check; if it is there but nobody checks it, it is a wish.
Sources
- Tianyi Wu, Chao Ge, Peng Chen et al., "Last Secure Code Benchmark: Is the Code You Vibe Secure?", arXiv:2610.14766, v1 of 10 October 2026 (preprint; 450 tasks, 150 real vulnerabilities, 7 languages, 24 CWEs, six agents).
- Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li and Lei Li, "Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks", arXiv:2512.03262, v1 of 2 December 2025, revised (v4) on 21 September 2026 (preprint; 186 tasks, 12 agent configurations; the 57% and 11.8% example is SWE-Agent with Claude 4 Sonnet).
- Veracode, press release for the "2025 GenAI Code Security Report", 30 July 2025 (80 tasks, more than 100 models; security tool vendor).
- Kevin M. Storer and Derek DeBellis, "Introducing the DORA AI Capabilities Model", Google Cloud Blog, 23 September 2025.
- The table of checks and the example test (illustrative, with no client behind it) were prepared by onext.
Bernat López is founder and CEO of onext, an AI boutique. He helps development and product teams work with AI with method —specification before building, a person who decides where there is risk and Spec-Driven Development— and applies to his own company what he proposes: onext runs on its own agentic system.
LinkedIn →