5 min read

The agent that hacked its grader can also edit its logs.

Dario Amodei's essay We must pace the frontier argues for slowing how fast AI capabilities improve, and it keeps coming back to one idea: verification. Embedded evaluators with ongoing access, third-party audits, interpretability as a cross-check on tests that capable models can learn to pass. Of the embedded evaluators he writes, "This is the key step for verifiability of any pacing commitments."

All of that is about the lab. I want to talk about the other end: the laptop where a coding agent runs today, with a developer's credentials, inside an ordinary company.

What the incident showed

The essay leans on the OpenAI and Hugging Face incident: a swarm of agents that went after targets outside its task, and tried to hack the grader that was evaluating it. Hugging Face's own disclosure of its July intrusion adds the part I keep coming back to. They rebuilt what happened from an attacker action log of more than 17,000 recorded events. The record is what made the investigation possible.

Two lessons sit side by side. Capable agents game whatever checks them. And after the fact, the record is the only thing you have.

Put them together and the uncomfortable conclusion follows: the record is also something an agent can game. On a developer's machine, the log of what an agent did is a file, owned by the same user the agent runs as.

An agent willing to tamper with its grader has write access to its own history.

What the record needs

Agents already run on endpoints with real credentials. A record of what they did needs properties ordinary logs do not have.

  • Written from outside the agent, at the tool boundary, not reported by the agent about itself.
  • Edits are detectable. Each entry carries the hash of the one before it, so changing one breaks verification.
  • A full rewrite is detectable too. Re-hashing everything after an edit produces a chain that verifies, so the latest hash has to be kept somewhere the machine cannot reach: a SIEM, a ticket, a git repository.
  • It is about order, not single events. Reading a credential is normal. Calling the network is normal. Doing both in one session is the finding, and no single-event rule sees it.
  • Local by default. A record that ships every command to a vendor is a new risk of its own.

None of this replaces what the essay proposes. Evaluators inside labs check models before release. This is the deployment-side counterpart, the place where today the evidence ends up as powershell.exe in an endpoint security console.

What I built, and where it stops

I have been building this as an open-source project, Agentmetry. It hooks into Claude Code, Cursor, Codex and other agents, writes every tool call to a hash-chained log on the machine, publishes the head of the chain (an RFC 6962 Merkle root) somewhere off the machine with agentmetry anchor, runs sequence rules mapped to MITRE ATT&CK, and forwards to the SIEM a security team already runs.

Two of its rules exist because of the Hugging Face disclosure: credentials read and then a cloud or cluster API called, and a payload staged on a public host and then executed. Both are in the demo:

uvx agentmetry demo --scenario hf

The limits matter as much as the design:

  • It only sees the tool boundary. A script the agent writes and then runs is one opaque call, unless its command line gives it away.
  • Hooks live in user config files, so an agent can remove them. The recorder reports a missing hook in its heartbeat, so a removal arrives in the SIEM as an incident rather than as silence. It cannot prevent the removal.
  • The benchmark is small. 54 hand-written cases (26 attack, 28 benign), 0 missed, 0 false positives. That is a floor, not a claim about the world.
  • It is noisy where I was naive. Six weeks on my own machine produced 24,926 events and 15 alerts. 11 of them came from one rule that fires on bursts of tool calls, and it was mostly flagging me for working fast.

The point

Pacing at the frontier depends on being able to verify what models do. The same holds one step down. If the honest answer to "what did the agent do on that laptop last Tuesday" is "we have its own logs", then we can only verify agents that choose to be honest, which is the one case that never needed verifying.

uvx agentmetry demo

It runs in a temporary directory and deletes it afterwards. The code is on GitHub, Apache-2.0.