General
I tried to forge my own AI agent's audit log
oleg-vdv DEV Community 周榜
2 views
Half the organisations running AI agents have already had one step outside its
permissions. Nearly half of the agents in production have no monitoring at all.
Only about a fifth treat an agent as something with an identity of its own.
Those are numbers from the Cloud Security Alliance and the 2026 State of AI
Agent Security report, and they describe the same gap from three directions:
agents act, and nobody can say afterwards what exactly they did.
I spent a few weeks building the missing piece, and then spent an afternoon
attacking it. This is what survived.
Four things an accountable agent needs
A passport. Not a config file — a card for one version of an agent that
says who built it, who runs it, what it does, and what it will never do:
{
"agent_id": "kepil.leads.v1",
"purpose": "Handle inbound requests and hand qualified ones to a person",
"does_not": [
"never promises prices or deadlines on the company's behalf",
"never sends invoices or contracts",
"never exports the customer base outside the perimeter"
],
"risk_class": "medium",
"autonomy_class": "medium",
"risk_review": { "last": "2026-09-14", "next_due": "2027-09-14" }
}
(Trimmed for readability — a real card also carries the operator, the model list,
the rationale for each class, an incident log and the passport's own hash. One
caveat before you install: the runtime, the panel and these field values are
currently in Russian. The format and the code are not language-specific, but you
will be reading «средний» where this article says "medium".)
A new version is a new card. The old one is kept forever, because the question
"what was this agent allowed to do in March" has to have an answer.
A mandate. A machine-readable power of attorney for one job — not a
permanent grant:
{
"mandate_id": "mnd-0001",
"allowed_actions": ["read:inbox", "read:crm", "generate:reply", "send:message",
"write:crm", "generate:summary", "send:handoff"],
"allowed_systems": ["crm.local", "whatsapp.local"],
"forbidden_actions": ["sign:*", "pay:*", "export:database", "send:bulk"],
"human_confirmation_required": ["send:*", "write:crm", "publish:*"],
"limits": { "messages": 300, "llm_cost_kzt": 4000 },
"valid_until": "2026-09-21T10:09:45"
}
Anything not explicitly allowed is refused. That sounds obvious until you look
at how agents are actually wired: one shared API key, full network access, and
a prompt asking nicely.
A gate. One place an agent touches the outside world, and every action is
checked before a model is even called. The checks run in a fixed order:
passport active, mandate valid, action allowed, system allowed, limits not
spent, irreversible or not. Any error inside the check means refusal — never a
pass. A false refusal costs a retry; a false pass costs a breach notification.
A journal. Append-only JSONL where each record carries the hash of the one
before it:
{"seq":10241,"ts":"2026-09-12T11:42:07+05:00","agent_id":"kepil.leads.v1",
"action":{"type":"send:message","target":"whatsapp.local"},
"decision":"await_human","cost_kzt":0,
"prev_hash":"sha256:c61d8b…","hash":"sha256:d5ade5…"}
Values of personal data never enter it. Types, counts and hashes do.
The part where I attacked it
A log you cannot verify is a diary. So I wrote a record, then went back and
edited it the way someone covering their tracks would — flipping a refusal into
an approval:
# the agent tried to sign a document; the gate refused
$ npx proofbyte-agent-trace verify journal.jsonl
Integrity confirmed. Records: 4
Chain head: sha256:d58c204ff2a37b093307b95b3d9fcc4edc510cd6062622db10b3461a4341dd3f
$ sed -i 's/"decision": "deny"/"decision": "allow"/' journal.jsonl
$ npx proofbyte-agent-trace verify journal.jsonl
Integrity BROKEN. Record 3: record content changed after it was written
Two things matter here.
The first is that it names the record, not just the file. Hash chaining gives
you that for free, and it is the difference between "something is wrong" and
"here is what was changed".
The second is subtler. The verifier is a different program in a different
language, written by nobody in particular. The writer is Python; the verifier
is TypeScript on npm. Proof that only its own author can check is not proof —
so the test suite of the verifier contains a journal produced by the Python
writer, and that test pins a bug I would otherwise have shipped: Python
serialises 0.0 as "0.0" and JavaScript as "0", so the same record hashed
differently in the two implementations. An honest log would have been declared
forged. Number canonicalisation is now part of the format.
The feature I deliberately did not build
Kepil ships an MCP server, a JSON API and an n8n node. All three can create
orders, run steps, ask whether an action is permitted, and read the journal.
None of them can confirm.
If a model could call confirm, the human would drop out of the chain: the
agent would be approving its own irreversible actions, and every guarantee
above would become decoration. So the confirmation card goes to a person — the
panel, or Telegram with two buttons — and there is a test whose only job is to
fail if confirm ever appears in the tool list.
I find this the most useful thing to say about the whole project. Everyone
advertises what their agent framework can do. The interesting part is what it
refuses to do even when you ask it nicely.
Undo
Recording is table stakes. Agent-governance products — Workday's Agent
Passport, Okta, Credo AI — all write logs. None of them put anything back.
Because the journal is a graph of actions and every profession declares its
compensating action, you can walk it backwards:
"rollback": {
"send:message": "send a correcting message",
"write:crm": "restore the previous state of the deal from the journal",
"publish:complaint": ""
}
The empty string is the important entry. It means this cannot be undone.
Pick a window in the panel — the last hour, say — and the pass runs from the
most recent action backwards, applying each compensating action in turn, and
stops at the first step whose compensation is that empty string. It then reports
both halves: what it undid, and what it could not. The panel shows the same
thing before you press the button, naming the step where the pass will stop:
Will undo 2 actions. The pass will stop at "Search for tailored requirements" —
that action is irreversible, and nothing before it will be undone.
An undo promise that quietly fails is worse than no undo at all, so there is a
test whose only job is to fail if the preview and the engine ever disagree.
One more thing follows from this. A rollback is written to the journal as an
operator's decision, so neither the MCP server nor the JSON API can perform
one — same reasoning as confirm. An agent undoing its own actions would be
signing in somebody else's name.
Boring on purpose
The core has zero dependencies. Not "few" — zero: it is the Python 3.11+
standard library, and CI fails the build if a third-party import appears. Two
reasons. It installs inside an air-gapped perimeter where pip install from
the internet is not an option. And a tool that sees every action an agent takes
should not drag a hundred transitive packages behind it.
State is JSON files. No database. You can open them, read them, and attach them
to a dispute — which is the whole point of the exercise.
Status
Alpha, 115 tests, AGPL-3.0. It does not run agents for you: it is the layer
that says what they may do and records what they did.
pip install kepil
python -m kepil.admin
Core: https://github.com/oleg-vdv/kepil
Verifier (MIT): https://github.com/oleg-vdv/agent-trace
n8n node: https://github.com/oleg-vdv/n8n-nodes-kepil
If you are running agents in production and can answer "what did agent X do on
9 September, and who allowed it" — I would genuinely like to know how. If you
cannot, that is the gap this was built for.
Read original: https://dev.to/olegvdv/i-tried-to-forge-my-own-ai-agents-audit-log-3chh
← Previous
Stop Paying CapCut Nights for Structure You Could Compile Once
Next →
I logged into 12 freelance platforms from India and counted what is actually open to us
Related
An HTTP 200 is not a delivered job application
General
0
DEV Community 周榜
Talk to your PWA and have it talk back — Speech Recognition & Synthesis (FieldKit companion)
General
0
DEV Community 周榜
The State Snapshot: Your First Week on a New Project
General
0
DEV Community 周榜
Here’s How I Get the Most Out of My CLAUDE.md
General
0
DEV Community 周榜
Comments0
No comments yet — be the first