ARCANA/MANIFESTOSoftware used to do what we told it. Now it proposes — writes the file, runs the command, calls the API, spends the budget — and asks in the half-second before acting whether anyone objects. That half-second is where the entire question of AI safety actually lives. Everything else is commentary.
An agent that acts creates a debt. Not a technical debt — an accounting debt. Someone, eventually, will have to answer for what the machine did: why it deployed, why it deleted, why it mailed the client, why it spent four hundred dollars of API credits fuzzing an endpoint nobody asked it to touch. The industry's current answer to that future question is a scrollback buffer and a prayer.
We think the answer should be evidence.
Every agent platform keeps logs, and every log shares the same fatal property: it was written by the thing under investigation. After the fact. By code whose correctness depends on the very behavior being audited. A log is testimony — sincere, perhaps, but self-reported, mutable, and structurally incapable of incriminating its author.
Proof is a different artifact. It is constructed before execution, not reconstructed after. The request is bound to an exact hash. The authorization is issued against that hash, usable once, expiring on its own. The receipt is attached to the effect it explains. Break the chain anywhere and the system says so — out loud, in red — instead of quietly filling the gap with something plausible.
When the question comes — and it always comes — testimony gives you a story. Evidence gives you a verdict.
The common safety mechanism is a mode: a toggle the human sets, asking the agent to behave differently within it. Plan mode. Auto-accept. YOLO. A mode is a promise the software makes about itself, enforced by the software's own good mood.
Governed autonomy replaces the promise with a gate. Actions do not pass through the model's judgment about right and wrong; they pass through a deterministic policy decision point that has never seen a prompt and cannot be talked to. The decision is then enforced by a separate enforcement point that holds no negotiation skills whatsoever:
Not authorized means not executed. No exception handler for eloquence. We did not write that line as documentation; we wrote ninety-five adversarial fixtures trying to break it, and the count of unauthorized executions stands at zero. The fixtures are public. Try them.
The next decade of agents is not one agent. It is agents spawning agents — researchers delegating to runners, planners delegating to executors, swarms coordinating across repositories. Which makes today's standard delegation pattern worth staring at: the subagent receives, effectively, a copy of you. Your credentials, your permissions, your authority — duplicated into a context you supervise less.
There is an older rule for this, from systems that survived their own complexity: authority must descend, never duplicate. A child capability is derived from the parent, scoped tighter than the parent, limited in uses, revocable at any moment, and traceable to the lineage that minted it. A subagent can never hold more power than the exact task it was born for. When the swarm arrives, this is the difference between an org chart and a riot.
Honesty about price is part of the discipline. Governance adds friction — deliberately, but only where consequence lives. Reading, searching, thinking: ungated, silent, fast. Writing, executing, spending, sending: decided by policy, and when policy says a human must choose, a human chooses. The friction scales with consequence, not with paranoia.
The decision itself is deterministic and local — milliseconds, on your machine, against rules you own. No committee in the cloud. No telemetry reading over your shoulder. Your keys, your terminal, your policy, your proof.
A manifesto that only claims is an advertisement. Here is what governed autonomy does not do, stated plainly enough to be held against us:
We publish our limits because a trust product that exaggerates is just a faster way to lose trust. Hold everything else we claim to the same standard.
Here is the position, whole: do not trust any agent — including ours. Trust is what you are forced to extend when evidence is unavailable. We would rather remove the need. Install the runtime, read the policy language, watch the gate refuse things, export the receipts, and check every claim on this page against the artifacts that back it.
The era of autonomous software is coming regardless of anyone's preference. The only open question is whether its actions arrive with authority attached — or with apologies afterward.
We built the first thing. The second thing is what everyone else is shipping.