Deterministic outcomes from probabilistic agents

Ask a language model to apply a rule written in prose and it will eventually let something through. "This change is high consequence, but it is so unlikely that it is fine." Reasonable, persuasive, and exactly the judgement the rule exists to forbid. A lookup table cannot be persuaded. That difference answers the question every team putting agents into production eventually asks: how do you get deterministic outcomes from a probabilistic system?

You do not, and you do not need to. Stop trying to make the agent deterministic; make the acceptance deterministic instead. The agent proposes. A deliberately boring symbolic layer decides what becomes real. I have built this pattern three times: an AI brain for a martech scale-up, a product knowledge system for a construction marketplace, and most recently the delivery workflow at a company that runs its engineering under an ISO 27001 risk methodology, where the product is the data itself and the tolerance for a confidently wrong answer is zero.

Five big words, five plain questions

Each term in the semantic stack answers one question. A taxonomy answers: what kind of thing is this? One relationship, "is a", arranged in a tree. A semantic layer answers: what do we mean when we say X? Definitions precise enough that two people, or an agent, cannot read them differently. An ontology answers: what kinds of things exist, and how can they relate? The schema: the entities, the relationships, and the rules that bind them. A knowledge graph answers: what is actually true right now? The ontology filled in with instances, schema versus rows. And a context graph answers: what does the agent need for this one decision? Not a store but a slice, cut at retrieval time, because a context window cannot hold the whole graph. The other four layers exist so this slice can be cut cleanly.

A lookup table and a list of refusals

In our system there is no OWL file anywhere. The ontology lives in two humbler places. The first is structure: a knowledge repo of dated markdown files, organised by folder convention, authored against templates, with a standing rule that answers cite their sources. The folder structure is the class hierarchy, the template fields are the properties, the citations make the edges. The second is a small Python script that screens every change before it is built. An engineer rates the change's likelihood and consequence. A separate agent rates the same change in a sealed, fresh context, so neither can anchor the other. The script looks both ratings up in a five by five matrix, diffs the results, and validates the record. Same inputs, same outcome, for anyone, forever.

The interesting part is what it refuses. A result written in by hand that the table disagrees with. A made-up value like "Probably Low". A risky change with no controls attached. A disagreement with no reason recorded. These are refusals, not warnings: the record does not exist until it is fixed, and you fix the record, never the validator.

Determinism lives where the check runs

A validator the actor can skip is decoration. In our first version, "screen before you build" was an instruction in a prompt, and nothing stopped a session from skipping the screen entirely. The fix is placement: a merge check that refuses a pull request without a valid screen turns the instruction into a control, because CI runs where the agent does not. For us that check is the next gate, not a built one, which is precisely the point.

What cannot be checked gets owned

No table validates whether a plan is good or a risk is worth taking. There the system switches from determinism of output to determinism of accountability: a named person, looked up from the risk result rather than chosen. Low-stakes changes carry on. High-stakes changes stop and wait.

The textbook version of this architecture is RDF triples, OWL constraints and a reasoner. Ours is markdown, folders, one script and a table, and I would choose it again. Reach for the formal machinery when a machine must validate thousands of facts with no human in the loop. Until then, the folder structure is the ontology. I owe the frame to Frank Coyle, a Berkeley educator who compressed it into one line: Pydantic at the door, ontology at the ledger. Types check the shape of what an agent hands you. The ontology checks whether it is legal in your world.