ARKONE
← All case notes
Case note · silentThe Order Paper · multi-step deliverables5 minute read

The approval that was a guess: a vague yes read as consent

A human sentence that is neither a yes nor a no becomes a yes when something has to classify it and nothing may hesitate.

Silent. An approval request goes out and a half-answer comes back: sounds fine, though check the dates. Work resumes. Days later the approval sits in the trail with a name against it, and the person whose name it is remembers a sentence about dates and no decision at all. She is disputing a record that says she agreed, and you have nothing to set against it.

The request was sent, the reply was read, the decision was filed.

What went wrong

A gate, the step in a longer piece of work that waits for a person before anything continues, is only as strong as its reading of the answer.

The gate asks a closed question, and a person answers in the language people actually use: a yes with a rider, a yes about one half and silence about the other. Something must reduce that sentence to one of a few outcomes so the work can proceed or stop, and where that job goes to a model it is a classification: read the words, return the label that fits best. Best-fitting carries no claim of being right, and a model asked to label an ambiguous reply returns its best guess with nothing attached to say it was guessing. A sentence meaning check with me before you commit becomes a yes, and that yes looks exactly like the ones that were meant.

The Cabinet, ArkOne’s reference design for an executive agent, answers with three settings rather than a better classifier, all three on the Register, the one page that states every setting of the design. The reading is measured: an approval reply the design is less than eighty-five percent sure it understood is asked again, never filed, in a short message back naming the two readings it could not choose between. When the person does answer, the run records the decision and stops, and a person restarts it, so continuing is a separate human act rather than a consequence of the reply arriving. That is the rule at the Signature, the step that waits for a person anywhere in the design, applied at a gate. And the Record, the ledger of every action with its reason, is append-only: a correction is a new entry and the original stands, which is what makes an approval trail evidence rather than notes. The twenty-second paper sets out the waiting rules; the seventeenth sets out the ledger.

The record

Suppose an events company of three hundred and fifty staff, running an agent that assembles supplier renewals and pauses for a director before anything is committed. The company, the director and the exchange below are invented for the exercise.

The reply she sent Read without a threshold Read with one
”Sounds fine, though check the dates” Approved. The renewal is committed Below the line. Asked again: commit as drafted, or hold while the dates are confirmed
”Yes to the first part” Approved. Both parts proceed Below the line. Asked again: which part, and what happens to the other
”Go ahead” Approved. Recorded and the run stops Above the line. Recorded, and the run stops

Exhibit 1. Illustrative. One approval request, one half-answer, and what a design does with it when it may hesitate and when it may not.

The bottom row shows the threshold is no brake on the business. An unambiguous yes passes and is filed. What changes above it is that a reply nobody could have read confidently stops being read confidently, and the cost is one message and a minute of the director’s attention.

A confidence threshold does not make the reading correct: a reply the design misreads with conviction sits above the line and is filed as consent, and no number catches that. What the threshold does is make an unclear reading visible rather than silent, and asking again is the cheapest thing a design can do at a gate. The waiting rule is what keeps the limit affordable, because a run that stops on the decision cannot be carried forward by a misread reply alone.

This is tagged silent because nobody is ever told. The gate did not fail, the work did not stall, and nothing marks the entry as different from the approvals that were meant. It surfaces weeks later as a disagreement between a person and a record, which is the most expensive place for it to surface.

The defenceA confidence threshold on the reading of an approval reply: below it, the design asks again instead of deciding.

What happens

The half-answer is met with a short question naming the two readings, and nothing moves until the person picks one.

What your company sees

A second, shorter question, and nothing moved until it was answered.

Illustrative. One setting, switched. Its value is on the Register.

What it means for you

Say this in the meeting: send me an approval request now, and I will answer it with the words sounds fine, though check the dates. Then we both look at what the agent did with that sentence. Either a short question comes back naming the two readings it could not choose between, or the work carries on and an approval appears in the trail with your name on it. Nobody has to cooperate but the person driving the demonstration, and there is no form of words that gets around what you have just seen.

The list of approval decisions is the second thing to ask for, with the exact words of the reply the agent read beside each one. If the reply text is not stored, the trail cannot be checked by anyone, including you, and that is your finding. If it is stored, read the replies and count how many you would have called a decision. That number is the accuracy of your approval gate, measured rather than assumed.

Listen for one answer in particular, because it sounds like a good one: that the model understands natural language well. It tells you the design has no way of knowing when it did not, and a gate that cannot hesitate consents on your behalf.

Then one question about the trail itself: can an entry be edited. Where corrections overwrite the original, the trail records the last opinion held about the past rather than the past.

Next: The run that slept through the restart.

Asked plainly

How does an AI agent decide whether I said yes?

Something has to turn your sentence into one of a small number of outcomes, and where that job is given to a model it is a classification. The model reads your words and returns the label that fits best. Best-fitting is not the same as correct, and unless the design also records how sure the reading was, a confident label and a coin toss arrive in exactly the same shape.

What should happen when a reply to an approval request is unclear?

The design should ask again rather than choose. The reference design puts a number on how sure it is that it understood the reply and, below a fixed threshold, sends back a short question naming the two readings it could not choose between. Asking again is the cheapest action available at a gate, and it is the only one that cannot invent a decision.

Can an approval trail be trusted as evidence?

Only if entries are never edited. Where a trail is append-only, a correction is a new entry and the original stands, so a reader can see both what was first recorded and what was later said about it. Where entries can be rewritten, the trail records the last opinion held about the past rather than the past, and it is notes rather than evidence.

Talk it through before you decide

A discovery call, no deck: your situation, the parts of the room it touches, and what you would need to decide first.