Your AI Systems Record Everything Except the Decision

When leaders ask whether their AI systems keep good records, engineering usually says yes, and engineering is right. Every request is logged, every response is timed, every error is counted. I am not convinced those records answer the questions that matter about AI. The systems record a great deal about each request. They rarely record the decision.

The logs were built to answer engineering's questions

Take one request at the health plan from the earlier posts. A physician's office submits a prior authorization request. The AI flags it for clinical review instead of approving it. A clinician reviews it and denies it, and the plan mails a letter to the member.

The plan's systems recorded that request in three ways. The logs noted that it arrived at a certain time and returned successfully in 180 milliseconds. The metrics added it to a chart of average response times. The traces followed it across the six services it passed through. Engineers built all three to keep the service fast and available, and they do that job well. All three are organized by time or by service. None of them is organized around the decision.

Months later, the member appeals. The appeal asks four questions:

• Which model version flagged the request?
• What did the model see, and what did it say?
• What did the clinician see before denying?
• Which version of the coverage policy applied?

The logs can't answer any of them. They were never asked to.

A decision record has to be written at the moment

The record the appeal needs is a different object: one entry per decision, written when the decision happens. It names the request, the model version and the policy version. It records what the model recommended and the reason it gave, what the clinician did, and which checks ran. It is protected so that nobody can change it quietly afterward.

Timing is the whole point. At the moment of the decision, every one of those facts sits in the system's memory, and writing them down costs a few fields. Months later, the model has been updated, the policy has been revised, and the clinician may have left. Rebuilding the same record then means pulling people into meetings to reconstruct what they think happened. It is cheap at the moment and expensive later. In Part 1 this was step 3, the observable signal, and it was the step most likely to break.

A review that happened is not a record that exists

Plans often point to their safeguards when asked about oversight. The guardrails limit what the AI may do. Clinicians review flagged requests. A policy document describes the intended process, and the vendor's evaluations tested the model before launch. Every one of those matters, and none of them produces the record. A guardrail constrains the AI, a review approves or rejects, a policy describes, and an evaluation judges. Only a record shows what actually happened on this request. If the clinician's review isn't written into the decision record, the plan can't show that it happened, however carefully it was done.

What to do this month

Pull one real AI-assisted decision from last quarter. Ask your team to answer the four appeal questions for it within a day, using only what your systems recorded at the time. Each question they can't answer marks a field your decision record is missing.