Ratifia Ratifia Ratifia
← All posts

Automation bias is a design bug

Show an AI's recommendation the wrong way and your reviewers stop reviewing. Automation bias is not a training problem or a user problem. It is a layout decision.

Engineering 4 min read approvalsdesignai-agents

You put an AI's recommendation in front of a human because you want a second opinion. But if the recommendation looks like an answer, you will not get a second opinion. You will get a signature.

The reviewer defers, the approve rate climbs toward 100%, and your audit log fills with consents that were never really given. This has a name — automation bias, the well-documented tendency to over-trust an automated suggestion, studied for decades in aviation and medicine. In an approval interface it is not a property of your users. It is a property of your layout.

Why the check collapses#

A reviewer is supposed to be a check on the model. The check only works if the human forms an independent judgment before they are anchored to the model's. Most approval UIs destroy that by construction: the model's verdict is the first and largest thing on the screen, set in the same neutral type as the facts, often with a confidence score and a pre-highlighted Approve. Every one of those is a thumb on the scale, and together they turn the reviewer from a decision-maker into a witness who nods.

The four thumbs#

Presentation as fact. "Approve — matches policy" in system typography reads as ground truth. The model is a witness, not the judge, so dress it like one: attribute the recommendation, name the model, and never let it render like a fact the system computed.

Order. Put the recommendation first and you anchor the reviewer to it; everything below is then read as confirmation rather than evidence. Move it below the action, the tool call, and the state delta. Let the human see the consequence and form a prior, then read the model as one more input into a judgment they have already started making.

The confidence score. "94% confident" is false precision. The number is almost never calibrated, and it reads as authority precisely because it is a number. Drop it, and surface what the model was unsure about instead. "I could not verify that account 8841 is the original payer" changes a decision. "94%" just ends it.

The default. A pre-selected or visually dominant Approve is the path of least resistance, and in a queue of forty pending items the path of least resistance is the only path anyone takes. Approve and Reject should carry equal weight until a human puts their weight on one.

Friction is the feature here#

On a high-stakes decision, make the reviewer do something — expand the exact tool call, reveal the before-and-after — before the buttons unlock. This is the one place in your product where adding friction is correct. A decision of record should cost a few seconds of genuine attention. If it costs zero, it is not a decision of record; it is a reflex with a timestamp.

If it never rejects, it isn't a control#

An approval system that never says no is not oversight. It is latency with extra steps. Watch two numbers and treat them as health metrics, not vanity ones: the reject rate and the dwell time. If nobody ever rejects, or if decisions resolve in two seconds flat, the human is rubber-stamping — and a rubber stamp is a compliance artifact, not a control. That is how an approval system decays into theater within a quarter, and the decay is invisible on every dashboard that only counts throughput.

This is not an argument against the AI#

You are not trying to hide the model. It is genuinely useful — it triages the queue, explains the case, and catches things a tired human at 4pm would miss. The goal is not less AI. It is to keep the AI in the role of evidence the human weighs, and out of the role of verdict the human ratifies. Those are the same information presented two ways, and only one of them is oversight.

The dividing line#

The question your interface should be optimizing is never "is the model right?" It is "did a human actually decide?" A UI tuned to make the model look right is optimizing the wrong variable — it will pass every demo and fail every audit, because the demo measures whether the answer was good and the audit measures whether anyone chose it.

Ratifia presents the model's reasoning attributed and placed below the delta, never as a system-generated fact, with no pre-selected verdict — because the entire point of a human in the loop is a human forming a judgment, and that only happens if the interface leaves room for one.