An answer arrives with no sense of its stability — users cannot tell whether a recommendation is robust or one assumption away from flipping.
← Back to patterns
Google
Trust & Verification
Research PatternCounterfactual Probe
What would have to change for a different answer?
An answer arrives with no sense of its stability: users can't tell whether a recommendation is robust or one assumption away from flipping. Counterfactual Probe exposes the levers — which inputs, moved how far, would change the verdict — and lets the user pull them: drag the budget 15% higher and watch the recommendation flip, with the boundary marked. Asking "why" gets a rationalization; probing "what would change your mind" shows the actual decision surface.
Metaphor
Lever
Interaction
Input Levers, Distance-to-Flip Indicator, Live Re-Decision
Use case
Recommendations, Risk Assessments, Approval Decisions
Failure mode
Sealed Verdict
Control mechanisms
Probeable Input Sliders · Decision-Boundary Marker · Flip Preview · Sensitivity Ranking
Reference examples
Google What-If Tool (PAIR) · Fiddler AI counterfactual explanations · Adverse-action reason codes (credit decisions)
Interactive Example
The live side-by-side is the comparison.
This prototype already contains the contrast between chat and the pattern response. It stands as the primary comparison surface for this page.
Why explains the answer you got. The boundary shows how nearly you didn't.
Vendor B wins on a €40k budget, twelve weeks, a team of three, strict compliance. Ask why and chat lists several reasons; ask what if the budget were €60k and it writes a fresh essay naming Vendor A. Two fluent answers sharing no structure — neither says whether the flip sits at €45k or €70k.
Pull the levers instead and the boundary sits on the control: budget flips to Vendor A at €52k, €12k away; timeline at 18 weeks; team size at six; the compliance toggle flips it alone. The margin reads 10.8 points — moderately stable, not robust — and sensitivity puts compliance at 41%, more than timeline and team size combined.
The levers here sit on a real weighted function, which is why the verdict recomputes honestly. That is also the open edge: a language model's verdict has no such function underneath, and levers bolted onto fabricated reasoning are theater. Credit denials have owed a counterfactual receipt for decades — the demand is older than the models.
An explanation you cannot move is not an explanation. It is a verdict with footnotes.
DOPE Lens
How this pattern improves AI interaction quality.
Observability
Each input shows its distance-to-flip, so the fragile assumptions are visible before anyone acts on the answer
Directability
The user probes the decision directly — drag a lever, watch the verdict respond — instead of prompting for hypotheticals
Predictability
The boundary is marked on the control: the user knows where the flip happens before crossing it
Explainability
Sensitivity is ranked — which factor actually drives this verdict — replacing prose rationalization with testable structure
Fit
Where it earns its place.
Use it
when
when
Decisions rest on the answer and users need to know how fragile it is before acting on it.
Avoid
when
when
Creative or generative tasks with no decision boundary to probe.
Closest neighbours
No product ships this exactly — these come closest.
This is a research pattern — we describe a UX move that no shipping product implements cleanly today. The entries below are adjacent: they share the spirit of the pattern even if they take a different shape.
Google · What-If Tool (PAIR)
Probe an ML model's decisions by editing datapoints and finding counterfactual twins — the closest shipped implementation of pull-the-lever-and-watch-the-verdict, aimed at practitioners rather than end users.
Visit Google ↗
Fiddler AI · Counterfactual explanations
Fiddler
Model-monitoring platform surfacing which feature changes would flip a prediction — decision-boundary visibility as an enterprise observability product.
Visit Fiddler ↗
Adverse-action reason codes
US consumer credit (regulated)
Credit denials must state the principal reasons — a legally mandated, decades-old counterfactual receipt ("what would have to be different"), proving the pattern predates AI.
Visit US consumer credit (regulated) ↗
Research Frame
Why this pattern exists, and where it breaks.
Expose the levers: which inputs, moved how far, would change the verdict — and let the user pull them, with the decision boundary marked on the control.
Asking "why" in chat gets a rationalization and "what if" gets a fresh essay — neither shows the actual decision surface or where it flips.
Counterfactuals on fabricated reasoning are theater — the levers must be causally tied to the actual computation, or honestly labeled as approximations.
Common questions
What people ask about this pattern.
When should I use the Counterfactual Probe pattern?
+
Decisions rest on the answer and users need to know how fragile it is before acting on it.
When should I avoid the Counterfactual Probe pattern?
+
Creative or generative tasks with no decision boundary to probe.
What problem does Counterfactual Probe solve?
+
An answer arrives with no sense of its stability — users cannot tell whether a recommendation is robust or one assumption away from flipping.
Why is chat the wrong fit for this?
+
Asking "why" in chat gets a rationalization and "what if" gets a fresh essay — neither shows the actual decision surface or where it flips.
Related Patterns
Curated connections, not just neighbors.
Hand-picked relationships — what this pattern pairs with in practice, what it replaces, what it builds on, and what builds on it.
Often paired with
Reasoning Trace
Reasoning Trace shows the reasoning the AI claims; Counterfactual Probe shows which inputs actually flip the verdict.
Often paired with
Confidence Signals
Confidence Signals say how sure the AI is of a claim; Counterfactual Probe shows how far an input can move before the verdict flips.
Often paired with
Second Opinion
Counterfactual Probe varies the inputs and Second Opinion varies the model, two different checks on the same answer.