The bias cut the number
of Black patients identified
for extra care by more than half.
Obermeyer et al.
Science, October 2019.
A risk algorithm running across US health systems
was flagging Black patients for extra care
at less than half the rate it should have.
17.7 percent of the patients it enrolled.
46.5 percent once the objective was corrected.
The algorithm was accurate.
It predicted healthcare cost
with precision.
Cost was standing in for need.
Less gets spent on Black patients
at the same level of illness.
So the model learned
they were healthier.
Nobody wrote a biased rule.
Nobody trained it on bad data.
Someone chose
what the model would predict.
That choice was the decision.
Made once, early,
by whoever was configuring the objective.
No name on it.
No review.
No authorization.
The same architecture
is moving into credit right now.
What guardrails usually check.
↳ Protected class variables excluded
↳ Approval rates compared across groups
↳ Model accuracy validated
What actually decides the outcome.
↳ What the model was told to predict
↳ Which proxy stood in for the thing itself
↳ Who authorized that substitution
An accurate model
answering the wrong question
passes every fairness test
built to check the answer.
CFPB Circular 2026-03.
May 2026.
A lender using a machine learning model
remains fully responsible
for specific and accurate reasons.
Complexity is not a defense.
Neither is the vendor.
The Governance Owner names
what the model is authorized to predict.
The Decision Owner holds
the call on the applicant.
The Handoff is where the objective
gets reviewed before it reaches anyone.
The Accountability Gap™ (TAG™)
does not open at the denial.
It opens at the objective function.
You already know which model in your stack
is optimizing for something
nobody in the building chose.
That is not a bias problem.
It is an authorization problem
that produces bias on schedule.
Who named the question your model answers?
Mo Johnson, MD MBA is a cardiothoracic surgeon and the founder of GPe Research. He created The Accountability Gap™ (TAG™), which names the moment the AI stops and the named owner starts. MedicoVigilance™ covers clinical AI. FinVigilance™ covers financial AI. Field Notes are short dispatches from the accountability frontier, published at tagframework.com.
Follow the work on LinkedIn: linkedin.com/in/mo-johnson