We Feared AI’s Flaw. We Built It First.
Why our politics and our AI share the same blind spot.
A note I posted recently about due process sparked a broader thought. I’ll come back to it at the end.
The thing people fear most about AI has been happening in politics for decades.
Let me explain.
Large Language Models, the technology behind ChatGPT, Claude, and others, are remarkable at a very specific thing. Within the boundaries of what they were trained on, they are extraordinarily fluent. Confident. Articulate. Persuasive, even. Ask them something well within their corpus, and they will give you an answer that sounds authoritative, well-reasoned, and complete.
The problem is that they have no internally grounded model of uncertainty. They do not know what they do not know. Push them outside their training distribution, ask them something genuinely novel, something that requires real-world judgment or lived context, and they do not slow down, qualify, or say “I’m not sure.” They confabulate. They produce an answer that has exactly the same tone, confidence, and fluency as a correct one. The model cannot tell the difference. And neither, often, can you.
This is not a minor technical quirk. It is a fundamental architectural limitation. And it is, increasingly, what critics of AI are most alarmed about: not that these systems are wrong, but that they are wrong with total confidence, and you cannot easily tell when.
What makes this architectural limitation so hard to solve is that it was not introduced by the engineers. It was inherited from the source material. The text that pre-trains these models is the accumulated output of human communication, not just the last forty years, but millennia of rhetoric, law, scripture, political argument, and power exercised through language. Human communication is already saturated with what the evolutionary biologist Robert Trivers identified as self-deception: confident assertion, motivated reasoning, moral certainty, suppressed counter-evidence. We perform our convictions fluently and our doubts reluctantly. The feedback step that follows pre-training compounds this: human raters reward confident, fluent answers because that feels authoritative. The system correctly learns what we prefer. The flaw was not built in. It was passed down.
Here is my question: when did this become new?
The question isn’t why politicians behave like LLMs. The question is why our systems select so ruthlessly for exactly that behavior and discard everything else.
The behavior itself is older than politics. Politics is just where we see it most clearly today. Underneath it is something more fundamental: the human knowledge claim. Any moment where one person asserts to others that they know something, where authority, status, or persuasion hangs on the assertion, selects for confident delivery and punishes hesitation. The shaman who qualifies the prophecy loses the tribe. The elder who admits uncertainty loses the counsel. The expert who hedges loses the room. This is not a modern failure of character. It is an ancient feature of how humans establish and defend claims to know.
This didn’t start with algorithms. It didn’t start with mass media. Leaders have performed certainty for as long as there have been leaders, ancient tribal chiefs whose authority depended on never being seen to hesitate, Roman emperors issuing decrees from a position of assumed divinity, monarchs whose legitimacy required the performance of absolute knowledge, demagogues across every century who understood that confident assertion outperforms careful reasoning in any contest for attention.
What’s new isn’t the behavior. What’s new is the system that now selects for it at an industrial scale, amplifies it in real time, and punishes the alternative more efficiently than anything in human history. Social media poured gasoline on a fire that has been burning since our earliest ancestors organized themselves around people who claimed to know. Platforms amplify the loudest, most certain voices and bury the nuanced ones. Politicians adapt. They perform with greater certainty, a certainty that is further amplified. Round and round. The system doesn’t just tolerate overconfidence. It breeds it.
Education shows the same pattern. As Steven Mintz recently observed, the habits that careful reasoning actually requires, matching claims to evidence, engaging opposing views fairly, and revising under scrutiny, are not just neglected in most educational settings. They are actively discouraged by the way students are evaluated and rewarded. The system doesn’t produce confident hallucination by accident. It trains for it.
Watch a political campaign.
Any campaign. Any party. Any country, though I will let you apply your own examples.
The candidate arrives with a platform. The platform is confident, clear, and internally consistent. It has an answer for everything. It does not hedge. It does not say “this is genuinely complicated and I am not sure.” It does not acknowledge that the opposing view has any merit, or that the world might resist the plan once it meets reality. Nuance is a liability. Certainty is the product.
Then they get elected. And reality arrives.
The economy does not behave as the model predicted. The allies do not cooperate. The opposition does not collapse. The budget does not balance. The promised jobs do not materialize, or they do but somewhere else, or they do but not for the people who were promised them. The world, it turns out, was not waiting for the correct policy to be implemented. It had its own ideas.
And here is where the parallel becomes most uncomfortable. A well-designed AI system, when confronted with a question outside its competence, should ideally signal uncertainty, flag that it is operating at the edge of its reliable knowledge. The best ones are getting better at this. But a politician, confronted with the gap between their platform and reality, almost never does. They double down. They reframe. They find someone to blame. They stay in the corpus.
Because admitting uncertainty, after running on certainty, is politically fatal.
This is not really about confidence.
Confidence itself is not the problem. Experts are confident. Surgeons are confident. A good structural engineer does not hedge when they tell you the bridge will hold. Confidence, earned through genuine competence within a well-defined domain, is exactly what you want.
The problem is the absence of awareness of uncertainty: knowing where your competence ends. This is one of the most underrated cognitive capacities a human being can possess. It is what separates a good doctor from a dangerous one. It is what separates a good leader from a demagogue. And it is, not coincidentally, what separates genuine intelligence from its simulation.
An LLM that cannot model its own uncertainty is not wise. It is fluent. These are not the same thing.
A politician who cannot model their own uncertainty is not strong. They are performing well. These are also not the same thing.
The voters who reward performance certainty over genuine competence are not getting what they think they are getting. They are getting the political equivalent of a confident hallucination.
A word about compromise.
Compromise is what uncertainty-awareness looks like in practice. It is the act of acknowledging that your model of the world is incomplete, and updating it in response to someone else’s. That’s not a weakness. That’s how functional systems avoid drifting further from reality.
I know. It is an ugly word right now. Mediocre. Weak. The language of people who do not believe in anything.
I disagree. Strongly.
Compromise is how the greatest human gains have actually been made. Not the heroic narrative version, not the lone visionary who refused to bend and changed the world. That story exists, but it is the exception, and we have collectively lost our ability to distinguish it from its imitation. The far more common story of human progress is negotiation, coalition, trade-off, and the slow, unglamorous accumulation of partial wins.
Some will argue that conquest is more powerful. The great advances came from decisive force, not from committees. I would ask anyone alive for any part of the last hundred years to sit with that honestly. Two world wars. The Cold War. Vietnam. Iraq. The wreckage of absolutism, administered at scale. The places where things actually got better, where poverty fell, where disease retreated, where life expectancy climbed, were almost always the result of sustained, boring, incremental cooperation between people who disagreed about things but agreed to keep working.
That is not a weakness. That is civilization.
So what are we actually asking for?
Not perfection. Not a political class of philosopher-kings who speak only in careful qualifications. Not AI systems that refuse to answer anything they are not certain about.
Just this: a restored tolerance for nuance. A willingness, in our politics, and increasingly in our AI, to reward “I’m not sure, but here is my best reasoning” over “here is the answer, delivered with complete confidence.”
But tolerance alone won’t be enough. Tolerance is a cultural mood. It comes and goes. What actually works is when systems structurally require engagement with contrary positions. Due process doesn’t rely on judges being humble. It builds humility into the architecture: evidence must be examined, dissent must be heard, certainty must be earned through scrutiny. The same logic applies to our politics and our AI. We don’t need leaders who personally appreciate complexity. We need systems that make engaging with it unavoidable.
The irony is almost too neat. We are having an urgent public debate about the dangers of AI systems that confidently hallucinate. We should be having the same debate about the human systems that have been doing it for millennia, doing it with industrial efficiency for the last forty years, and now, through AI, at a scale and speed nothing in human history has prepared us for.
We don’t need to accept either. But fixing both starts in the same place: understanding that nobody is right about everything, that uncertainty is not weakness, and that the gap between confident fluency and actual wisdom is exactly where the damage gets done.
Which brings me back to where I started. The rush to judge on allegations alone, before trial, before evidence, before verdict, is its own form of hallucination. A confident narrative is preferred over a slow, uncertain process. We have built an entire legal system on the principle that certainty must be earned, not imposed. That due process is not a weakness. It is the mechanism that keeps the system tethered to reality. It is, almost exactly, what we are asking for in our politics and in our AI. The question is why we seem to want it in our courtrooms but nowhere else.
Next week: The Category Error at the Heart of Justice. Our legal system is procedurally fair but causally illiterate, and the difference matters more than we admit.
Originally published on Substack.
