The Architecture of Truth
The cognitive toolkit is necessary. It is not sufficient. Here is what’s missing.
My wife Jen sent me a Boston Globe essay this morning by Michael Shermer, the Skeptic magazine publisher, titled “What is truth, anyway?” I read it twice. I agreed with most of it. And something nagged.
Shermer’s toolkit is the right one. Fallibilism. Bayesian reasoning. Extraordinary claims require extraordinary evidence. Signal detection. Active open-mindedness. Free critique. If more public discourse ran on this equipment, we would all be better off.
And yet, in the same essay where he carefully reminds us that “the principle of fallibilism requires me to admit that I could be wrong,” Shermer delivers confident verdicts on COVID origins, climate severity, and gender, the kind of verdicts that suggest the toolkit has, in fact, settled the matter.
That gap is what I want to write about. It is not a gap in Shermer specifically. It is a gap in the framework he is using, and in the way most of us, myself included, talk about truth. The toolkit tells you how to weigh evidence. It does not tell you when you have reached the edge of the toolkit’s applicability. Without that second layer of awareness, the toolkit does not produce truth. It produces confidence.
Truth, I want to argue, is not a possession you arrive at by running the right cognitive procedures. It is a practice that requires an architecture, and that architecture is what the rest of this post is about.
What Shermer Gets Right
Shermer’s essay is, at its core, a defense of the cognitive equipment the Enlightenment built for navigating uncertainty. It is worth taking seriously, piece by piece.
Fallibilism: the recognition that any of our beliefs could be wrong, and that intellectual honesty requires holding them provisionally. This is not a weakness; it is the precondition for learning anything new.
Bayesian reasoning: the discipline of attaching probabilities to claims rather than treating them as binary, and updating those probabilities as evidence shifts. Shermer’s example of assigning UFO aliens a 0.01 percent probability rather than zero is the right move. It leaves room for evidence to change his mind.
ECREE: extraordinary claims require extraordinary evidence—Sagan’s principle, simple and durable. A blurry photograph is not evidence of Bigfoot. A grainy video is not evidence of alien visitation. The bar scales with the size of the claim.
Signal detection theory: the 2x2 matrix of hits, misses, false alarms, and correct rejections. Shermer is right that most public arguments fail because people cite only the cell that confirms their view. RFK Jr.’s stance on vaccines is a clear example. So is most medical-cure folklore. So, frankly, is most political commentary.
Active open-mindedness: Tetlock and Gardner’s finding that the best forecasters are the ones who actively seek evidence against their own positions, treat changing their minds as a strength rather than a weakness, and accept that randomness shapes outcomes. The worst forecasters are the ones with grand unified theories who explain away every miss.
Free critique: Shermer closes by arguing that the most important norm is the freedom to challenge any and all ideas. He is right. Censorship is corrosive in both directions: if I silence you, why shouldn’t you silence me?
This is a serious toolkit, assembled over centuries. I use it. You should, too. Most public discourse would improve dramatically if more people did.
The question is not whether the toolkit works. The question is what it works on, and what it cannot, by itself, do.
What’s Missing: The Constraint-Awareness Layer
Here is the move I want to make.
Shermer’s toolkit tells you how to weigh evidence inside a question. It does not tell you whether the question is one the toolkit can answer. That second layer, the awareness of where the toolkit applies and where it reaches its edge, is what I have been calling constraint-awareness. It is the working definition of wisdom in my Wisdom Gap whitepaper.
Constraint-awareness is not the same as humility. Humility is a disposition. Constraint-awareness is a structural property of how you hold a belief: knowing the conditions under which your method works, the conditions under which it does not, and the conditions under which you cannot tell which you are in.
Without it, the toolkit produces something that looks like truth but isn’t. It produces calibrated confidence inside a frame that the thinker has not examined. Bayesian reasoning, applied to a question whose evidence base is geopolitically corrupted, gives you a probability estimate that feels rigorous and is in fact noise dressed in numbers. The same diagnosis applies to any other tool in the kit. Each was designed to operate within the conditions the thinker is responsible for noticing, and none of them can notice for itself.
An example from my own consulting career, to make this concrete. My team was selected to lead a post-acquisition system integration following a large data analytics company’s acquisition of a smaller, complementary firm. Our standard approach was sound: spend the first six weeks in discovery, talk to stakeholders on both sides, build a proposal that accounts for the operational realities of each party. We did exactly that. We treated both companies as equally weighted clients. We came back with a proposal that assumed significant adaptation of the acquirer’s systems to accommodate the acquired company’s business. The budget was large. The timeline was long. We presented, and we had our hat handed to us.
The executive sponsor on the acquirer’s side summarized the problem in one sentence: “We are acquiring them; they need to adapt and fold into our way of doing business, not the other way around.” The toolkit was right. The conditions under which it was developed (a normal client with stakeholders whose interests are roughly symmetrical) were not the conditions in which we were operating then (a post-acquisition integration, where the acquirer’s operating model is the destination, not a negotiable input). Constraint awareness would have let us see that before the proposal landed, rather than after.
Three thinkers I keep returning to have each pointed at this layer from different directions. Judea Pearl, in his ladder of causation, distinguishes association (Rung 1) from intervention (Rung 2) from counterfactual reasoning (Rung 3), and his core warning is that statistical machinery applied at the wrong rung produces confident nonsense. Donald Hoffman, in Fitness Beats Truth, argues that our perceptual systems were not built to deliver reality; they were built to deliver fitness, and the two are not the same. Nassim Taleb, in his work on skin in the game, insists that beliefs held without consequences drift away from the truth in ways the holder cannot detect from the inside.
Each of them is naming a different edge of the toolkit. Pearl: the edge where your inference machinery exceeds its causal license. Hoffman: The edge where your perceptual interface is not the territory. Taleb: the edge where your belief has no feedback loop to correct it.
Constraint awareness is what lets you see those edges from within your own thinking. It is not a tool you add to the toolkit. It is the meta-property that determines whether the toolkit produces truth or confidence.
And here is the harder claim, the one I will spend the rest of this post and the next whitepaper defending: constraint-awareness cannot be generated by individual cognitive effort alone. It requires an architecture: an attention-experience feedback loop, sustained over time, inside institutions and developmental pipelines that test beliefs against consequences and reveal their edges. Strip-mine that architecture, and you do not get a population of bad reasoners. You get a population of good reasoners producing confident verdicts at the edges of frames they cannot see.
Which brings us to COVID.
Exhibit A: COVID Origins
Take Shermer’s COVID example. He writes: “I believe that the COVID virus is slightly more likely to have originated in a lab than a wet market.”
I am not going to tell you whether he is right or wrong. That is the point.
What I want to ask is a different question: what would have to be true for the toolkit to deliver a probability estimate on this question that means what a probability estimate is supposed to mean?
A Bayesian probability is only as good as the evidence base it draws on. The machinery was designed for evidence generated by a process you can characterize, from sources whose reliability you can estimate, with a sample space you can bound. COVID origins fail all three conditions, and the failures are not accidental.
The evidence base is geopolitically corrupted. The Chinese government has restricted access to the Wuhan Institute of Virology, withheld early case data, and shaped which samples reached international researchers. Some of what is missing is missing on purpose. You cannot run Bayesian updating on an adversarially curated sample space.
The evidence base is institutionally entangled. Western intelligence agencies, public health institutions, and research funders all faced reputational and legal exposure if the lab-leak hypothesis proved true. Some of the early dismissals were sincere; some were defensive; from the outside, you cannot reliably tell which. The reliability weights you would need to plug into Bayes’ rule are themselves contested.
The evidence base is epistemically novel. This is not a question like “did this defendant commit this crime” or “is this drug effective,” questions for which we have centuries of method and calibration. It is a question about a singular event in a domain where the base rates themselves are unknown and possibly unknowable. In Frank Knight’s terms, this is uncertainty rather than risk: a domain where probability estimates were never the right tool to begin with.
Each failure mirrors one of the edges from earlier. Corrupted evidence is Taleb’s edge: beliefs without honest feedback drift in ways the holder cannot detect. Entangled evidence is Hoffman’s edge: what reaches you is shaped by an interface (institutional, in this case) that was not built to deliver truth. Novel evidence is Pearl’s edge: inference machinery applied where its causal license does not extend.
None of this means we should not consider the origins of COVID. It means the cognitive procedure Shermer is using cannot, by its own internal logic, produce a calibrated probability here. The output looks like a Bayesian estimate. It is, more honestly, a prior, the starting belief one brings to a question before evidence updates it, given weight by a procedure whose conditions for operating were not met.
Shermer’s reasoning is not sloppy. It is rigorous reasoning applied to a question whose epistemic conditions the rigor itself cannot diagnose. You need a layer above the toolkit to notice that, and that layer is constraint-awareness.
The same diagnosis, applied differently and to varying degrees, fits his confident verdicts on climate severity and on gender. They are not the same kind of question as the COVID origins question, and treating them as equivalent would itself be a constraint-awareness failure. The whitepaper will work through them separately.
Why This Is Architectural, Not Personal
The point of the COVID example is not that Michael Shermer is a bad thinker. He is a good one. He is using the best cognitive equipment the Enlightenment produced, and using it carefully.
The point is that individual cognitive effort, however rigorous, cannot generate constraint-awareness on its own.
Constraint-awareness is not a tool you reason your way to. It is a property that develops over time through exposure to consequences within an architecture that connects beliefs to feedback. You learn where your toolkit reaches its edge by watching it fail, by being wrong in ways you cannot dismiss, in domains where the cost of being wrong is paid by you and seen by others. That loop, between attention, experience, and revision, is what produces the meta-awareness of where one’s own thinking does and does not work.
This is the argument I have been building across three previous papers, each from a different angle.
The Attention Crisis describes what happens when the input side of that loop is strip-mined, when attention itself is captured, fragmented, and monetized to the point where sustained engagement with any single question becomes structurally difficult. AHI describes the constructive alternative: an architecture in which AI augments human judgment rather than replacing it, preserving the developmental pipeline that turns junior practitioners into wise seniors. The Wisdom Gap describes what is structurally missing from current AI systems: the attention-experience feedback loop itself, without which knowledge cannot mature into wisdom.
What does a working truth-producing architecture actually look like? Consider the National Transportation Safety Board. When an aircraft crashes, the NTSB investigates with statutory independence from the airlines, the manufacturers, and the FAA. Investigators have access to physical wreckage, flight data recorders, and crew records. Their findings are public. Their recommendations have teeth. The feedback loop runs from event to investigation to industry-wide design and procedural change, and the cycle takes years, not news cycles. The result is one of the best-documented improvements in safety in any industry: commercial aviation has become extraordinarily safe over decades, not because individual pilots got smarter, but because the architecture turns each failure into a constraint the whole system inherits.
Notice what that architecture has structurally: independence from the parties whose interests are at stake, access to ground-truth evidence, public outputs, enforcement authority, and time horizons that exceed any individual career. Notice what it does not require: superhuman cognition in any individual investigator. The architecture produces constraint-awareness at the system level that no single mind, however rigorous, could generate alone.
The governance whitepaper I am working on will address the question of what an equivalent architecture for AI would look like. That is a harder problem, and one I do not pretend to have fully solved. But the NTSB shows that truth-producing architectures are possible, that we know roughly what they look like when they work, and that the question is not whether to build them but how.
Each of those papers, I now see, was circling the same underlying claim from a different side. Truth, the kind that earns its name, not the kind that is merely confidently asserted, is the output of an architecture. Attention is its input. The feedback loop is its mechanism. Wisdom is its mature form. And constraint awareness is the property that lets a thinker, working within that architecture, recognize the limits of what their own methods can deliver.
What worries me about the present moment is not that we have run out of careful thinkers. We have not. Shermer is one of many. What worries me is that the architecture that produces constraint-awareness is being degraded faster than careful thinkers can compensate for it individually. Attention is captured. Developmental pipelines are being shortcut by AI tools that mimic the output of judgment without the underlying loop that produces it. Institutions whose job was to test beliefs against consequences (academia, journalism, public health, the regulatory state) are themselves under stress. Power is also at work here, in ways this post deliberately leaves to the governance whitepaper: frames remain unexamined not only because constraint-awareness is hard but because powerful interests often benefit from their remaining unexamined. And the AI systems increasingly mediating public reasoning have inherited the toolkit but not the architecture: they can run the procedures, but they cannot, by themselves, generate the constraint-awareness that tells them when the procedures no longer apply.
The result is not a shortage of rigor. It is rigor without constraint-awareness: careful work, seriously equipped, producing confident verdicts at the edges of its own reach.
Truth as Practice
When I told a friend recently that I sometimes wonder whether there is any truth beyond physical reality, he pushed back. Others have too. They hear in that kind of statement a slide into postmodern relativism, the move in which everything becomes a matter of perspective, and nothing can be said to be true.
That is not what I mean, and I think Obi-Wan Kenobi, of all people, said it better than I have managed to: what I told you was true, from a certain point of view.
Read carelessly, that line is relativism. Read carefully, it is constraint-awareness. It is the recognition that a claim can be true given the conditions under which it was formed and still be incomplete, partial, or wrong from a vantage that those conditions did not include. Holding a belief that way, knowing the conditions of its formation, knowing what it can and cannot deliver, is not weakness. It is what separates truth-seeking from confidence-issuing.
If constraint-awareness develops within an architecture, the question for an individual reader is what to do as we build or rebuild that architecture. One practice that follows directly from the argument of this post: attach to every confident verdict you issue, including silently to yourself, an explicit statement of the conditions under which your method would be unreliable here. Not as a hedge, and not as performative humility, but as a structural check. If you cannot name those conditions, you have not finished thinking. If you can name them and they apply, you have not yet earned the verdict. This is a habit, not a toolkit, and it scales: from individual practice to institutional design to, eventually, the governance of AI systems whose confident outputs will increasingly shape the conditions under which the rest of us reason.
None of this is an argument against the toolkit. In domains with stable feedback loops, observable outcomes, and aligned incentives, clinical trials, well-instrumented engineering, and mature science, the toolkit produces real and reliable knowledge. The argument is that those conditions are not the universal case, and that issuing toolkit-shaped verdicts in domains that lack them is the failure mode worth naming. Where conditions are absent, and a decision is unavoidable, the right move is not to disguise a prior as a posterior but to act on the prior with the conditions named, and to update faster than a calibrated estimate would warrant when ground truth arrives.
Shermer ends his essay with a defense of free critique: the freedom to challenge any and all ideas, from which “in time the truth emerges.” He is right that this freedom is necessary. He is wrong, I think, that it is sufficient. Free critique produces truth only within an architecture that sustains attention, preserves the feedback loop between belief and consequence, and develops in its participants the constraint awareness to recognize the edges of their own methods. Without that architecture, free critique produces something else: a marketplace of confident verdicts, each rigorously defended, none of them constrained.
That architecture is what I want to build out in the next phase of this work. The previous three papers diagnosed what is breaking. The next ones will be about what to build. I will start with a longer treatment of The Architecture of Truth: what it is, how it forms, how it degrades, and what it would take to rebuild it in an age when the inputs to attention itself are under siege.
For now, the stake in the sand: truth is not a possession arrived at by procedure. It is a practice that requires an architecture. Each of us is already inside one, helping to sustain or degrade it with every confident verdict we issue. The work ahead is to build it deliberately.
Next week: the flaw isn’t AI’s. It’s ours, and we’ve been building it for a long time.
Originally published on Substack.
