The Better Design Waits for the Rule
Why good ideas that everyone agrees with still don’t happen, or happen too slowly
The Model That Changed Nothing
Around 2010, I worked on a model that scored every bridge in the National Bridge Inventory, the federal record of highway bridges on America’s public roads, on two things at once: what condition it was in, and what it would cost if it failed. A lightly used rural bridge in poor condition and a busy interstate crossing in fair condition carry very different risks. Yet most bridges were inspected on the same two-year cycle.
I presented the model at the Federal Highway Administration (FHWA) headquarters in Washington, alongside colleagues from Lloyd’s Register. In the room were the Associate Administrator for Infrastructure and several of his colleagues from the bridge side. He was supportive, but he made no decision and no commitment. Bridge leads at state transportation departments gave it the same reception. We had linked the model to Google Earth, so we could fly over each state’s highest-risk bridges. None of the state bridge engineers who saw it disagreed with what it showed. There was real interest, and then nothing.
The inspection program did not change. Not then.
The information existed, the people closest to the problem found it credible, and it still did not turn into action. I have come to think this is not unusual. It is one of the common ways a better design loses.
This series argues that the answer to AI’s judgment problem is design: build systems so the people who bear the consequences of a decision stay part of making it. If that is right, there is an uncomfortable follow-up. Why does the design answer so often lose in practice?
The bridge model is my best evidence of how it happens, because it happened in front of me.
Two Columns
The usual reason a better design loses is cost. Doing things properly takes more money up front, and budgets are set in the present. That is often a fair objection, and it would be dishonest to pretend the design answer is always cheap.
The bridge model removed that excuse. It worked from the data states were already collecting for the inventory. Nothing new had to be gathered. Rolling it out wouldn’t have been free, but it was modest compared with the size of the program.
The benefits were real, but they took a different form. Few people would argue that a simple concrete beam bridge, under twenty years old and used mostly by farm vehicles, needs the same attention as a major interstate crossing. Inspecting such bridges less often would free up effort for better inspection and targeted maintenance on the large, busy bridges where failure matters most. Those bridges are costly to maintain, but replacing one costs far more. If that attention added five or ten years to the life of a major crossing, the savings could be large. The biggest benefit of all is a failure that never happens.
That is the problem. A failure that never happens leaves no record. No one can point to the bridge that did not close, or the replacement that was pushed back a decade because a problem was caught early. The benefits go in the ledger as estimates about events that, if the design works, will never occur.
The other column, it turns out, was not money. It was the risk of change itself, and who would carry it. That is where the rest of this essay goes.
Tonie-Marie Gordon, writing about AI at work, describes the same accounting error from the other direction. The productivity gain from a new tool gets booked in full, while the human effort needed to check its output is left off the ledger. Either way, what gets counted tends to win.
The Number and the Money
At the time, the inspection data did more than describe bridges. It helped decide how much federal money each state received for them. Under the Highway Bridge Program, a state’s share of the funding was based on the estimated cost of repairing or replacing its deficient bridges, those formally rated as in poor condition or outdated for today’s traffic, calculated from the inventory.
That made any change to how bridges were inspected more than a technical question. The same data that recorded a bridge’s condition also fed the formula that divided up the money. A state’s bridge staff might find a risk model persuasive and still have good reason to be cautious. If the new approach changed what was collected, or how it was read, it might change what the state received. Nobody would want to be the one who found out the hard way. This is not a claim that anyone gamed the numbers. The caution was reasonable, given how the money worked.
A slow federal process added to the caution. Changes to a national program move through years of rules, comments, and consultation, and a program that seems to be working invites the natural response: if it isn’t broken, don’t fix it. The trouble is that “working” meant the inspections got done and the numbers came in. It did not mean the effort was going where it did the most good.
Once a number decides where money goes, protecting that number can become a goal in itself. The Congressional Research Service noticed a version of this in 2008, observing that a formula paying states according to their deficient bridges gave them little reason to reduce the count.
Through her reporting, Rana Foroohar has traced the same pattern across the wider economy. When firms were judged on financial figures, managers came to steer by the figures rather than by the work they measured. The figures were persuasive, and too few people asked what they left out.
In this case, the better design did not lose on cost. I cannot prove why it lost. I believe it lost at least in part because it threatened the numbers the money depended on.
Not Just Bridges
The bridge model is the clearest case I have, but I have seen versions of it in technology consulting.
One project stays with me. I worked with a large software company on how its sales system handled leads: how a first contact became a qualified lead, and then a real prospect. Marketing was judged on how many leads it passed along, and sales on how many it turned into customers. Sales believed marketing was passing along weak leads to boost its numbers. Those leads rarely went anywhere, so the sales team wasted time chasing them and looked worse when they failed to convert.
Nearly everyone recognized the problem when we laid it out. Fixing it meant changing how both teams were measured, at the same time. Our engagement ended with recommendations to do that, and an expectation of follow-on work to implement them. That work never came. As far as I know, the company kept things as they were.
The rule there was not a federal regulation but a pair of targets. The pattern was the same: a better arrangement everyone could see, and no one in a position to change the rule for both sides at once.
The Asset Nobody Counts
One more thing is missing from both columns, and it matters most for AI.
Most organizations depend on judgment that lives in their people rather than their procedures. It shows up in the inspector who knows which joint on which bridge to look at twice, or the sales manager who can tell a real lead from a hopeful one within a few minutes on the phone. Little of it appears in a budget. You learn it through practice, pass it on informally, and notice it mainly when it is gone. I think of it as cognitive collateral: the capacity an organization relies on to catch its own mistakes, which never shows up on its balance sheet. If offloading judgment is a kind of borrowing, cognitive debt is what accrues, and cognitive collateral is what is lost when the debt comes due and cannot be paid.
AI tools put this capacity under pressure in a particular way. When a system does the routine work, people get less practice at it. In 1983, the engineering psychologist Lisanne Bainbridge pointed out the irony: the skills that fade during smooth automated operation are the ones needed when the automation fails.
A design that keeps people exercising their judgment, rather than only approving what a system produces, protects that capacity. But the protection is another benefit that never appears in any column. The simpler design’s efficiency gain is immediate and measurable. The judgment it quietly spends is neither, until the day it is needed and turns out to be thinner than anyone knew.
That is the design answer’s problem in its most general form. What it protects is invisible, and what it asks people to change is not.
Asking to Be Ruled
This month, the heads of several of the largest AI labs renewed a call that industry leaders have made, on and off, for years: they asked for rules. At the United Nations Security Council, several told members that AI needed global oversight. Anthropic’s Dario Amodei and OpenAI’s Sam Altman have both said development should slow down, though Amodei has argued the leading labs should slow down only as far as they can while still leading China. Earlier this month, the White House rejected calls for federal guardrails.
There are two serious ways to read the request.
The first follows this essay’s logic. A lab that slows down on its own bears the whole cost: it falls behind competitors who did not. The benefit, a failure that never happens, is shared by everyone and credited to no one. That is the bridge problem again. A state could not change the inspection schedule on its own; the change had to come from the body that set it for everyone. It eventually did, but only after Congress required it.
The second reading is less generous, and just as structural. Rules a large company can absorb easily can be heavy for a small one, and the biggest labs are doing the asking. Vice President JD Vance has called the push a possible Trojan horse. This is the lesson of the bridge funding formula in another setting: a rule that decides what counts also decides who gets ahead, and those shaping it have interests of their own.
The two readings are not mutually exclusive. A company can sincerely want rules and still benefit from the ones it helps write. Either way, the questions from an earlier essay in this series apply: who sets the threshold, who pays when it is wrong, and does it ever switch off?
The same rule could protect the public or protect the people already in front.
A Decade Later
The story has a coda I only learned while writing this. In 2012, Congress rewrote the federal highway program. The funding formula tied to deficient bridges was retired, and the Secretary of Transportation was told to consider a risk-based approach to how often bridges are inspected. In 2022, the national standards were updated. States can now inspect low-risk bridges less often and high-risk bridges more often.
I cannot say whether our work played any part. The firm I worked for, Cambridge Systematics, had for years maintained the bridge management software many state transportation departments used, and the risk model grew out of that experience. My colleague Dr. Gurenich built its computational engine. As far as I know, it was among the first national risk models built on the inventory data, but others were thinking along the same lines.
What is worth noticing is what it took. The merits were visible years earlier. What finally changed was the rule, for everyone at once, and a funding formula that no longer depended on the old numbers.
That is a good outcome. But it took more than a decade, and the official accounting never recorded what those years cost.
I will be plain about where I stand. I think we are repeating the bridge timeline with a technology that changes much faster than concrete. The bridges could afford twelve years. I do not think our judgment can. I may be wrong about the pace, and I would rather be early and wrong than right and late. But if the design answer waits for the rule again, the cost of waiting will once more go unrecorded, and we will learn what it was only when we need what we lost.
Originally published on Substack.
