Turtles All The Way Down
Every layer of checking rests on the one below it. Here is what happens at the bottom, and how long you can go without looking.
A scientist finishes a lecture on the solar system. An elderly woman in the audience stands up and tells him the whole account is wrong: the world rests on the back of a giant turtle. The scientist, politely, asks what the turtle is standing on. She has the answer ready. It is turtles all the way down.
The story is usually told about Bertrand Russell. It is also told about William James, and about various astronomers whose names change with the telling. Each version reaches us through someone who did not say where they got it. Nobody has established which lecture hall, or which century, or whether it happened at all. The provenance of the story about infinite regress has no bottom either.
The Number I Was Holding Up
For several years I worked for a consulting firm that ran a Delivery Excellence dashboard. Every project reported weekly against a set of components: scope, schedule, budget, client satisfaction, team performance, and one or two others I no longer remember. Each one got a color. I entered the scores myself.
On a program for a biotech client, I reported client satisfaction as green, week after week, and the score was accurate. The client contact was satisfied. Every concern he raised got addressed in the order it arrived. He was told early that the scope began to shrink because we could not deliver all of it, and hearing it early is most of what makes a shrinking scope tolerable.
Underneath that, a team was struggling on several counts, and had been almost from the start.
So the numbers were true. Scope was green because we had already cut scope. Satisfaction was green because I spent a substantial part of every week producing it. Every component reported a state, and none had a field for what that state cost to maintain. The effort was the risk, and the effort was the one thing the format could not carry.
I did not falsify anything. I reported what the components asked for, and I raised the delivery problems internally, every week, in the space the form left for them. The technical lead and the others read the situation as managed, and I did not want to go out on a limb by myself against my own team’s assessment. I also thought that if we all knew, we would fix it before it came due. So a room full of people who each knew something was wrong went on filling in a form that had nowhere to put it. I have watched a version of this happen more than once, and never once because anyone was hiding anything.
What eventually surfaced the problem was not the dashboard. In January, I sat down with the office general manager and argued that we should put three hundred and fifty thousand dollars of our own money into a three-and-a-half-million-dollar job to fix what we had not delivered. The technical lead and two others backed the position. The technical lead told him the only reason we were still in the room was the account management. He agreed to the money. We spent it carefully, the work landed, and the client was pleased in the end.
The dashboard recorded none of this. It had been green throughout, and it was green because of what I was doing underneath it.
The Tower We Are Building Now
The dashboard failed in a specific way, and the way matters more than the failure. It published an accurate number and gave nobody access to what produced it. Whoever read the green had to take on trust that the process behind it was sound, and the process behind it was me. Reported figures come loose from the work that generated them. That is not a property of consulting. It is a property of reporting.
A model’s output is evaluated by another model, because human evaluation does not scale. The evaluating model was trained on data that a model filtered for quality. Both are scored against a benchmark, and benchmark content has repeatedly been found in the training data of the systems being scored. The results are then summarized for the people who have to decide something, increasingly by a model, because the volume is large and the summary is fast.
Every one of those steps is defensible on its own. Each is documented, published, and argued for by people who know more about it than I do. None of them is hiding anything.
What is missing is not honesty. It is anyone whose job is to ask what the bottom layer is standing on. Each layer treats the one below it as ground, and the question of grounding gets handed down the stack until it runs out.
In Defense of Turtles
I want to concede this properly rather than in passing, because the concession is most of the argument.
Abstraction is how anything large gets built. Nobody writing an application re-derives the behavior of a transistor. Proxies are how you manage what you cannot measure directly, and almost everything worth managing cannot be measured directly. I have never inspected a bridge before driving over it, and I am not going to start.
Stacking is not the sin. A working tower of abstractions is one of the more impressive things people do. The engineer who trusts the steel certificate is not being lazy. That certificate is doing real work, and behind it there is a mill, a test, a standard, and a name.
The failure is narrower than the stacking. It is losing track of whether there is a bottom at all. Deferring the question of grounding to the layer below is fine, and unavoidable, as long as the deferral terminates somewhere. The tower becomes a problem only when every layer in it has been built on the assumption that somebody further down already checked.
Where It Stops
The regress has to stop somewhere, and the only place I have ever seen it stop is at someone who pays if it is wrong and cannot hand the paying to anyone else.
I want to be careful about how much that carries. It is not a guarantee of anything. People with their names on things sign off on work they have not understood. Systems with clear lines of liability fail badly and often, and the person carrying the cost is frequently the last to see it coming. The engineer who signs the beam did not derive the metallurgy, watch the testing, or stand in the mill. Finding the name does not tell you the answer is right, and it does not mean the person can check the work. It tells you the chain has a bottom instead of continuing indefinitely. The claim is that narrow: without someone in that position, there is no bottom to have.
I have spent most of this series on that claim, and I am not going to re-argue it here. The engineer who signs off on the beam carries it if the beam fails. The certificate is not the ground, and the signature is not the ground. The ground is that a person’s license, career, and name are attached to the load, and that the attachment cannot be undone by delegating.
On the biotech program, the ground floor was not the dashboard. It was a room in January, a general manager who had to spend money he would answer for, and an account lead recommending it who would answer for it too. Neither of those costs could be passed sideways. That is why the problem got fixed. It is also why it took until January.
What You Can Use
The woman in the story was wrong about cosmology and right about almost everything else.
So here is the test. Pick any validation you currently rely on. A green status, a certification, a score, a rating, a benchmark result, a maturity level. Ask what it is standing on. Then ask the same question of the answer.
Keep going until the answer stops being a process and starts being a name.
If you work down through the whole stack and there is nobody there, you are standing on a turtle. It may well hold. A great many of them do, for years at a time. But you should at least know that is what you are standing on.
I stood on one for most of a year, and I entered the number myself every week.
Originally published on Substack.
