AI Governance and the Architecture of Practice
This paper specifies the four design requirements any AI governance framework must meet to keep human judgment in authority over the machine, arguing that current frameworks inspect the wrong layer.
AI is being deployed into judgment-laden institutional roles (clinical, financial, legal, regulatory) faster than governance frameworks for those roles are adapting. This paper argues that the frameworks being built are designed against the wrong failure mode, inspecting artifact quality when the variable that matters is the cognitive architecture beneath.
Core Argument
Drawing on Bainbridge’s ironies of automation, Kahneman’s dual-process framework, and the structural-dissent literature, this paper argues that whether AI-mediated judgment is wise depends not on whether AI is used but how: as scaffolding for System 2 deliberation, or as amplification for System 1 reflex. This variable is invisible at the artifact layer. Two practitioners producing identical compliant outputs can be running opposite cognitive architectures, and cognitive debt accumulates silently beneath the inspection regime that current frameworks rely on.
Four Design Requirements
The paper specifies four design requirements any governance framework must meet:
- Cultivate the cognitive conditions under which wise judgment forms
- Require structured deliberation at defined decision points
- Preserve the developmental pipeline through which practitioners acquire judgment
- Surface the cognitive architecture beneath compliant artifacts
These four requirements, together with the interlocking liability architecture developed in the final section, form a single integrated specification rather than a list of independent recommendations.
How This Fits
This paper is the third in the constructive arc following the AHI and Wisdom Gap papers. Where AHI made the case for the goal and Wisdom Gap explained why AI is structurally capped below it, this paper specifies what governance frameworks must do given those conditions.
Abstract
AI is being deployed into judgment-laden institutional roles faster than governance frameworks for those roles are adapting. This paper argues that the frameworks being built are designed against the wrong failure mode, inspecting artifact quality when the variable that matters is the cognitive architecture beneath. Drawing on Bainbridge’s ironies of automation, Kahneman’s dual-process framework, and the structural-dissent literature, it demonstrates that whether AI-mediated judgment is wise depends not on whether AI is used but how: as scaffolding for System 2 deliberation, or as amplification for System 1 reflex. Two practitioners producing identical compliant outputs can be running opposite cognitive architectures, and cognitive debt accumulates silently beneath the inspection regime that current frameworks rely on. The paper specifies four design requirements any governance framework must meet to keep human judgment in authority over the machine: cultivate the cognitive conditions under which wise judgment forms, require structured deliberation at defined decision points, preserve the developmental pipeline through which practitioners acquire judgment, and surface the cognitive architecture beneath compliant artifacts. These four requirements, together with the interlocking liability architecture developed in the final section, form a single integrated specification rather than a list of independent recommendations.