Decision rights belong where the knowledge is, and AI products need them written down

On the evening of 27 January 1986, engineers at Morton Thiokol in Utah told NASA officials on a teleconference that Challenger should not launch below 53 degrees Fahrenheit, the lowest O-ring temperature of any earlier flight. The forecast for the next morning was a low near 22 degrees and about 26 at the 9:38 a.m. launch time then scheduled. At roughly 10:30 p.m. Eastern time, Thiokol's managers took their side of the teleconference offline for about half an hour. When the call resumed, the company recommended launch, and a Thiokol vice president signed the rationale and faxed it to Kennedy. Challenger lifted off at 11:38 the next morning and began to break up 73 seconds later. All seven crew members died.

The Rogers Commission, which investigated the accident for the president, concluded in 1986 that the launch decision was flawed. Officials at the top of NASA's flight readiness chain did not know the history of O-ring damage on earlier flights, and they did not know that Thiokol had first recommended in writing against launching in the cold. The Commission judged it highly unlikely that they would have approved the launch had they known.

The people in this story were competent, and the engineers were right. What failed was the allocation of decision rights: who could waive a safety limit, who had to be told, who could say no and who had to prove what. My judgment is that slow organizations and reckless ones often share this cause, since in both nobody can point to a written answer for who decides. An AI product team meets versions of the same questions every week. Who may ship a model? Who may move a threshold, override a prediction, accept a residual risk or stop a launch? Most teams have never written the answers down. I think a short written table is worth its cost for those decisions, and the Challenger record supports two design rules for it: the right to stop should be cheap to use, and the right to ship should depend on evidence the shipper does not control.

At Challenger, the person who set the safety limit also waived it

For anyone designing decision rights, the most useful passage in the report is the Commission's account of flight readiness. In July 1985, after a nozzle joint on mission 51-B showed erosion of a secondary O-ring, which meant the primary seal had failed, a launch constraint was placed on flight 51-F and the flights after it. The Commission's account is that Lawrence Mulloy, the Solid Rocket Booster project manager at Marshall, imposed it and regularly waived it, six consecutive times before 51-L. NASA's review structure had four tiers, and Level II, the Shuttle program director's level, was where critical problems were supposed to be brought forward. The Commission reported that neither the constraint, nor its reason, nor the waivers were known to the Level I and Level II officials above Mulloy. It concluded that no system made it imperative for all levels of management to consider launch constraints and their waivers. At the L-1 readiness meeting on 25 January 1986, all Flight Readiness Review actions were reported closed out, and a Level I directive issued two days earlier said no problems with the booster had been identified.

So the system had a safety control, and the same person who imposed it could lift it as often as the schedule required. Mulloy told the Commission that the Thiokol objection was a Level III issue that had been resolved, and the normal route for escalation, as he described it, ran from him to Reinartz to Aldrich to Moore. The officials at Levels I and II did not know the constraint existed, so the one check the structure offered depended on the person being checked. Whether or not a reporting step was skipped, the arrangement put the author of the constraint and its judge in one chair.

Other links in the chain show the same pattern: rights that exist on paper and do not connect. Reinartz, the manager of Marshall's Shuttle Projects Office, confirmed to the Commission that he decided not to carry the Thiokol objection to Level II. Marshall's director, Lucas, asked Reinartz and Mulloy at the launch control center around 5 a.m. on launch day how the previous evening's concern had been settled, was told it was resolved, and did not tell Aldrich or Moore, saying that was not the reporting channel because Reinartz reported directly to Aldrich. Each person acted inside a lane that made sense. The Commission's Recommendation II adds a structural reading: element project managers felt more accountable to their home center than to the shuttle program, and funding and work definition bypassed the national program manager. In the vocabulary of decision rights, the line of accountability and the line of resources did not meet.

The Commission's response is itself a list of decision rights. It asked NASA to write a policy governing how launch constraints are imposed and removed (Recommendation V), to record flight readiness reviews, and to create a safety, reliability and quality assurance office with agency-wide authority, reporting to the NASA Administrator and independent of program responsibilities (Recommendation IV). None of those remedies adds engineering talent. Each moves authority to a different place, and the office in Recommendation IV gets a reporting line that does not run through the program being checked.

The knowledge sat with engineers and the decision sat with managers

Michael Jensen and William Meckling wrote a paper, published in the volume Contract Economics in 1992 and reprinted in 1995, that gives the relevant principle. They define specific knowledge as knowledge that is costly to transfer between people, and general knowledge as knowledge that is cheap to transmit. Because specific knowledge is costly to move, they argue, getting it used in decisions requires pushing decision rights toward the people who hold it. Setting aside agency problems, giving the right to the person with the relevant knowledge raises efficiency. They add the necessary caution in the same abstract: self-interest means a control system has to push the holder to use the right well.

On the night of 27 January the knowledge about how cold affects a rubber seal was specific in exactly this sense. It lived in the heads and charts of Boisjoly, Arnold Thompson and their colleagues, and it was an inference from thin data, since no flight had been that cold. Thiokol's first recommendation said as much. Then the right to decide moved. According to the testimony recounted by the Commission, Thiokol senior vice president Mason asked the engineering vice president, Bob Lund, to take off his engineering hat and put on his management hat. Only the four managers took part in the final review, and Boisjoly later testified that he was never asked or polled. Lund testified that the roles had kind of switched, and that Thiokol felt it had to prove to NASA that it was not ready.

The Commission treated that reversal as a finding in its own right, writing that NASA appeared to be requiring a contractor to prove that it was not safe to launch. Hardy, a senior Marshall engineering official, said on the call that he was somewhat appalled at Thiokol's initial position and would not recommend launch over its objections. Under a default of launching unless someone proves otherwise, an engineering uncertainty became a launch decision.

Richard Feynman's appendix to the report shows how large the gap between knowledge and decision had grown. Its introduction describes estimates of vehicle loss ranging from roughly 1 in 100 to 1 in 100,000, with the higher probabilities coming from working engineers and the lower ones from management. In the separate discussion of main-engine reliability, he records 1 in 10,000 from Rocketdyne engineers, 1 in 300 from Marshall engineers, 1 in 100,000 from NASA management, and 1 or 2 in 100 from an independent consulting engineer. Those figures concern different estimates and should not be pooled into a measured shuttle failure rate. Their value here is the disagreement between people inside the same program, which Feynman examined as a problem of communication and judgment.

Jensen and Meckling's caution applies here, and it cuts against engineers too. Engineers have interests of their own. They may prefer caution because they bear blame for a failure and get no credit for a launch, and a regime that lets any engineer halt anything can stall a program for good reasons and bad ones. The design work is to decide which choices each party is best placed to make and to attach a control to each, so that the holder has a reason to use the right well.

A decision-rights table for an AI product team

The following eight decisions are the ones I would write down first. Role names will vary by company, and the structure should not: one named decider per decision, vetoes limited to people who hold a different kind of specific knowledge, and a default for when nobody has acted. I borrow the vocabulary from the RAPID method that Paul Rogers and Marcia Blenko of Bain & Company described in the Harvard Business Review in January 2006, in which roles are Recommend, Agree, Perform, Input and Decide, and the Decide role belongs to a single person.

1. Ship a model, prompt or retrieval change that passes the agreed evaluation gate. Decider: the engineering lead of the feature. No meeting is required, the evaluation owner is informed automatically, and the gate itself (metrics, slices, minimum sample sizes) was fixed before the change was built.
2. Ship a change that fails the gate or falls outside it, such as a new language, customer segment or data source. Decider: the product owner. The evaluation owner holds a veto that can be lifted only by row 5, so the product owner cannot simply override it.
3. Change a decision threshold, such as an auto-approval cut-off or an escalation confidence. Decider: the business owner who bears the cost of both error types. The data scientist recommends, states the expected error rates with an interval, and the owner records the choice and its date.
4. Override an individual prediction. Decider: the human who handles the case, with no approval step. The override and its reason are logged, and a named person reviews overrides in aggregate every month.
5. Accept a known residual risk, such as a failure mode that will not be fixed before launch. Decider: a named executive who owns the outcome and does not report to the team that built the system. The acceptance is in writing, names the specific risk and expires on a stated date.
6. Pause or roll back a live model. Decider: anyone on a published on-call list, with no proof of harm required. Restarting needs the product owner and the evaluation owner to agree after a review.
7. Stop a scheduled launch for a reason the gate did not anticipate. Decider: any named reviewer may place a hold. Lifting the hold goes to the executive in row 5, who must record why in writing, which makes an override visible instead of silent.
8. Amend this table. Decider: the executive sponsor, on a fixed quarterly review, with the incident log as input.

Three features of the table answer the Challenger record. Rows 5 and 7 separate the person who benefits from the schedule from the person who accepts the risk, so no one can impose a constraint and quietly lift it, as happened with the 51-B constraint. Row 6 reverses the burden of proof: a pause costs a restart, and ship decisions stay evidence-gated. The veto in row 2 exists to make a conflict visible and cannot win it. RAPID works the same way, since the Agree role signs off on a recommendation while the commitment to act rests with the single person in the Decide role.

What a threshold decision looks like with real numbers attached

Row 3 is the one most teams argue about, so a worked example helps. The numbers below are made up for illustration. Suppose a lender uses a model to approve small loans automatically when its score is above 0.90, and the team proposes lowering the cut-off to 0.85, which would send an extra 600 applications a month to automatic approval. Assume an approved loan that repays earns 150 in margin, and a defaulted loan loses 1,200.

If the default rate among those marginal 600 is 8 percent, each additional approval has an expected value of 0.92 times 150 minus 0.08 times 1,200, which is 138 minus 96, or +42, and the change is worth 25,200 a month. If the true default rate is 12 percent, the same calculation gives 132 minus 144, or -12 per approval, and the change loses 7,200 a month. Break-even sits at a default rate of 150 divided by 1,350, or about 11.1 percent. A one-percentage-point error changes monthly expected value by 8,100 and can flip the decision when the estimate is near that boundary; moving from 8 to 9 percent would still leave it profitable under these assumptions.

Two kinds of specific knowledge meet in that calculation. The data scientist knows how uncertain the 8 percent figure is, which depends on how the model was trained and how the marginal applicants differ from those in the evaluation set. The business owner knows what a lost margin of 150 and a default loss of 1,200 mean for the business, and how much variance the company can carry. Neither can make the decision alone. A table that assigns the decision to the owner of the cost and requires the recommender to present an interval, not a point estimate, puts each fact where it is used. If the builder alone decides, I would expect thresholds tuned to the evaluation metric; if the business owner alone decides, thresholds tuned to a quarterly target. That is a prediction from the incentives and I have no data on it.

Regulators and standards bodies already ask for this in writing

Banking supervisors asked for these written answers long before generative AI, and standards bodies and legislators have repeated the request since. In April 2011 the Federal Reserve and the Office of the Comptroller of the Currency issued SR 11-7, the guidance on model risk management that governed US banks for fifteen years. It said that effective challenge of a model depends on incentives, competence and influence, and that influence comes from a combination of explicit authority, stature within the organization and support from senior management. It said model owners hold ultimate accountability for model use and performance, that control staff should have the authority to restrict the use of models, and that they may grant exceptions to validation procedures only temporarily and subject to timelines and limits on model use. On overrides, it said a high override rate, or an override process that consistently improves performance, is often a sign that the model needs revision. Rows 4 and 5 of the table follow that logic. The Federal Reserve replaced SR 11-7 with SR 26-2 on 17 April 2026. The new text keeps effective challenge, defined as analysis by people with expertise, sufficient independence and the organizational standing and influence to effect change, and it states in a footnote that generative and agentic AI models fall outside its scope. For generative and agentic AI teams in banks, that exclusion is not an exemption from governance: the same footnote says the institution's risk management practices should determine appropriate controls. Traditional statistical and non-generative, non-agentic AI models remain in scope.

The US National Institute of Standards and Technology reaches a similar place in the GOVERN function of its AI Risk Management Framework playbook. It calls for documented risk management roles, a separation between development and testing functions so that systems can be corrected independently, and executive leadership that declares risk tolerances and delegates authority to act on them. One of its suggested documentation questions asks who receives concerns that a system no longer meets the organization's standards and whether that person can modify, limit or stop its use. That question is row 6 in plain words.

Regulation has started to require pieces of it. Article 26(2) of the EU AI Act says deployers of high-risk systems must assign human oversight to people with the necessary competence, training and authority. Article 14(4) lists what those people must be enabled to do: understand the system's limits, stay alert to automation bias, set aside or reverse an output, and intervene or halt the system with a stop control. The European Commission states that the AI Omnibus entered into force on 27 July 2026 and that the rules for high-risk systems in Annex III apply from 2 December 2027. Those dates concern that category, rather than every AI system. The text lists authority and a stop control as separate requirements next to competence. Any one can be missing while the others are present, and a reviewer who understands the system but cannot halt it is in the Challenger engineers' position again.

The bureaucracy objection is right about reversible decisions

The strongest objection to everything above is that explicit rights create approvals, and approvals create slowness. Jeff Bezos made the point in Amazon's letter to shareholders for 2015. He divided decisions into Type 1, consequential and nearly irreversible one-way doors, and Type 2, changeable two-way doors. His warning was that as organizations grow they apply the heavyweight Type 1 process to most decisions, which he said produces slowness, risk aversion, too little experimentation and diminished invention. His 2016 letter argued for deciding with around 70 percent of the information one would like, and for the practice of disagree and commit. Netflix's published culture statement describes a single accountable person for each significant decision, an informed captain, and says the company avoids decision by committee because it slows things down and blurs accountability.

I accept the objection for every decision that can be reversed cheaply, and the table is built to respect it. Row 1 contains no approval at all: a change that passes a gate fixed in advance ships on the lead's say-so, and the gate does the work a committee would otherwise do. The table covers eight decisions, and each is either hard to reverse (accepting a residual risk, shipping outside the gate) or involves a conflict of interest that a team cannot resolve by itself (stopping a launch). A rule I would apply when someone proposes a ninth row is to ask whether the decision is a one-way door or whether someone who carries the schedule would be tempted to quietly waive it. If neither is true, the decision stays with the team and does not appear in the table.

A second answer is that unwritten rights still carry bureaucratic cost, only less visibly. The meeting where nobody knows whether the product manager or the data lead decides, and the escalation that wanders for a week, consume time as well, and they consume it on the decisions that matter most. Bain reports on its own page for the Rogers and Blenko article that a survey of more than 350 organizations found roughly 15 percent making decisions effectively. That is a consultancy describing a problem it sells a remedy for, and I would not rest an argument on it. I mention it only because it points the same way as the Challenger record, and I would treat it as a hint and not as a measurement.

There are limits that the objection also exposes. First, writing a right down does not make people use it. Diane Vaughan's 1996 study of the launch decision, which I know only through a secondary summary, concluded that the NASA managers acted in accordance with NASA's rules, and that the problem lay in normalized acceptance of known flaws, schedule pressure and information thinning out as it moved up the hierarchy. A table could have been followed to the letter at Marshall. For that reason the table above contains defaults, expiry dates and an independent holder of the risk, which go further than assigning a name. Second, the evidence base is thin. The Rogers Commission is a single case examined with hindsight, SR 11-7 describes expectations and not measured outcomes, and I know of no controlled study showing that a decision-rights table reduces AI incidents. The recommendation rests on mechanism and precedent. A team that adopts it should keep the incident log that would show whether it was wrong.

The first row comes from tracing your last launch

The cheapest way to start is to take the most recent model or prompt change that reached production and trace its decisions backward. Find who approved the release, who could have blocked it, who knew the evaluation results, and who would have been told if someone had objected. Then look for the three failures the Challenger record makes easy to recognize: a person who set a limit and could also lift it, a person with specific knowledge who was not consulted at the point of decision, and a default that put the burden of proof on the person asking for caution.

Whatever that trace turns up first becomes the first row of the table. A team deploying a high-risk system covered by the EU AI Act can run the same trace against Article 26(2), on the competence and authority of human overseers, together with Article 14(4)(e), on the ability to intervene or halt the system. I cannot settle from the sources how small a team can be before the table becomes overhead. My guess is that at five people everyone sees everything and the gap between informal and written authority stays invisible until the first launch that someone wants to stop, but that is a guess.

Sources

1. Presidential Commission on the Space Shuttle Challenger Accident (Rogers Commission) (1986). Report to the President, Volume I, Chapter V "The Contributing Cause of the Accident", Chapter VI "An Accident Rooted in History", Chapter III "The Accident", and Recommendations II, IV and V. NASA History Office, nasa.gov/history/rogersrep/.
2. Feynman, R. P. (1986). Appendix F, "Personal Observations on the Reliability of the Shuttle", in the Rogers Commission Report, Volume II. nasa.gov/history/rogersrep/v2appf.htm.
3. Jensen, M. C., Meckling, W. H. (1992). Specific and General Knowledge, and Organizational Structure. In Werin, L., Wijkander, H. (eds.), Contract Economics, Blackwell, pp. 251-274; also Journal of Applied Corporate Finance (1995). Working paper at papers.ssrn.com/abstract=6658.
4. Rogers, P., Blenko, M. (2006). Who Has the D? How Clear Decision Roles Enhance Organizational Performance. Harvard Business Review, January 2006 (publication details confirmed on hbr.org; RAPID roles and the survey figure of more than 350 organizations as described on Bain & Company's page bain.com/insights/who-has-the-d/).
5. Bezos, J. (2016). Letter to Amazon shareholders for fiscal year 2015 (Type 1 and Type 2 decisions); Bezos, J. (2017). Letter to Amazon shareholders for fiscal year 2016 (two-way doors, 70 percent of information, disagree and commit). aboutamazon.com and the SEC filing of the 2015 letter.
6. Netflix. Netflix Culture (informed captain, farming for dissent, context not control). jobs.netflix.com/culture, read October 2026. The longer account is Hastings, R., Meyer, E. (2020), No Rules Rules, which I did not open.
7. Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency (2011). SR 11-7, Supervisory Guidance on Model Risk Management, 4 April 2011 (sections on effective challenge, ongoing monitoring of overrides, and governance roles; read in the archived copy of the attachment, since the Federal Reserve has withdrawn the original file). Board of Governors of the Federal Reserve System (2026). SR 26-2, Revised Guidance on Model Risk Management, 17 April 2026, federalreserve.gov/supervisionreg/srletters/sr2602.htm and attachment SR2602a1.pdf.
8. National Institute of Standards and Technology. AI RMF Playbook, GOVERN function (subcategories 1.7, 2.1, 2.3). airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern.
9. European Union (2024). Regulation (EU) 2024/1689 (AI Act), Articles 14(4) and 26(2), Official text at https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng. European Commission (27 July 2026). AI Omnibus enters into force. https://digital-strategy.ec.europa.eu/en/news/ai-omnibus-enters-force.
10. Vaughan, D. (1996). The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA. University of Chicago Press. Read only through the secondary summary at aerossurance.com (Challenger launch decision, 30 years on).
11. Illustrative threshold arithmetic: 600 additional monthly approvals, 150 margin per repaid loan, 1,200 loss per default, default rates of 8 percent and 12 percent, break-even at about 11.1 percent. All figures are made up for this essay.