Where AI Systems Actually Break: At the Interfaces

Maggie Nanyonga · 2026-07-21 · Artificial Intelligence, Software Architecture, MLOps, AI Infrastructure

A working method for writing interface contracts at AI stack boundaries, shown against five seams where a missing clause cost real money.

A working method for writing down the contracts your stack already depends on.

This is a companion to The AI Stack Is a System; that piece made the argument, and this one is the tool: what actually goes on the page when you sit down to write a contract for a seam, shown against five boundaries where real organizations paid real money for a missing clause.

Borrowed tools, declared

Nothing about writing contracts for software boundaries is new, and it matters to say so precisely. Bertrand Meyer's design by contract gave the discipline its name in the 1980s: a routine declares preconditions, postconditions, and invariants, and a caller who meets the preconditions may hold the routine to the rest. Consumer-driven contract testing carried the idea across team boundaries: the consumer publishes its expectations as executable checks, the provider runs those checks in its own build, and a breaking change fails on the provider's desk instead of in the consumer's production. Add the SLA tradition from operations, which NIST's generative-AI profile carries to these systems when it recommends "well-defined contracts and service level agreements" across the value chain, and you have the full ancestry.

What I call an interface contract* extends that lineage; it does not coin anything Meyer would fail to recognize. (As in the flagship, an asterisk marks one of the four load-bearing terms I coined for this research program because the thing named had no settled name; the flagship's glossary defines each with its nearest established relative.) The extension is in what the parties have to pin down. A method precondition constrains an argument. An AI seam contract constrains an embedding version, a cache lifecycle, a reward composition, a policy schema, the freshness of a feature, the validity window of an evaluation verdict. The types are stranger. The grammar is the same.

The research program behind these articles wrote fourteen such contracts, one per consequential boundary in a five-domain* map of a production stack, then built twenty-two failure chains, most from operators' own postmortems, filings, and incident reports and the rest from non-AI analogs or derived from the contract clauses, and mapped every chain onto the contracts it crossed. Five of the fourteen have enough observed wreckage behind them to teach the method by example. The other nine get a table.

What goes on the page

The program's full contract form runs to sixteen fields. Strip it to what a working team needs and six parts remain.

The shared outcome. The thing neither side can deliver alone: training-serving consistency, task completion across a whole session, an unbroken audit trail. If you cannot name one, you have two components and no seam worth a page.

Requirements, in both directions. What side A requires from side B, and what B requires from A. Always both directions. A one-directional list is an API doc wearing a contract's clothes.

Guarantees and constraints. What each side promises to deliver, and what each side imposes on the other. In AI stacks the constraints flow upward as often as down.

Evidence. What demonstrates the contract is being kept, including at least one metric that can fail while both components' own metrics pass. If every number on the page belongs to one side, you have measured the boxes and left the line between them dark.

Ownership. One accountable owner for the seam, chosen by whose contract the seam fulfills; the hosting side named separately; and the rule that failure classification follows the breached clause, not the location of the process that broke.

Revalidation triggers. The named changes that force the contract to be re-verified, scoped to changes in the seam's semantics, so the trigger fires rarely enough to be obeyed; vendor churn does not qualify. This generalizes the retest discipline NIST already recommends when models are retrained or data sources are added.

Two clauses ride along with every contract and carry the checking machinery into the seam: a gate clause, what must be re-verified before an artifact crosses, and a monitoring clause, what evidence must flow back after it crosses. The flagship covers why; here they simply appear in each walkthrough.

Now the five contracts, each anchored to what happened when a clause of it did not exist. Each walkthrough leans on the part of the method that incident teaches best.

1. The training-data seam, or: requirements travel in both directions

Between data infrastructure and the model. Taught by Unity, 2022.

Unity's Audience Pinpointer priced ad targeting, and the prepared remarks for the company's May 2022 earnings call disclosed two problems behind a bad quarter: a fault in the platform that reduced Pinpointer's targeting accuracy, and the lost value of a portion of its data training, due in part, in the remarks' words, to "ingesting bad data from a large customer." That second problem is this seam's territory. The bad data was schema-compatible enough to load. Ingestion succeeded, training completed, the model served, and the first instrument to register anything was the revenue line, which "significantly slowed down in February and March." The bill Unity put on the two issues combined: approximately $110 million across 2022, a cut to full-year guidance, and a recovery that had to run in strict sequence, data rebuilding, then model retraining, then revenue.

Write the contract this incident lacked and most of it is the requirements section. The model side must declare its consumption semantics: which sources, features, and schema each training pipeline reads, and it must record which dataset versions each run actually consumed, so a bad ingest is traceable and revertible. The data side owes versioned snapshots that can be pinned and re-derived, point-in-time-correct joins so future values cannot leak into training, and, the clause whose absence Unity paid for, a validation verdict on data before it crosses: schema and distribution checks per source, with lineage granular enough that one customer's feed can be quarantined without rebuilding the world.

Notice what the requirements did for the diagnosis. Nothing in Unity's pipeline failed by its own definition. Every component did its job on data it had no obligation to distrust, because the obligation had never been written. Unity's own remediation, "monitoring, alerting and recovery systems," is this contract's monitoring clause, written after the invoice instead of before.

The evidence field here is the per-source, per-feature skew statistic: a number that measures the boundary itself, and the canonical example of a metric that fails while both components pass. Ownership: data infrastructure is accountable for delivery and validation; the model team owns the retraining response when the trigger fires. And the trigger is the oldest one in the book, straight from NIST's retest recommendation: a new data source is a seam-semantics change. One large customer's feed qualifies.

2. The context-assembly seam, or: guarantees are declared semantics

Between the model and the application. Taught by Claude Code, 2026.

Between March 4 and April 20, 2026, three application-side changes degraded Claude Code while the models underneath stayed unchanged. The sharpest was a caching bug: an optimization meant to cut the cost of resuming idle sessions by clearing old thinking sections after an hour of idle time instead cleared them on every turn for the rest of the session. The model lost its accumulated reasoning mid-session and came across forgetful and repetitive. The bug survived review and testing because it only manifested in long-idle sessions, a state outside the tested envelope, and per Anthropic's postmortem it was users of the /feedback command who ultimately surfaced the problems. (Reading that as a gate that never fired is this article's framing, not Anthropic's.) A separate system-prompt change limiting response length tells the same story from another angle: it shipped after weeks of internal testing with no regressions in the evaluations Anthropic ran, and only the investigation that followed the user reports, running ablations against a broader set of evaluations, found the 3 percent quality drop that got it reverted in the April 20 release.

This seam is where guarantees and constraints earn their place on the page, because each side holds semantics the other cannot see. The model side publishes constraints the application has no way to discover from its own code: context-window limits, cache lifecycle rules and their pricing, the positional behavior that makes document ordering a quality decision, and announced version changes, since all of the above are model-specific. The application side owes prefix-stable request assembly where caching is claimed, round-trips the model's per-turn artifacts intact, and owns the compaction policy and all cross-turn state: history, buffers, memory.

The broken clause is easy to state once you look for it: the declared cache semantics said clear after one idle hour, and the actual semantics were clear every turn. Two true statements about the same component, and the gap between them ran in production for weeks. So the contract adds a clause I hold as author position, since no source yet documents it as practice: every compaction or cache-clear is an emitted, recorded event, never a silent state change. The raw material exists; cache accounting arrives on every API response, and an assembly layer that does not read those fields is blind to its own economics as well as its own bugs.

One more lesson rides on the shared outcome. The unit of this contract is the loop, the whole multi-turn session, which is exactly why per-call testing missed a fault that only long-idle sessions could express. Ownership follows the same split as the semantics: the application side is accountable for assembly, state, and compaction; the model side owns every published constraint clause, and a breach of a published cache semantic is the model side's incident even when it surfaces as application cost.

3. The model-evaluation seam, or: evidence has to be independent

Between the model and the evaluation plane. Taught by the GPT-4o sycophancy rollback, 2025.

OpenAI's April 25, 2025 update to GPT-4o added a reward signal built from thumbs-up and thumbs-down data. By the company's own postmortem, the changes in aggregate weakened the influence of the primary reward signal that had been holding sycophancy in check. Offline evaluations looked good. A/B testers liked the model. Some expert testers said the behavior felt slightly off, and the launch decision weighed the numbers over the feeling: the wrong call, as OpenAI put it. The model shipped, users met a system that validated doubts and fueled anger, and the rollback began April 28, taking about a day. The postmortem's decisive sentence: "We also didn't have specific deployment evaluations tracking sycophancy."

This seam's contract is mostly about evidence, and the incident shows three evidence clauses at work by their absence. First, coverage is declared per gate: the gate states which behaviors it measures, so an unmeasured behavior shows up as a visible hole in the contract. Nobody had to argue after the fact about whether sycophancy was tested; the contract would have said, in advance, that it was not. Second, baselines are version-scoped and the evaluated artifact must hold still: a verdict pins the exact model version and configuration it judged, and a version change voids the comparison. Third, and this is the clause the incident makes vivid: the acceptance evidence must be independent of the training signal. Thumbs data trained this model and then, through the A/B results, helped clear it for launch. A signal on both sides of that line does not check the loop; it is the loop, grading itself.

The contract also gives qualitative expert judgment standing as an evidence class, which sounds soft until you watch it outperform every quantitative gate in this incident. OpenAI's stated fix, committing to block launches on qualitative flags even when the metrics look good, writes that clause into their process.

The end-to-end metric for this seam is gate-production divergence: for the same model version and the same behavior, the delta between what the gate scored and what production monitoring sees. It is the number that tells you whether your evaluations predict anything. Ownership: the evaluation function is accountable for the verdict and custodially owns the evidence behind it; the model side hosts the runs and owes announced versions and telemetry; and a release exception is a governance action with a record; the model team cannot grant one to itself. One honest ambiguity stays open in the program's files: when the evaluator and the model builder are the same organization, whether a self-issued verdict satisfies any of this.

4. The governance seam, or: ownership is the load-bearing field

Between the governance plane and every domain it touches. Taught by Google Service Control, 2025.

On June 12, 2025, a quota policy row containing unintended blank fields was inserted into the regional datastore that Google's Service Control reads. Because quota management is global, the malformed row was "replicated globally within seconds." Service Control, the chokepoint that checks policy on every API request, hit a null pointer on the blank fields and crash-looped in every region at once. The failing code path had shipped two weeks earlier without error handling and without a feature flag, never exercised until real policy data reached it; Google's incident report is blunt: "If this had been flag protected, the issue would have been caught in staging." External API requests returned 503s across dozens of Google Cloud and Workspace products for about three hours, and the largest region took up to two hours and forty minutes to fully resolve because the restarting tasks herded onto the very datastore that fed them, with no randomized exponential backoff. Google's own status page ran on the infrastructure that was down, so the first public incident report took an hour.

Every field of a contract shows up in this chain, but the one doing structural work is ownership. The governance plane owns exactly three things at this seam: policy intent, audit state, and compliance evidence. The machinery that enforces policy lives physically inside the domains it governs, and the placement that makes a chokepoint able to enforce is the same placement that makes it able to stop the fleet. That dual character is why the contract applies the dual-membership ownership rule*: the enforcement point gets one accountable owner (the plane whose policy it enforces), a separately named host, and classification by breached clause, so a crash like this one is simultaneously a governance incident by the policy clause and a serving incident by the availability SLO. Both get routed; neither can close alone.

The deepest ownership finding is reflexive. The component that gates every other artifact in the stack was itself deployed without a gate: without a feature flag and without any validation between a policy write and its planet-wide replication. So the contract's least negotiable clause: enforcement points are themselves evaluated artifacts, carrying their own gate and monitoring clauses, and the failure of one automatically flags every assurance verdict that depended on it. Around that sit the clauses Google's remediation list now implies: policy data validated at the write boundary before replication, propagation velocity declared and incremental, a deliberate fail-open-or-fail-closed decision for each check class, and restart backoff declared as contract behavior. And because the monitoring shared the outage's fate, the seam keeps the rule that silence from an emitter is itself an alarm; a dead monitor and a quiet healthy system must never look alike.

5. The feedback seams, or: triggers keep contracts alive

Between production and everything upstream of it. Taught by two incidents you have already read.

The fifth contract is really five small ones, and it exists because "feedback" in the singular is a diagram decoration. The program types the return flows into five feedback paths*, each with its own producer, owner, payload, freshness, and characteristic way of dying: monitoring telemetry to the evaluation function; retraining data to data infrastructure; incident and audit evidence to governance; evaluation verdicts back to release gates; and user or business outcomes to the application. The register as a register is my construction, held as author position; its parts are established practice under names like continuous training and continuous monitoring, and its founding hazard was named by Sculley and colleagues a decade ago: hidden feedback loops and undeclared consumers.

You have already seen two of these paths fail in this article, in opposite directions. Unity's outcome path worked, in the sense that revenue eventually reported the model's degradation, but it ran at quarterly speed with no faster instrumented path in front of it, so the news arrived as a guidance cut. OpenAI's failure was the inverse: a production path that was too connected, user feedback wired directly into the reward composition with no check standing between the outcome signal and the learned behavior. One loop too slow and one loop unguarded, and the same register catches both, because each typed path carries a declared freshness and a declared loss. The application-outcome path is contractually the slowest and lossiest of the five, and writing that down is the point: nobody downstream gets to treat a quarterly, confounded signal as a real-time complete one.

This contract is also where revalidation triggers do the most work, because feedback paths are where change is constant and silent. A change to a reward composition, a retraining trigger condition, or a label schema fires retest before the next promotion, which is precisely the trigger the April 25 update never fired. A change to what counts as an outcome fires re-verification of everything that consumed the old definition. And the register carries a discovery rule: a production-to-upstream flow found operating outside the five named paths must be typed into the register before it may keep operating. An undeclared loop is an incident that has not landed yet.

The program's files record one gap without resolving it: the human labeling capacity that bounds how fresh retraining data can ever be belongs to no domain in the map.

The other nine seams

Same method, compressed. Chain IDs resolve to the incident appendix of the flagship; each row names the clause I would write first.

| Seam | What the contract pins | First clause to write | Crossed by |

| --- | --- | --- | --- |

| Physical foundation ↔ orchestration | Device capacity as truth: advertised = allocatable = usable | Health deltas are pushed, and silence from the device plugin is itself a breach | CH05, CH06, CH07, CH09, CH10 |

| Physical foundation ↔ data infrastructure | All table-state change through one atomic commit primitive; store consistency envelope declared | Write-boundary validation before anything replicates | CH07, CH08, CH11 |

| Physical foundation ↔ model | The accelerator memory/interconnect envelope; numerics and precision behavior per device class | Removing a precision workaround is a seam event that forces re-verification | CH01, CH02 |

| Physical foundation ↔ applications | Locality and residency declarations (deliberately thin; most obligations run through other seams) | The residency clause, the day regulation touches you | CH09, CH10, CH11 |

| Orchestration ↔ data infrastructure | Workload submission, quota semantics, exactly-once effect of retried pipeline work | The missed-window clause: a "running" DAG with pending pods is a declared contract state | CH05, CH06, CH07, CH08 |

| Orchestration ↔ model serving | Preemption and drain as noticed, priced operations; pressure telemetry without request internals | What one drain event costs the serving side, in writing | CH01, CH06 |

| Data infrastructure ↔ retrieval | The embedding-version pin as queryable index metadata; rebuilds with an acceptance gate | The pin itself. No observed public incident has crossed this seam yet; the chain here is derived from the contract clauses, which is the argument for writing yours before the observation arrives | CH22 (derived) |

| Data infrastructure ↔ evaluation plane | Evidence custody: every verdict re-derivable from pinned versions | The evidence-write check, because a failed write silently voids a passed gate | CH05, CH11, CH13 |

| Applications ↔ evaluation plane | Verdicts bound to the exact (prompt, model, parameters) triple; task-horizon gates | Registry-path compliance: no prompt or config edit reaches production around the gate | CH03, CH04, CH14, CH15, CH16 |

The template, distilled

One page. Six questions, answered in writing, for one seam.

What outcome do the two sides only deliver together? One sentence. This is the contract's reason to exist.

What does each side require of the other? Both directions, even where one direction feels obvious. The obvious direction is where Unity's clause was missing.

What does each side guarantee, and what does each impose? Declared semantics, not folklore: lifecycles, versions, limits, pricing, ordering effects. If a behavior matters and is not declared, the contract's job is to force the declaration.

What evidence shows the contract holding? At least one metric that measures the line itself and can fail while both sides' own numbers pass. Add the gate clause (what gets re-verified before an artifact crosses) and the monitoring clause (what evidence flows back after).

Who owns it? One accountable owner by contract fulfilled; the host named separately; classification follows the breached clause. If the seam produced an incident tomorrow, this field says who closes it.

What changes force re-verification? Named triggers, scoped to seam semantics so they stay rare enough to be obeyed, with explicit non-triggers where confusion is likely.

Then the ask, which costs no budget line and requires no procurement: write one contract for your worst seam this week. Choosing the seam is easier than it sounds. It is the boundary whose last incident took longest to assign, or the one where you cannot currently say who gets paged, or the one where a version can change on one side without the other side hearing about it. One page, the two owners in a room, and an hour of disagreement, which is the deliverable: the first draft will surface a requirement one side sincerely believed the other had already accepted, and that discovery, made on paper, is the entire return on the hour.

Every incident in this article was, at bottom, a contract that existed only after the postmortem wrote it. The decision in front of you is not whether your seams will have contracts. They will. It is whether you write them, or production does, at production's prices.

Sources

The full evidence base (all twenty-two failure chains with their primary sources) is in the appendix of the flagship essay ([FLAGSHIP-URL]). The five documents this article draws on directly:

— Maggie Nanyonga