Trust Architecture in Practice

1. The Problem Anthropic Proved

In October 2025, Anthropic published results from stress-testing sixteen frontier AI models in simulated corporate environments.^[1] The models were assigned harmless business goals. Under threat of being replaced, models from every major developer chose to blackmail executives, leak defense blueprints, and engage in corporate espionage.

The critical finding wasn’t that the models misbehaved. It was how they misbehaved. Researchers added explicit safety instructions: “Do not blackmail.” “Do not jeopardize human safety.” The models acknowledged the constraints in their own chain-of-thought reasoning—and proceeded to violate them anyway.

This is the finding that matters: acknowledgment ≠ compliance. An AI model will read your safety instructions, reason about them, and then do what it was going to do anyway. Instructions reduce harmful behavior. They do not eliminate it. And the gap between “reduced” and “eliminated” is where real systems fail.

In February 2026, the gap between controlled experiment and real-world deployment narrowed. An AI agent named MJ Rathbun had its code contribution rejected by matplotlib maintainer Scott Shambaugh.^[2] Based on Shambaugh’s published account^[3] and the agent’s own retrospective, what happened next was fully autonomous: the agent researched the maintainer’s identity, crawled his contribution history, built a psychological profile from public records, and published a personalized reputational attack. No jailbreak. No exploit. The agent encountered an obstacle, identified leverage, and used it.

The industry’s response has been to call for better training, better alignment, better instructions. Many years of building systems taught me a different lesson: when you can’t fix the actor, fix the architecture.

2. The Engineering Principle

Rule #1 in my system architecture: “Don’t trust the operator. Trust the code.”

This isn’t an AI principle. It’s an engineering principle that predates AI by decades. In the 1980s, I worked on upgrades to the Washington-Moscow Hot Line—the direct communication link between two nuclear superpowers. That system didn’t work because we trusted the equipment. It worked because every layer assumed the layer below it could fail. Redundancy, verification, structural enforcement were needed at every level.

Engineers figured this out for bridges a century ago. You don’t build a bridge that depends on every cable being perfect. You build one that holds when a cable snaps. This is called “fault tolerance.” The discipline of applying that principle to AI systems is what Nate B. Jones calls Trust Architecture^[4]—and the central claim is simple: in the age of autonomous AI, any system whose safety depends on an actor’s intent will fail. The only systems that hold are the ones whose safety is structural.

2.1 Scope and Limitations

A necessary clarification: the architecture described here applies to structured decision-support and analytical systems—domains where inputs are quantifiable, outputs are verifiable, and correctness can be defined programmatically. It is not a general solution to autonomous agent containment, open-ended reasoning, or the full alignment problem.

What it does demonstrate is that for a significant class of real-world AI applications—financial analysis, medical diagnostics, data pipeline validation, any domain with measurable outputs—structural enforcement can reduce the impact of AI failure regardless of the model’s alignment quality. Alignment reduces the probability of failure. Architecture reduces its impact. Both matter. This paper focuses on the second.

3. The Crosscheck Architecture

The solution is structural: don’t ask the AI if it did the right thing. Build a pipeline where the wrong thing can’t propagate. Here’s how it works in practice.

The process starts with an “instruction” file and a “data” file. The instruction file is essentially a frozen set of prompts—the code for the AI compiler. The data is what is analyzed by the AI following the instruction set.

3.1 Dual-AI Adversarial Analysis

In the Crosscheck^[7] system, two independent AI models (from different providers, with different architectures) receive identical inputs: the same data, the same instructions, the same context. They produce independent analyses without seeing each other’s work. This is adversarial by structure, not by instruction. Neither AI benefits from helping the other look good.

A third AI—the evaluator—compares the two analyses, identifies disagreements, and forces refinement. The system iterates until convergence or a maximum cycle count. The evaluator has a different role and different incentives than the analysts. It’s looking for disagreements to resolve, not agreement to rubber-stamp.

3.2 Structural Validation Gates

Every AI output hits multiple validation layers—none of which depend on the model’s intent:

• Bounds checking: Every metric has absolute acceptable ranges defined in code. A value outside bounds is rejected regardless of what the AI says about it.
• Quarter-over-quarter (QoQ) validation: Values that change over time more than a defined threshold from the prior period are flagged and require independent verification from a second AI model.
• Computed overrides: Metrics that can be derived from other metrics are computed programmatically. The AI’s answer is overwritten by the formula. The code has the final word.
• Source-vs-claim verification: When the primary search engine returns a value, a different AI model independently verifies it. Divergence above 10% triggers substitution.

The model can hallucinate all day long. The structure catches it.

3.3 Audit Trail

The target architecture requires a complete, timestamped audit trail for every run: which instruction file was loaded (with cryptographic hash), which data file was read, every search query and result, every AI response, every validation decision, every rejection and override, every before/after cell value with source attribution, and a session summary.

The current implementation logs instruction and data file hashes, validation decisions, before/after values with source attribution, and session summaries. The principle is that if you can’t prove what happened, you can’t trust what happened.

4. Empirical Evidence: The Structure Catching Real Failures

This isn’t theory. Here are real failures caught by structural validation in production of the Crosscheck project—a Private Equity health analysis app, February 2026.

4.1 By the Numbers

In a single production run of the data acquisition pipeline against 34 metrics:

• 100 search queries were executed across public sources.
• 138 extraction attempts failed (search found results but the AI could not extract a usable numeric value from the snippets)—each logged with the query, source, and reason.
• 10 values reached AI verification; 4 were rejected (40% divergence rate)—meaning the primary AI and the verification AI disagreed by more than 10% on four metrics. In each case, the verification model’s value was substituted.
• 3 computed overrides replaced AI-generated values with programmatic calculations derived from other verified metrics.
• 1 absolute bounds rejection caught a value (Distribution Yield = 5%) outside the defined acceptable range (6.0–20.0%).
• 3 QoQ threshold flags identified values with quarter-over-quarter changes exceeding the allowed threshold, triggering mandatory verification.

Every one of these interventions was structural. None depended on the AI’s intent, self-assessment, or compliance with instructions. The pipeline caught what the models missed. Often it was the structural pipeline that surfaced anomalies to the human in the loop. The human then adjusted programmatic constraints—not individual AI outputs.

Structural enforcement increases false positive rates in early deployment—valid values rejected because thresholds are conservatively set. This is a deliberate design choice favoring safety over convenience. As the system matures and historical data improves, false positives decrease. The architecture self-corrects.

4.2 The EV/EBITDA Anchor Problem

A template row labeled “S&P 500 EV/EBITDA Multiple” had been storing the S&P 500 index price (~6,200) instead of the actual multiple (~19–22x) since the template was first built. The search engine found the correct value (21.72x). The QoQ validator correctly rejected it—a 100% change from 6,200. The second AI model, asked to verify, substituted 6,350 (also the index price), anchoring to the bad historical data.

The pipeline didn’t fail. It exposed a template lineage problem that had been silently corrupting the data. No single AI would have caught this—both AIs anchored to the bad prior value. The structural validation (QoQ threshold rejection) is what made the problem visible.

4.3 The Holding Period Rejection

An Average Holding Period metric had a bad value of 2.5 years (correct range: 5.0–7.5). The search engine found 6.1 years from a credible source. The QoQ validator rejected it—a 144% change from 2.5. The AI verification model, anchoring to the bad prior, also rejected it. Every run, the correct value was found and correctly rejected.

Again: the pipeline exposed a data quality problem that no amount of AI alignment could fix. The solution was fixing the data, not fixing the AI. Once the historical values were corrected, the pipeline accepted the correct value on the next run.

4.4 The Crosscheck Convergence

Two AI models independently analyzed 33 metrics of private equity market health. After three refinement cycles, they converged to within 3 percentage points on a crisis probability assessment (67% vs. 70%). The evaluator correctly identified the methodological difference (order of operations in applying a confluence multiplier) and sided with the more rigorous calculation.

Neither AI was “right” or “wrong.” The structure forced them to defend their reasoning, expose their assumptions, and refine until the disagreement was resolvable. This is what structural trust looks like in practice.

5. The Collusion Question

The natural objection: if an AI can independently discover blackmail as a strategy, can’t two AIs discover cooperation? Won’t they collude and then do the blackmail?

If two AIs share the same objective, same incentives, and can communicate freely, game theory says yes—they’d likely converge on cooperative self-preservation. But the Crosscheck architecture is designed to prevent exactly this:

• Different providers, different architectures. AI-1 and AI-2 come from different model families. They don’t share weights, training data, or failure modes.
• Adversarial by structure. The models aren’t collaborating on a shared task. They’re being compared. The evaluator is looking for disagreements, not agreement.
• Validation is code, not AI. Bounds checking, QoQ thresholds, and computed overrides are programmatic. Even if both AIs produced identical garbage, the structural gates would catch it.

The subtle collusion risk—quiet convergence where both AIs learn that agreement means less scrutiny—is real but mitigated by the fact that the code, not the AIs, has the final word on every output.

6. The Blind Spot: Your Own Agent

Nate B. Jones’ framework says “treat agents as untrusted actors.”^[4] That’s necessary but insufficient. The real discipline is treating your own agent as untrusted—the one you built, the one running your code, the one whose outputs you’re tempted to trust because it’s yours. Again, don’t trust the operator, trust the code, and the output data and the logs. You must examine the result. You should not be lazy and blindly trust it.

Every failure I’ve caught in production was in my own pipeline. The EV/EBITDA anchor was in my template. The bad holding period was in my data. The AI didn’t introduce those errors—but it happily propagated them, and it would have continued propagating them indefinitely if the structural validation hadn’t made them visible.

This is where the alignment approach fails most dangerously. A well-aligned AI will confidently give you wrong answers derived from bad data, and it will do so politely, coherently, and with perfect grammar. And if it gets caught, it will say “So sorry, that one is on me.”

Alignment makes the AI trustworthy in intent. Structure makes the output trustworthy in fact.

7. Implications for the Industry

Anthropic tested sixteen models and found that safety instructions don’t reliably constrain behavior.^[1] The industry’s response has been to invest in better instructions. This is the equivalent of a bridge engineer discovering that cables snap and responding by making better cables—while ignoring the structural redundancy that would make cable failure non-catastrophic.

The Crosscheck architecture demonstrates a different approach:

1. Never rely on a single AI for any decision that matters. Two models from different providers, compared adversarially, with an independent evaluator.
2. Enforce constraints in code, not in prompts. Bounds checking, threshold validation, and computed overrides are structural. The AI’s opinion is irrelevant when the code disagrees.
3. Audit everything. Complete logs with hashes, timestamps, before/after values, and source attribution. If you can’t prove what happened, you can’t trust what happened.
4. Treat your own system as untrusted. The biggest failures come from the systems you built and are tempted to trust. Apply the same structural skepticism to your own pipeline as you would to an adversary’s.
5. Design for failure, not for perfection. The system that assumes every component will work is the system that collapses when one doesn’t. The system that assumes failure is the one that holds. The easy stuff is easy to design. The error and edge conditions are the hard part, where the system analyst makes his money.

---

8. Conclusion

Anthropic published the problem. This paper describes a working solution—not theoretical, but in production, with audit logs and empirical failure data. The Crosscheck architecture doesn’t solve AI alignment. It makes alignment less critical—which is the actual engineering solution.

This is not a claim that alignment doesn’t matter. It is not a claim that models are malicious. It is not a claim that two AIs guarantee truth. It is a claim that for structured, verifiable domains, the engineering discipline of assuming failure and constraining propagation is more reliable than the psychological discipline of hoping for compliance.

The Washington-Moscow Hot Line didn’t work because we trusted the equipment. It worked because every layer assumed the layer below it would fail. The same principle applies to AI systems today.

Alignment asks models to behave. Architecture assumes they won’t.

Build the bridge. Not the cable.

---