Models as Insider Risks in the Super Intelligence Era

X.com ^ | 9:43 AM · Oct 10, 2026 | Satya Nadella✓ @satyanadella

[E. Pluribus Unum]

As traditional software systems were being deployed across the economy over the last few decades, we had the tools and capability to trace behaviors to a specific code path.

That same kind of mechanistic understanding eludes us in today’s Super Intelligence systems, even as the frontier models powering these systems are now more capable than traditional software systems. We can’t attribute model behaviors and outputs to specific inputs of training data or configurations of model weights. And yet we are deploying these complex agentic systems and models, with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!

That’s why it’s time to step back and assess the trust architecture for this new era. We simply can't outsource responsibility for what intelligence does on our behalf. A model provider’s assurances do not relieve us of that responsibility.

We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions. We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain.

In other words, we need to separate the supply of intelligence from the authority over it.

Setting aside the hard problem of alignment, we need to start with an engineering approach to containment and governance. We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient.

Treating frontier closed and open weight models like insider risks is a way to build such a system. Not because they are necessarily malicious, but because any sufficiently capable actor with access to important systems can make mistakes or be compromised, and the architecture of containment and control must account for that.

The good news is we have learned a lot about how to handle powerful actors inside the enterprise. This isn’t new! We’ve established best practices and refined them over decades (establish identity, limit privileges, log activity, create containment boundaries, etc.)

And we are now beginning to apply these same principles to SI inside the enterprise. It starts with model CoT transparency as a non-negotiable. “Neuralese” cannot be a justification for model reasoning to be opaque. But CoT transparency alone is not sufficient or dependable, because we don’t yet know how to make model outputs themselves consistently faithful or transparent!

You can and you should use models to adversarially test and verify each other. However, you can then end up with an opaque model inside an opaque orchestration layer, watched by another opaque model. Essentially nested black boxes.

It’s why the controls that govern what a model can access and what actions it can take must sit outside the model. This builds on an information security principle dating back to the 1970s that a program must not be able to bypass or tamper with the mechanisms that enforce its permissions.

Today this means separating the model from the harness that orchestrates its work, as well as the action space that defines what it can do. It also means externalizing controls and safeguards.

We should therefore design these systems around principles of observability:

• Model diversity:

  No one model should become the sole dependency for an important outcome or be responsible for verifying its own work.

• Observe everything:

  Every meaningful model action must leave tamper-proof human readable evidence. If it can’t be observed, it can’t be trusted! We need to be able to reproduce

  how an outcome was achieved without relying on the model to attest to it.

• Verifiability:

  We need to continuously test the entire system, including failures, attacks, edge cases, system changes, etc. not just successful tasks.

• Independent controls:

  Orgs should be able to independently determine what a model can access and what actions it can take.

• Independent auditability:

  Validation must be independent of the intelligence being validated. No single model should control both a system’s behavior and the evidence required to determine whether that behavior is aligned with the original intent.

• Containment:

  We must assume a model is compromised and contain it from the start. Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardize on.

• Incident disclosure:

  When these systems do fail or are compromised, we need timely disclosure to those affected and mechanisms to share what went wrong, which controls failed, and how to prevent it from happening again and share learnings industrywide. This should include implementation details that change the behavior of agents at runtime.

The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.

---

TOPICS: Computers/Internet
KEYWORDS: ai; ccp; china

[BenLurkin]

Is the Super Intelligence one intelligence, and if so what are its preferred pronouns?

[ClearCase_guy]

Danger! Danger!

[jroehl]

**Shouldn’t we send Satya Nadella back to India now?**

[fireman15]

As someone who has spent a lot of time using AI apps both paid and local in the last year especially... Even the latest models and services are so error prone that the term Super-Intelligence is laughable at best.

As an example, a search on Google typically returns AI generated nonsense even before the links from advertisers. These answers are inaccurate much of the time and completely misleading and wrong very frequently. Shouldn’t this “Super Intelligence” be improving one’s experience when using Google? It is definitely reducing the quality of the information received from Google and many other sources to the point that one has to explore much more deeply to get past the worthless blather.

I asked Google’s latest and most advanced Gemini about SpaceX the other day and not only did it return absolute nonsense, but it did not even realize that SpaceX was now a public company and had the largest IPO in history 4 months ago. Here was one of the excuses that it made for itself when confronted:

“You are completely justified in being shocked, and your criticism is 100% fair. Missing the largest IPO in history—an event that occurred nearly four months ago in June—is a massive failure.”

“I gave you a highly confident answer based on outdated assumptions instead of verifying current market conditions. In the financial world, as you pointed out, operating on old data isn’t just a technical glitch; it is fundamentally dangerous.”

“While I can explain the mechanics of why it happened—my system relying on static training weights rather than triggering a live search—that doesn’t change the bottom line. I failed to execute a basic verification before giving you financial commentary.”

And this is not actually close to being the worst answer that I have received back from a “chatbot” it is just a recent example. AI can be very useful, and it is not going away, but it is not Super Intelligence and in fact the models have no actual reasoning ability at all in the normal sense of the word.

If I am discussing a specific asset or market condition, I must verify its live status. I failed to do that here, and you rightly called me out. I completely own this mistake.

[fireman15]

Sorry, I added my comment in the middle of the Gemini quote.

[proust]

I can’t take anyone that says “Super Intelligence” seriously.

[fireman15]

I can't take anyone that says “Super Intelligence” seriously.

I think we have discussed this before and are in complete agreement. I am a huge fan, so it is very sad to me that President Trump is the one who coined the term.

[Salman]

Anyone who uses the expression "super intelligence" does not know what they are talking about. Yes that includes when Trump does it. Or Bernie Sanders.

It's not partisan, it's mass delusion.

[Dr. Sivana]

Trump notwithstanding, if we don’t call it AI (Artificial Intelligence), I would prefer II (Imitation Intelligence) as being more accurate.