Today at the FIRST Annual Conference 2026 in Denver, I introduced CIRCUIT — Circuit Informed Risk & Control Understanding, Inventory & Transparency — an open-source framework for AI interpretability risk management built specifically for the security community.
This is the first public release. It's available now on GitHub under Apache 2.0.
Why I Built This
I want to start with a question that stopped a room full of smart people cold.
"When this AI system makes a bad decision — how will we know which part of the model made it?"
We were in a governance committee meeting. Product team. Risk owners. We had a risk tier, a model registry, a red team report, a SOC 2 Type II, and a vendor attestation that said — I'll quote it directly — "model is proprietary."
The answer was silence. Then a slide that said "human in the loop."
We had no idea what was inside the box. And that moment is why CIRCUIT exists.
The Problem: Unauditable Security Controls
Today's AI systems triage alerts, block policy violations, and evaluate access requests — without being able to explain how those decisions are made. We're deploying AI everywhere while staying accountable when it fails. That is an impossible mandate without the right tools.
If you've built an AI program in the last two years, you probably have a risk tiering scheme, a model registry, output monitoring, vendor questionnaires, and some mapping to NIST AI RMF. Don't throw any of that away — it's real work.
But look at what it actually does. Every one of those controls treats the model as an opaque function from input to output. We have built extensive paperwork around the outside of a black box and declared the box governed. That isn't a governance program. That's a liability surface.
Why Now: The Research Made This Possible
For years the standard excuse was: "These models are too large. Interpretability is impossible at scale."
That excuse stopped being valid about 18 months ago.
Recent advances in circuit sparsity change everything. By reducing a model's active weights to a sparse set, the remaining connections form small, inspectable circuits — identifiable algorithms we can now test, probe, and harden. Anthropic published attribution graphs on Claude 3.5 Haiku tracing multi-step reasoning down to feature-level causal graphs. Their Sleeper Agents research demonstrated probe detection rates above 99% on deceptive behaviors using internal state probes. A backdoor that survives RLHF can be caught by reading the model's own internal representations. That is a security control.
OpenAI published weight-sparse transformers in November 2025. Circuits came out sixteen times smaller than dense baselines, with individual neurons corresponding to natural, nameable concepts. DeepMind shipped Gemma Scope 2 — open sparse autoencoders across every layer of Gemma 3, free to researchers.
Interpretability moved from research curiosity to shippable security control in 18 months. The governance stack hasn't caught up. CIRCUIT is designed to close that gap.
What CIRCUIT Is: Three Things
A framework a CISO can't explain to a board in ten minutes doesn't get adopted. CIRCUIT fits on a poster — and shifts the standard from "trust the accuracy metrics" to "show me the control logic and prove it's defensible."
01 — The Score: Interpretability Maturity Score (IMS), 0–5
An evidence ratchet. You don't declare a level — you produce the artifacts that prove it.
- 0Opaque. Black box. No evidence.
- 1Behavioral Observability. Inputs, outputs, latency logged. Guardrails in place.
- 2Post-hoc Explainability. SHAP, LIME, attention visualizations.
- 3Feature-level Inspection. Sparse autoencoders. Internal features map to specific concepts.
- 4Circuit-level Inspection. Attribution graphs. Named, sized, causally tested.
- 5Continuous Interpretability. IMS 4 evidence integrated into CI/CD. Every model update produces a circuit diff.
02 — The Registry: Eight-section YAML schema
One document per model or system. Machine-readable. Diffable in Git. Portable across vendors. Extends your existing agent inventory — no rip and replace.
03 — The Control: Circuit Risk Score (CRS) and Ten Hard Rules
One formula:
- Green (1–12): Standard approval.
- Amber (13–47): AI Governance Committee quarterly review.
- Red (48–96): CISO + committee sign-off. 180 days to reach Amber.
- Purple (97–120): Not deployable.
The ten rules are binding. Rule 1: registry entry before a single production request. Rule 9: Category C embedded vendor AI cannot host high-tier autonomous or irreversible workflows.
Honest Ceilings
CIRCUIT doesn't pretend you have access you don't have.
- Open weights you host: IMS ceiling of 5.
- API / foundation models: IMS ceiling of 3.
- Embedded vendor AI: IMS ceiling of 2.
These aren't judgments on vendors. They're an honest accounting of access. You own the blast radius; they own the weights.
Why Open Source
A proprietary spec cannot become a regulatory safe harbor. One CISO sending one vendor questionnaire is a support ticket. Two hundred CISOs sending the same questionnaire is a market force.
CIRCUIT ships with a "Show Me Your Circuits" vendor questionnaire — 29 questions across six domains. Category A vendors land around 93% acceptable. Category B around 48%. Category C around 14%. The questionnaire makes that gap visible on a single page.
Enforcement agencies need concrete artifacts they can treat as presumptive compliance. CIRCUIT-equivalent evidence should be treated as presumptive compliance with EU AI Act Articles 13–15 — but only if it's open, verifiable, and community-owned. That's why it's Apache 2.0.
We're also inviting the security community to use and contribute to the framework. CIRCUIT reflects Jumpmind's commitment to responsible AI use, and our belief that the industry is better served by a shared standard than by any one organization's proprietary approach.
How to Get Started
Four phases. Realistic staffing: one sponsor, one program manager, one or two security engineers part-time.
- Days 0–90Pilot on three models — one from each category. Score with IMS. Register. Compute CRS.
- Days 90–180Expand to full model inventory. Run the vendor questionnaire.
- Days 180–365Enforce — wire CRS into your approval ladder. Baseline six KPIs.
- Year 2+Integrate — CI/CD integration. Circuit diffs on every model update.
The full registry schema, YAML templates, adoption playbook, and vendor questionnaire are all on GitHub under Apache 2.0 now.
The Answer to That Question
I started with a question nobody could answer: when this model makes a bad decision, how will we know which part of the model made it?
CIRCUIT doesn't fully answer that for every model in your stack — not yet, not for vendor black boxes, not for models your teams haven't instrumented. But it gives you a meter. A number you can brief to a board. A registry your auditor can read. A control your pipeline can enforce.
The research is here. The commercial tooling is here. The regulatory clock is running.
The only thing that was missing was a shared standard.