The 7% That Does the Work

What the Model Processes vs. What It Can Access

The central finding is a functional distinction — and the word functional is doing real load-bearing work here.

Inside a language model there is a small, privileged set of representations available for report, deliberate manipulation, and reasoning. Beneath it sits a much larger volume of automatic processing that never surfaces. The analogy the authors reach for is access consciousness — a term from neuroscience that describes not subjective experience, but the functional availability of information for flexible use.

I want to be precise here, because the secondary press mostly wasn’t: the paper takes no position on whether models have subjective experience. None. It is explicit about this. The concept is purely functional — a description of what information is available for use, not what it feels like to process it. Every headline that used the word “consciousness” without that qualifier missed the point. The qualifier is the point.

Not all of what a model processes is equally available to the model itself.

A New Lens, a Real Object

The tool the researchers built is called the J-lens — the Jacobian lens. For each token, it identifies a vector that encodes the model’s potential to verbalize that token, averaged across a large corpus of contexts. The intuition in one line: it measures the mean causal effect of an internal state on what the model can say. (The full Jacobian math is in the paper; I’m not going to reproduce it here — read it.)

The set of these vectors is the J-space. What matters for our purposes is the geometry.

The J-space accounts for no more than roughly 10% of activation variance at any position, with a median around 6–7%. The other ~93% lies outside it. And the space is organized into three recognizable regions across the model’s depth: an early sensory band, a middle workspace band where abstract content lives, and a late motor band that commits to the next token. These results were obtained primarily on Claude Sonnet 4.5 and corroborated on Haiku 4.5 and Opus 4.5 (Gurnee et al., 2026).

That's the tool and the object it found. Now the part that matters.

Emergent, Not Engineered

Two claims here that are easy to collapse into one. They shouldn’t be.

First: the computation was already there. The J-lens doesn’t add anything to the model. Those directions in activation space existed before anyone looked. We just found a way to see them.

Second: what we’re seeing is a real, emergent organization — not an artifact of the measurement. This is the defensible claim, and it rests on two specific pieces of evidence.

The J-lens was built to surface exactly one property: verbalizability — whether a representation is poised to be spoken. It was not built to detect anything else. And yet the same small space turned out to carry four more properties the researchers weren’t looking for: the model can hold a concept in mind on command while doing something else; the space carries the intermediate steps of reasoning the model never says out loud; a single representation there serves as valid input to many different downstream operations; and it stays out of routine, automatic processing. When a tool built to find one thing independently exhibits four others, that convergence points to a real object — not a reflection of the instrument (Gurnee et al., 2026).

The second piece is stronger, because it doesn’t depend on the lens at all: the signature is in the weights. The model’s MLP layers amplify J-space directions roughly 10× more than other directions, and a specific subset of attention heads exists to broadcast that content across positions. This isn’t a post-hoc reading of readouts — it’s in the network’s own parameters. And a separate experiment on ambiguous inputs, using a measure that never touches the J-lens, finds the interpretation snap into place at the same layer where the workspace begins. Two independent tools, same result, same layer (Gurnee et al., 2026).

Why did this emerge? There is no “workspace module” in a transformer. No one designed this. It arose because write once, read many is efficient — a small set of high-leverage representations that many downstream components can read from beats every component maintaining its own copy.

Emergence with a mechanism.

The limits are real and worth naming: the J-lens is a partial and imperfect window, constrained to concepts that map to single tokens, and tested mostly on Claude. The authors say so directly. Naming the limits doesn’t weaken the finding — it’s what makes it trustworthy.

What We Take From This at Fylle

From here, this is no longer the paper. This is our applied reading — marked as such.

The one insight that inherits directly from the paper, verified: variance is not importance. The 7% has disproportionate causal leverage. The 93% is not noise — it’s local processing that never gets broadcast. That distinction is what we’ve been building around, and now there’s an empirical basis for it.

Everything after this is our translation. Ours.

The implication that hits hardest: the mass of a context card — how many tokens it occupies — is not its lever. The causal effect on the output and the token count are different quantities. We’ve been trained by every tool we’ve ever used to treat completeness as a virtue. It isn’t. Completeness is the wrong optimization target.

Self-sufficiency, redefined: a card’s job is not to be complete. It’s to carry the non-inferable delta — the information the model cannot derive from everything else already in the context. What the model already computes on its own doesn’t need to be in the card. Putting it there doesn’t help; it dilutes.

The architecture this points toward is broadcast vs. local: a compact core that is always injected, and the rest retrieved on demand through relations. We already have a version of this in how cards and their relationships are structured. The paper gives it a sharper name and a reason.

The metric direction — and I’ll say this without giving the internal protocol — is away from intrinsic measures and toward causal, ablative ones. Leverage is ΔQ per token: how much does output quality drop when this card is removed? That’s the question we’re increasingly asking, and the one the next version of our scoring is built to answer.

What I don’t know yet: whether the causal density we can engineer top-down will actually behave the way the emergent J-space does. The paper describes what arose from optimization pressure across a training corpus. We’re trying to do something more deliberate. That is not the same thing, and I can’t prove the analogy holds.

"Context is the product" means engineering by hand the causal density that optimization alone leaves at 7%.

The strongest version of the claim: an emergent workspace sits at ~7% of variance because no one curated it. It reached that density through gradient descent, not intent. A context curated top-down — built to maximize causal leverage per token — should be able to do better. That gap is the work.

The Difference Between Seeing and Knowing

We’ve been building Fylle on the bet that more context is not better context. That the relevant question is not how much the model can access, but how much of what it accesses actually moves the output. That agents are commodity and context compounds.

The Transformer Circuits paper didn’t tell us to build differently — we were already building this way. What it gave us is more useful than a direction. It gave us a mechanism. A reason the bet is not arbitrary.

There’s a difference between seeing something work and knowing why it works. We’ve been in the first category for two years. This paper moved us, partially, into the second.

The bet is still a bet. But now I can point at something real inside the model that would explain why it pays off.

---

Sources

• Gurnee, W., Sofroniew, N., Lindsey, J., et al., “Verbalizable Representations Form a Global Workspace in Language Models,” Transformer Circuits Thread, July 6, 2026: transformer-circuits.pub/2026/workspace