Can a model unpack its own representations on demand?
During pretraining, a language model has to compress an entire universe of facts, styles, and concepts into a few thousand geometric dimensions. To survive this bottleneck, it relies on superposition—packing multiple semantic features into overlapping, non-orthogonal directions. It is a brilliant mathematical compromise for raw capacity, but it leaves internal activations deeply polysemantic and nearly impossible to steer cleanly.
The current instinct in mechanistic interpretability is to fix this from the outside: freeze the model and train a large Sparse Autoencoder (SAE) to decode the activations after the fact. But that approach treats superposition as an unalterable given. What if it isn’t? When a model is running a bounded downstream task—like evaluating sentiment or moderating toxic speech—its active feature vocabulary is dramatically smaller than the rest of the web.
If the task only requires a handful of features, the model shouldn’t need to cram them on top of each other. That raised a simple question: rather than observing superposition from the outside, could we use a lightweight LoRA adapter to rotate the model’s internal representation space into an orthogonal coordinate system while it adapts?
Turning low-rank channels into an orthogonal coordinate frame.
Standard LoRA updates a frozen weight matrix with a low-rank factorization, ΔW = BA. Mechanistically, each rank is already structured like an internal encoder-decoder: a row of A acts as a detector probing the input, while the corresponding column of B writes a feature direction back into the residual stream. In ordinary fine-tuning, however, those write directions end up heavily entangled.
Ortho-LoRA changes that dynamic by adding an attribution-conditioned penalty alongside the normal task loss. Crucially, it doesn’t blindly demand that all write directions be perpendicular. Instead, it scales the penalty by token co-activation: if two channels frequently fire together on real task examples, the loss aggressively forces their vectors apart. If they never co-occur, it leaves them alone. A light sparsity penalty on the bottleneck prevents the network from cheating by firing all channels at once.
The beauty of this setup is operational simplicity. The adapter trains on a modest single GPU without second-order gradients, and when training is done, ΔW folds directly into the base weights with zero runtime latency. The open question was whether forcing this geometric separation would break the model’s ability to perform the task.
A stress test across geometry, steering, and capacity limits.
To test whether the representations were genuinely unpacking, we ran experiments across two architectures—Qwen-2.5-0.5B (modern Gated SwiGLU) and Pythia-410M (standard dense GELU)—across sentiment classification (tweet_eval) and safety moderation (real-toxicity-prompts).
We focused our primary evaluation on a rank-32 adapter placed at Layer 12, tracking three things in parallel: geometric decorrelation (off-diagonal Gram coherence and effective rank utilization), task performance (downstream cross-entropy loss), and causal steerability. If a channel has truly become an isolated semantic lever, dialling its activation up or down should move the intended behavior without scattering unintended token noise across the rest of the vocabulary.
Finally, we trained a matched Top-K SAE on the exact same layer activations to see how an endogenous low-rank update compares to an exogenous probe in feature recovery and latency.
A 10× drop in cross-talk without sacrificing downstream accuracy.
The geometric shift was immediate. In our primary Qwen run, standard LoRA converged to an entangled subspace with an average off-diagonal coherence of 0.7188. Ortho-LoRA collapsed that cross-talk down to 0.0696—a 10.3× reduction. More importantly, it prevented rank collapse: while standard LoRA crowded 90% of its variance into just 4 or 5 singular directions, Ortho-LoRA equalized the spectrum, utilizing over 87% of its available rank capacity.
Crucially, this geometric clean-up didn’t cost the model its predictive power. Test loss shifted only marginally from 0.5894 to 0.6365 on Qwen, and on Pythia, Ortho-LoRA actually improved generalization (0.7884 vs. 0.8359), acting as a healthy structural regularizer against co-linear overfitting.
The most compelling evidence, however, came from activation steering. When we scaled a candidate sentiment rank, Ortho-LoRA produced a clean, monotonic probability increase of +0.1673 while suppressing off-target vocabulary drift to just 0.0676—a 16× reduction in collateral damage compared to classic residual activation addition. When write vectors are orthogonal, injected energy simply has nowhere to leak.
Cleaner geometry does not automatically equal human meaning.
It is tempting to look at orthogonal Gram heatmaps and declare that superposition has been solved. But geometry and semantics are not synonymous. Proving that two vectors are mathematically perpendicular on a validation split does not guarantee that each vector maps cleanly to a single, stable human concept across out-of-distribution prompts.
What the experiments do show is a clear boundary condition: the Johnson-Lindenstrauss capacity limit is real. When we restricted the adapter to rank 4 on sentiment, applying the orthogonality penalty triggered an immediate performance collapse—test loss spiked to 1.1211. If a task intrinsically depends on five or six interacting primitives, forcing a 4-channel adapter to be strictly orthogonal deprives the model of the superposition "cheat code," forcing it to drop vital cues entirely.
In short: you cannot eliminate superposition unless your adapter budget is at least as large as the task’s intrinsic complexity (r ≥ Ktask). Layer depth also matters: mid-layers (Layer 12) proved to be the sweet spot for conceptual steering, while late-layer interventions merely nudged token-level un-embedding biases.
Where this breaks and what next.
Our method is built for circumscribed domains, not open-ended foundation pretraining. Trying to eliminate superposition across the entire internet with low-rank adapters would cause catastrophic capacity starvation. Furthermore, our interventions are layer-localized: representations arrive from earlier layers already superposed, and subsequent non-linearities can partially re-tangle them before the final output.
With that in mind, there are several follow-ups avenues worth pursuing. First, we could experiment with applying Ortho-LoRA across several depths simultaneously (e.g., layers 6, 12, and 18) to prevent downstream layers from undoing the orthogonalization. Second, we could try to uildi a rank scheduler that detects capacity starvation during early training steps, allocating more orthogonal channels when a task demands them. Finally, further exploration could test whether orthogonalized adapters trained on separate tasks can be merged together without the classic interference and task-erasure problems common to standard PEFT merging.
Code and paper
All experimental pipelines, and draft materials are open source.