Case study · Active · Apr 2026 →
Orpheus
A mesa-optimization probe — looking down into the forward pass.
Do small transformers trained on in-context linear regression implement gradient descent internally during their forward pass? If so, where in the network does the optimization emerge, and what computational signatures distinguish a mesa-optimizer from a lookup table?
Probe spec
- Task
- ICL linear regression
- Depth
- 1, 2, 4 layers
- Harness
- TransformerLens
- Method
- Activation patching
- Baselines
- OLS · Ridge
- Status
- Active
A rough conceptual sketch
Why this question
In-context learning — the phenomenon where a pretrained transformer, given a few examples in its prompt, appears to fit a new task on the fly — is one of the most peculiar empirical facts about large language models. It looks like learning. It isn't training. Something in the forward pass is doing the work that gradient descent would do offline.
In 2023, von Oswald et al. showed constructively that a linear self-attention layer can implement one step of gradient descent on a least-squares regression problem. That is a proof of possibility. It doesn't tell us whether any network, trained on a specific task distribution, actually does this — versus simply storing an enormous lookup table and retrieving from it.
Orpheus is the probe for that distinction. The stakes aren't academic. If models learn to run optimization loops internally, their capabilities scale in a qualitatively different way than memorization-based models, and safety properties (goal misgeneralization, deceptive alignment) need to be argued at the level of the thing that is being optimized, not the training loss.
Three-phase methodology
Train & baseline
Train small transformers (1, 2, 4 layers) on synthetic in-context linear regression following von Oswald et al.'s experimental setup. Establish baseline performance against ordinary least squares and ridge regression. Vary model depth, context length, and task distribution spread.
Activation analysis
Use TransformerLens to extract activations at every layer. Apply activation patching and causal tracing to identify which attention heads and MLPs are causally responsible for the in-context prediction. Compare activation trajectories against the weight updates gradient descent would produce.
Mesa-optimizer classification
Develop quantitative criteria to distinguish mesa-optimizers from lookup tables. Test OOD generalization to function classes unseen at training, following the Garg et al. protocol. Ablation studies to identify the minimal circuit required for the optimization behavior.
What counts as an answer
A result of "the transformer implements gradient descent" is too soft to be falsifiable on its own. Orpheus commits in advance to the signatures that would count as evidence one way or the other:
Activations evolve step-by-step along the descent direction.
The layer-by-layer residual stream should trace out a trajectory in parameter space that matches the explicit gradient-descent iterate — not just its endpoint.
OOD generalization to unseen function families.
If the transformer solves function classes it was never trained on (e.g. sparse linear, then shallow neural), it has encoded the optimization procedure — not the answer.
Accuracy collapses outside the training task distribution.
A lookup-table model will memorize the training distribution well and fail sharply outside. Orpheus checks this by design.
Ablating any single head destroys the behavior.
A genuine algorithmic circuit has redundancy and modularity. A brittle single-path is a signature of memorization, not optimization.
Why the name
Orpheus — the poet who walked into the underworld to retrieve what he could not see. Mechanistic interpretability is the same move: you know something is there, you walk in, and the rule is that if you look too directly, too early, at too coarse a resolution, you lose it. The tools matter. The patience matters. The willingness to accept an answer of "not yet" matters.
Key references
- [1] von Oswald et al. (2023). Transformers Learn In-Context by Gradient Descent. ICML. Constructive proof that linear self-attention can perform one GD step on a least-squares objective.
- [2] Garg et al. (2022). What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. NeurIPS. The OOD evaluation protocol — train on one function class, test on another.
- [3] Hubinger et al. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. The original framing of the mesa-optimizer problem.
- [4] Olsson et al. (2022). In-context Learning and Induction Heads. Anthropic. The methodological precedent for mechanistic analysis of ICL.