A Comparator Atlas of Small Language Models

1,966 attention heads, 10 models, and 11 predictions written before the runs

· 21 min read · ai-ml

I scored all 1,966 attention heads of 10 small language models on the same tasks. The models share a small vocabulary of attention patterns, but only 23% of heads fit it, and the other 77% matter just as much.

I wrote my predictions and their pass rules before each run. Six of the eleven failed. The scorecard near the end lists all of them.

What a comparator is, and how I measured it

An attention head does two jobs. Its query and key weights decide where to look. Its value and output weights decide what to copy from there. I call the first part the comparator: it scores how related two tokens are, and the softmax of those scores is the attention pattern.

My question was practical. If comparators are standard parts, the same few in every model, they could be written down once and reused instead of learned again each time.

I took every head of 10 small models and gave each the same probes.

ModelHeads
GPT-2 (124M)144
GPT-Neo 125M144
Pythia-70M48
Pythia-160M, 4 training seeds4 × 144
SmolLM2-135M270
Qwen2.5-0.5B336
Qwen3-0.6B-Base448

The probes use random tokens, so they work for any tokenizer:

  • Repeat: 100 random tokens, then the same 100 again. This scores attention to the previous token, to the token itself, to an earlier copy of the current token, and to the token after that copy.
  • Categories: 150 words from about 10 categories. This scores attention to a different word of the same category.
  • Natural text: Wikipedia windows of 256 tokens. This scores attention to the first token, to punctuation, and how spread the pattern is.

A head gets a name when one score passes a fixed threshold.

Six kinds recur in every model, but they cover only 23% of heads

The same six kinds of comparator appear across architectures, tokenizers and training sets. Together with 3 heads of a rare seventh kind, they account for 456 of 1,966 heads.

KindThe head attends toHeadsModels with at least one (of 10)Typical relative depth
Semantica different word of the same category10310early
Inductionthe token after an earlier copy of the current token9190.5 to 0.7
Selfthe current token7910varies
Sinkthe first token, on natural text765late
Previousthe token just before6110about 0.3
Same-tokenan earlier copy of the current token4310early
Unnamednone of the above1,51010all depths

Three notes on this table.

  • The sink depends on training. Models trained with a start token park idle attention on it. Pythia was trained without one and parks attention on punctuation instead: 18 to 27 heads do this in 3 of the 4 Pythia-160M seeds.
  • Pythia-70M has no induction head by my threshold. Its best score is 0.27 and the threshold is 0.30.
  • The kinds are islands, not clusters. K-means on the score vectors does not recover them (adjusted Rand index 0.14). Head behaviour is a continuum, and the named kinds are small dense spots in it.

Retraining reproduces the kinds, not the heads

The four Pythia-160M models share architecture and data. Only the initial weights and the data order differ. They are my replication, and they do not give the same heads.

KindHeads in seed 0 / 1 / 2 / 3Largest over smallest
Induction13 / 7 / 12 / 101.9
Previous6 / 6 / 4 / 82.0
Semantic7 / 9 / 9 / 91.3

I also matched each head to its most similar head in another seed, by attention pattern on natural text. Only previous-token heads have a clear twin (median correlation 0.78 to 0.89). Induction heads range from 0.58 to 0.89, and unnamed heads from 0.59 to 0.74. For comparison, the most similar other head inside the same model scores 0.69 to 0.75.

Depth is more stable than identity. A head and its best match in another seed sit within about one layer of each other.

So a model does not learn a fixed set of heads. It learns a budget of kinds, placed at roughly fixed depths, and the count of each kind can vary by a factor of two.

Only same-token matching stands alone

A reusable part should work without the rest of the model. I tested this by feeding each head the raw token embeddings, with no lower layers, and measuring how much of its defining score it keeps.

100% = SCORE IN THE FULL MODELSAME-TOKEN80%SELF60%SEMANTIC40%PREVIOUS10%SINK2%INDUCTION1%

share of the defining score kept without the layers below · median over heads

Same-token heads need only token identity, and the embeddings already carry that. Previous-token heads need position, which in these models arrives through the layers below. The one exception is Qwen2.5, where previous-token heads keep 68%.

Same-token heads are also the one kind that is the same function across models. I compared heads by their scores on 132 words that are single tokens in every tokenizer. The nearest neighbour of a same-token head in another model is a same-token head 72% of the time, against a base rate of 2%. For semantic heads it is 18% against 5%. For every other kind it is 6% or less.

Two kinds can be written down; the rest only partly

If a comparator is a standard part, a hand-written rule should replace it. I tested this on natural text. For each kind, I swapped the attention patterns of all its heads for a substitute and measured the language-model loss.

The score is recovery R. Removing a comparator means uniform attention over earlier tokens, which raises the loss. R is the share of that damage the substitute repairs: 1 is a perfect stand-in, 0 is no better than removal, and below 0 is worse than removal.

I tried two substitutes that need no training: a hand rule over positions and token identity, and a comparator built from the cosine of the model’s own input embeddings.

KindHand ruleR, hand ruleR, built from embeddingsWritten down?
Sinkattend to the first token0.99yes
Same-tokenlatest earlier copy, else the first token0.790.69partly
Same-tokenlatest earlier copy, else the head’s own pattern0.92yes
Previousattend to position i - 10.70partly
Selfattend to position i0.61partly
Inductiontoken after the latest earlier copy, else the first token0.500.41partly
Semantic-0.13no

Values are medians over the models where removing the kind costs at least 0.01 nats (4 models for the sink, 9 for induction and semantic, 10 for the others).

Replacing every named head at once recovers 0.74 with hand rules and 0.80 when same-token and induction heads keep their own pattern on rows with no match (range 0.73 to 0.92). I added the second variant after I saw the first result: in GPT-2, sending no-match rows to the first token was badly wrong in layers 0 to 3.

The semantic result is a warning about names. Embedding cosine alone finds same-category words about as well as the learned heads do in the Pythia models. Yet swapped into the model it is worse than removing the heads. Those heads do more than their name says.

The unnamed 77% are not dead weight

A natural guess is that the unnamed heads are spare capacity. I tested it by ablation on held-out Wikipedia text, with three predictions fixed in advance. Two failed.

  • Per head, an unnamed head matters as much as a named one. I removed each head’s output and measured the loss increase. The median importance of named heads is 0.93 times that of unnamed heads. I had predicted at least 2.
  • Unnamed heads cannot be pruned in bulk. Removing the least important ones first, only 10% of them can go before the loss rises by 1% (median over models, range 0% to 30%). I had predicted at least 30%.
  • The most critical single heads are often unnamed and in layer 0. Removing GPT-2 layer 0 head 10 costs 2.06 nats. Removing GPT-Neo layer 0 head 8 costs 15.2 nats.

Named heads still matter more as a team. Removing all named heads hurts more than removing a random set of the same size, clearly in 9 of 10 models. In Pythia-160M seed 1 the cost is 0.86 nats against 0.26 to 0.33 for random sets. GPT-Neo is the exception: two of its three random sets contain layer 0 head 8.

One prediction passed, with a caveat. Unnamed heads are 64% of each model’s top-10% most important heads (median), above my 50% rule. But they are about 79% of all heads, so named heads are over-represented at the top.

The random-token probes name what is easy to name. Much of the work a small model does with attention is outside that vocabulary.

Head scores follow short equations, and one law is nearly universal

Names are coarse, so I also described each head with an equation. I fitted the head’s score before the softmax as a function of 23 simple features of the query and key pair: distance, position, token match, embedding similarity, punctuation, word frequency and others. The fit uses Wikipedia text.

50% = NO PREFERRED SIGNDISTANCE FROM QUERY TO KEY (LOG)97% NEGATIVEKEY FOLLOWS AN EARLIER COPY OF THE QUERY96% POSITIVEPOSITION OF THE KEY IN THE TEXT (LOG)94% NEGATIVEKEY IS PUNCTUATION93% POSITIVEKEY IS IN THE SAME WORD AS THE QUERY86% NEGATIVEKEY IS THE SAME TOKEN AS THE QUERY85% POSITIVEKEY IS THE FIRST TOKEN72% POSITIVEEMBEDDING SIMILARITY OF QUERY AND KEY44% POSITIVE

share of 1,966 heads whose fitted coefficient has the stated sign

The first row is a recency law. A score that falls linearly in log distance gives attention that falls as a power of distance:

score(i, j) = c - a * log(1 + d)    =>    attention(i, j) ∝ (1 + d)^(-a)

Here d is the number of tokens between query i and key j. The fitted exponent a runs from 1 to 4.5. Log distance is the dominant term in 854 of about 1,050 unnamed heads with a usable fit. Many unnamed heads are this law and little else.

The equations also work as parts, one head at a time. I took the 10 most important heads of each model, 100 in total and 43 of them unnamed. I replaced each head’s attention pattern by the softmax of its equation.

Equation typeMedian R over the 100 heads
Sparse linear, median 4 terms0.95
Linear, all 23 features0.94
KAN, additive0.90
Symbolic regression (PySR)0.84

Named heads reach 0.96 to 0.99 with every method. Unnamed heads reach 0.65 to 0.83.

Two cautions.

  • The equations fit poorly in the usual sense. The median held-out R² for unnamed heads is 0.38. R² weights every key equally, while the softmax only cares about the few top keys. An equation can replace a head and still explain little of the score variance.
  • Symbolic regression did not beat the sparse linear fit. Its held-out R² is lower by 0.14. In my setup, nonlinear terms bought nothing.

Some equations repeat across models. Sink heads in Qwen and SmolLM2 are one term: a constant of 18 to 27 on the first token. Induction heads in three Pythia seeds share one form: the token before the key resembles the query token, with the first token and punctuation as fallbacks.

Replaced together, named equations mostly hold and unnamed equations fail

Swapping one head at a time is the easy test. A reusable part must also work when many heads are swapped at once, because errors can add up through the layers. In each of the 10 models, I replaced every head of a group by its equation, on text held out from the fits.

NAMED, ALL 23 FEATURESNAMED, SPARSEUNNAMED, ALL 23 FEATURESUNNAMED, SPARSE0.00.20.40.60.81.0PASS RULE: 0.8 OR MORER: SHARE OF THE REMOVAL DAMAGE REPAIREDGPT-2GPT-NEO 125MPYTHIA-70MPYTHIA-160M SEED 0PYTHIA-160M SEED 1PYTHIA-160M SEED 2PYTHIA-160M SEED 3SMOLLM2-135MQWEN2.5-0.5BQWEN3-0.6B

every head of a group replaced by its equation at once · 10 models

My rule, fixed before the run, was R of at least 0.8 for the named heads together.

  • Named heads mostly pass. With all 23 features the median R is 0.91, and 8 of 10 models pass. GPT-Neo (0.61) and Pythia-70M (0.77) fail. With the sparse equation the median is 0.80, and 5 of 10 pass.
  • Unnamed heads fail in every model. The medians are 0.54 with all 23 features and 0.37 with the sparse equation, and the best model reaches 0.62. Swapped one at a time, the important unnamed heads recovered 0.65 to 0.83. Small errors in many heads compound.
  • All heads at once fail too. The medians are 0.49 and 0.27.

One caution on the first point. The rule did not say which equation type counts. I ran GPT-2 first, and there the sparse equation was better (0.93 against 0.85). In 8 of the 10 models the equation with all 23 features is better. So the named result is a clear pass with one equation type and a borderline one with the other.

Scorecard: 6 of 11 predictions failed

Every prediction and its pass rule was written before the run. Three passed, six failed and two were mixed.

Prediction, fixed before the runResultVerdict
Previous, same-token, induction and sink heads exist in all 10 modelsinduction in 9, sink in 5Fail
Head counts per kind are stable across Pythia seeds (ratio at most 1.5)up to 2.0Fail
Named heads have a twin in other seeds (correlation at least 0.8)only previous-token headsFail (same prediction as the row above)
Previous-token heads survive without the layers below (at least 0.7 kept)0.10 keptFail
Induction heads do not survive without the layers below (under 0.3 kept)0.01 keptPass
Hand rules replace sink and previous heads (R at least 0.8)sink 0.99, previous 0.70Mixed
Named heads matter at least 2 times more per head than unnamed ones0.93 timesFail
Unnamed heads are at least 50% of the top-10% most important heads64%Pass, below their 79% base rate
At least 30% of unnamed heads can be pruned for 1% loss10%Fail
A sparse equation explains unnamed heads’ scores (held-out R² at least 0.5)0.38Fail
At least 3 dominant equation terms are shared by 7 or more models3 termsPass
Equations replace important heads (R at least 0.5)0.84Pass
Symbolic regression beats the sparse linear equation (R² gain at least 0.1)-0.14Fail

The table has 13 rows because two of the 11 predictions had two parts.

What this does not show

I want to be precise about the limits:

  • One run. I ran no seeds of my own. The four Pythia-160M seeds are the only replication.
  • Small models only. The largest has 0.6B parameters. Nothing here says the picture holds at 7B or above.
  • Thresholds decide names. They were fixed in advance, and some heads sit near them, such as the Pythia-70M induction score of 0.27 against 0.30.
  • One text source. Replacement and ablation use 64 Wikipedia windows per set.
  • R is unstable when removing a group costs little loss.
  • The joint replacement result depends on the equation type. Named heads pass in 8 of 10 models with all 23 features and in 5 of 10 with the sparse equation.
  • Prior work is not yet compared in detail. Induction and previous-token heads, head pruning and head roles all have a literature. I have not yet checked each of my numbers against it.

Where this goes next

Two results point to the next step. Equations for named heads work alone and, in 8 of 10 models, together, so most of those comparators can be produced without training. Equations for unnamed heads fail together, so the 23 features miss part of what those heads do. The next experiment studies the residuals of the unnamed heads to find the missing features.

One link back to why I started. The only comparator that stands alone and is the same function in every model is exact token matching. That fits my earlier experiments, where heads trained in isolation worked on exact symbols and failed on paraphrased text.