Mohamed EL HARCHAOUI — AI Engineer. Intelligence for thinking, robotics for doing.

This is where I share reflections on systems reshaping our world.

Featured

A Comparator Atlas of Small Language Models

1,966 attention heads, 10 models, and 11 predictions written before the runs

I scored every attention head of 10 small language models on the same tasks. They share six kinds of attention pattern, but only 23% of heads fit them, and the other 77% matter just as much.

· 21 min read