https://t.co/9f9GOTyeWw

Cita · 1954237350512619868 · ℏεsam @Hesamation · 2025-08-09 · en X

The perfect weekend article just dropped by @rasbt on gpt-oss architectures.

It’s wild to see how fast these architectures have leveled up since GPT-2 in 2019.

abs positional emb → RoPE
GELU MLP → Swish + SwiGLU
one dense feed-forward → MoE
Multi-head attention → GQA
… https://t.co/IN8p08d0o9