---
id: "1954237350512619868"
requested_id: "1954237350512619868"
status: ok
level: 1
user: "Hesamation"
name: "ℏεsam"
created_at: "2025-08-09T17:44:26.000Z"
in_reply_to: null
quoted: null
source: tweet-result
url: "https://x.com/Hesamation/status/1954237350512619868"
via:
  - quote: "1954620090801942616"
media:
  - data/external_media/1954237350512619868-0.jpg
---

The perfect weekend article just dropped by @rasbt on gpt-oss architectures.

It’s wild to see how fast these architectures have leveled up since GPT-2 in 2019.

abs positional emb → RoPE
GELU MLP → Swish + SwiGLU
one dense feed-forward → MoE
Multi-head attention → GQA
… https://t.co/IN8p08d0o9