Attention Is All You Need
Why Hacker News recommends it
HN recommends it as an accessible, clear walkthrough of the transformer paper for people with some AI background.
What it is
Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.
Recommended by 4 HN commenters across 4 different months, 2020 to 2025.
What Hacker News said
Strong rec bfung · 2023-01 · on “The Transformer Family”
YouTube channel that explains the paper in detail: [this video] And subsequent follow-ups (ROME, editing transformer arch): [LINK] I find the channel amazing explaining super complex topics in simple enough terms for people who have some background in AI.
Recommends kanche · 2020-10 · on “Rethinking Attention with Performers”
Everyone has a different way of approaching. I like video walk-throughs of papers. If you are the same, for the theory bit you may find Yannic Kilcher's videos [0,1] helpful. On implementation side, the best project out there is [2] with good documentation. Also spacy has transformer package [3] (I personally haven't tried it), so maybe it will be easier for you to jump in if you have prior experience with spacy. [0] [this video] [1] [LINK] [2] [LINK] [3] [LINK]
Recommends visarga · 2021-12 · on “The potential of transformers in reinforcement learning”
For accessibility I recommend Yannic Kilcher video review of "Attention Is All You Need" [this video] Yannic has been making about 62 other transformer paper reviews since. You can find the usual suspects. [LINK]
Recommends throwawaymaths · 2023-05 · on “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”
The Yannic kilcher review is quite good. [this video] I can't ELI5 but I can ELI-junior-dev. Tl;dw: Transformers work by basically being a differentiable lookup/hash table. First your input is tokenized and (N) tokens (this constitutes the attention frame) are encoded both based on token identity and position in the attention frame. Then there is an NxN matrix that is applied to your attention frame "performing the lookup query" over all other tokens in the attention frame, so every token gets a "contextual semantic understanding" that takes in both all the other stuff in the attention frame and it's relative position. Gpt is impressive because the N is really huge and it has many layers. A big N
Cites throwawaymaths · 2025-05 · on “Writing an LLM from scratch, part 13 – attention heads are dumb”
> I love these word-salad exaltations of LLMs - people don't even know what they're writing half the time but it sure is provocative! Well FWIW I have done an implementation of Llama LLM (3 8-70b to be specific) in non-python so I do sort of know what I'm talking about. I'm not the originator of the hash table analogy. I got it from here: [this video] Hell, the vectors that are generated are called K/V, so. Yeah it's a hash table. And the idea that first order logic and other facets of intelligence can just be cleverly arranged lookup tables comes from here: [LINK]