Lexical Scanning in Go - Rob Pike
Why Hacker News recommends it
HN returns to it for years as an elegant introduction to writing a lexer that is easy to follow even without a parsing background.
What it is
Rob Pike shows how the Go text/template lexer is built, using state functions, goroutines and channels. The talk was given at a Google Technology User Group in Sydney in 2011.
Recommended by 13 HN commenters across 12 different months, 2012 to 2025.
What Hacker News said
Strong rec willvarfar · 2012-06 · on “Parsing huge XML files with Go”
Personally, I have a penchant for writing my own pull parsers. Its a mind-expanding exercise. The neat thing about Go is that parsers can return functions that consume the next token. Rob Pike has an excellent video about this: [this video]
Strong rec TomNomNom · 2014-05 · on “Templating in Go”
Rob Pike did a really good talk entitled 'Lexical Scanning In Go'[0], which covers how the lexing for text/template works. It's far more interesting than it sounds, and even if you don't know much about lexing/parsing it's easy to follow. [0] [this video]
Strong rec grose · 2016-10 · on “Ask HN: What's your favorite tech talk?”
Lexical Scanning in Go by Rob Pike [this video] I love everything about this talk. It walks you through building a lexer from scratch in a simple and elegant way, through a very interesting use of coroutines. I appreciate the bits of humor in the talk as well.
Strong rec hu3 · 2024-11 · on “Rust for tokenising and parsing”
Related, I love Rob Pike's talk about lexical Scanning in Go (2011). Educational and elegant approach. [this video]
Recommends drbawb · 2012-03 · on “Go is amazing, period.”
As an aside: watching Pike's talk on lexing and parsing in Go really hits this point home. It shows how language support for cheap threads (goroutines) and channels really change the way you come up with an algorithm. [this video]
Recommends jbarham · 2013-02 · on “Function types in Go”
> Building a lexer out of a recursive function type actually sounds like a really cool thing to try Rob Pike did just that for Go's text/template package ( [LINK] ). See the source lexer source code at [LINK] and note that stateFn is defined recursively. There's a video by Pike describing how he built the parser at [this video] .
Recommends mseepgood · 2013-02 · on “Concurrency is not Parallelism. Rob Pike at Waza 2012 [video]”
> What most people don't realize is that for concurrency to be useful, there must be parallelism. No, as far as I understand from this and other talks, Rob Pike argues that concurrency is useful even without parallelism: it's a better way to structure software. A nice example for this is shown in his talk about lexical scanning in Go: [this video]
Recommends thisishugo · 2013-04 · on “Building a toy template engine in Python”
I'm not sure I like the idea of parsing templates with regular expressions. There is a better way, as demonstrated in Robe Pike's video[1] detailing how the Go template language is implemented. Edit: which isn't meant to detract from the fact that this is a great article. [1] [this video]