Rust at speed — building a fast concurrent database
Why Hacker News recommends it
HN points to it as a detailed, captivating talk by the Noria author, including a good explanation of deduplicating data between maps using unsafe.
What it is
A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.
Recommended by 2 HN commenters across 2 different months, 2019 to 2022.
What Hacker News said
Recommends harikb · 2020-06 · on “There’s plenty of room at the Top: computer performance after Moore’s law”
Anecdotal - One of favorite YouTube live-streaming personalities, Jon Gjengset[1] is a Ph.D student at MIT CSAIL. His talks are very detailed[4] and captivating[3] [1] [LINK] Talks [2] [this video] [3] [LINK] Rust Details(my favorites) [4] [LINK]
Recommends Orou · 2021-12 · on “Noria: Dynamic, partially-stateful data-flow for high-perf web applications”
The author of Noria, Jon Gjengset, gave a great talk about the system: [this video] The speeds achieved are impressive but you do not get consistency.
Cites dpezely · 2019-01 · on “Rust at speed – building a fast concurrent database [video]”
He explains this at roughly the 40 minute mark. [1] "There's 7 more lines of `unsafe` that avoids keeping both maps. You still keep two maps, but you de-duplicate the data between those two maps." [this video]
Cites Jonhoo · 2022-11 · on “How PlanetScale Boost serves SQL queries faster”
:wave: Author of the paper this work is based on here. I'm so excited to see dynamic, partially-stateful data-flow for incremental materialized view maintenance becoming more wide-spread! I continue to think it's a _great_ idea, and the speed-ups (and complexity reduction) it can yield are pretty immense, so seeing more folks building on the idea makes me very happy. The PlanetScale blog post references my original "Noria" OSDI paper ( [LINK] ), but I'd actually recommend my PhD thesis instead ( [LINK] ), as it goes much deeper about some of the technical challenges and solutions involved. It also has a chapter (Appendix A) that covers how it all works by analogy, which the less-technical am