Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp

Erlang Solutions · 49K views · 2014 · 49:15

Open on YouTube

Distributed systemsFunctional programmingSoftware performance & optimizationtalk

Why Hacker News recommends it

The go-to talk showing how Erlang let a tiny team run a massive messaging system.

What it is

Rick Reed of WhatsApp explains at Erlang Factory 2014 how the service scaled to billions of messages with a very small engineering team. He covers the FreeBSD and Erlang stack, Mnesia usage, server counts and bottleneck work.

Recommended by 6 HN commenters across 10 different months, 2014 to 2022.

What Hacker News said

Strong rec rdtsc · 2014-07 · on “Revisiting 1M Writes per second”

My other favorite "scalability" study is from WhatsApp: [this video] That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp (that walk title was create before the acquisition and was mean to imply message count, after the acquisition it got a secondary meaning). The one thing that is fascinating about it, is how small their team was compared to the volume and complexity of the operation.

Strong rec kornish · 2017-07 · on “How Discord Scaled Elixir to 5M Concurrent Users”

Not sure why this comment saw a couple downvotes earlier. mbesto is correct: for most startups, most of the time, competitive advantage doesn't come from the underlying tech stack. To make a general statement, most things could be done similarly on any of several platforms. However, when product requirements match exceptionally well with a specialized technology, you can see things that would simply be infeasible or extremely tough using a different stack. WhatsApp + Erlang was one of those cases (watch this talk and imagine trying to recreate that system with only a handful of server engineers using any other tech: [this video] ). Discord + Elixir appears to be another. Curious if anyone has any

Strong rec kornish · 2020-09 · on “Scaling Erlang Developer Experience at WhatsApp [pdf]”

One of my favorite talks is by Rick Reed about scaling Erlang at WhatsApp. What an absolute savage. He flies through an articulate and in-depth curriculum on system performance and bottleneck mitigation. The talk is called "That's 'Billion' with a 'B'" and makes for a great lunchtime watch: [this video]

Recommends rdtsc · 2014-10 · on “Mnesia and CAP”

Unrelated, but maybe interesting: WhatsApp was running Mnesia and made it work fantastically. Especially with such a small team of engineers. Here is a video of Rick Reed talking about how they did it (hint it is not a one monolithic Mnesia cluster). [this video]

Recommends rbsn · 2014-11 · on “Yahoo Mail moving to React”

Just to follow up on topic of WhatsApp and Erlang, here is a presentation given at Erlang Factory 2014 about their goals with scaling WhatsApp to billions of simultaneous users. [this video]

Recommends rdtsc · 2015-09 · on “Why WhatsApp Only Needs 50 Engineers for Its 900M Users”

Not "parent" but search for Erlang Factory talks they talk about it. Here is one from Rick Reed. He explains how the product works : [this video]

Recommends ddeck · 2016-05 · on “Introducing WhatsApp's Desktop App”

It's required because your phone is where your messages are stored. Whatsapp don't retain messages/media after they've been delivered to your phone, which is a compelling privacy feature for many. It's also what allows them to serve such an enormous user base with limited hardware. Their technology stack (FreeBSD/Erlang) is pretty interesting, more info here: 2014 talks by Rick Reed: [this video] [LINK] Slides: [LINK] There's likely no technical reason why you couldn't use a pc instead of a phone for users that want to use the pc as the primary client (with the phone optionally accessing the DB on the pc in the same way that the desktop client does for the phone). Perhaps they've decided that thi

Recommends otterley · 2017-03 · on “A comment left on Slashdot”

Here's a great talk from Rick Reed (a former mentor of mine at Yahoo!) about how WhatsApp scaled their Erlang infrastructure: [LINK] Here's another talk: [this video]