ONS 2015: Wednesday Keynote - Amin Vahdat
Why Hacker News recommends it
HN cites it as a rare public look at Google's internal network design and how it scales to many VMs.
What it is
Google's Amin Vahdat gives the ONS 2015 keynote on Google's datacenter networking, including its custom hardware, software-defined networking and very high bisection bandwidth.
Recommended by 1 HN commenters across 1 different months, 2015 to 2017.
What Hacker News said
Strong rec mikecb · 2016-03 · on “What's the Best Cloud? Probably GCP”
If you're interested in datacenter networking, can't miss ONS. Amin Vahdat, the guy in charge of networking at Google, has keynoted the past two years, with hardware in 2015[1] and software in 2014[2]. Mark Russinovich also spoke about Azure's SDN, but it doesn't come close to Amin's presentation. Every once in a while you'll hear some random spec from one of these companies, and it's always pretty surprising, but Amin's team has achieved 5 Petabit/s in bisectional bandwidth. It's more than surprising. [1] [this video] [2] [LINK] [3] [LINK]
Cites jsolson · 2015-06 · on “Is the Network the Limit? Dealing with Weave, CoreOS and Azure”
Hi Joe :) Conveniently, Amin Vahdat recently shared some content publicly about why it should scale to lots of VMs: [this video] (And, to others, disclaimer: I work on some bits of GCE networking at Google)
Cites jsolson · 2016-01 · on “Non-volatile Storage: CPUs no longer more performant than I/O devices”
The numbers in this are daunting, but I personally believe massively multi-core systems make the problem a lot less daunting than the article makes out. Core counts in big servers can get up over 100 per server for Intel (see Amazon's new EC2 offerings for public evidence of this). Intel's Xeon Phi series of processors offer core counts approaching ~300. Going to 300x takes the required latencies per request from microseconds up to the millisecond range. POWER systems can go even higher. Moreover, for many workloads that actually leverage this sort of compute you can do something horrifying with the new DRAM-addressable persistent storage: DMA directly from the NIC into block storage. Some (
Cites whatupmd · 2016-08 · on “Google Compute Engine Incident #16015”
They switched to openflow on internal networks for this reason, deterministic: [this video] External network is BGP though and sounds like they didn't detect it until user reports hit. They can't predict the problem and there detection isn't working well either.
Cites zaroth · 2017-04 · on “Espresso – Google’s peering edge architecture”
For some reason they didn't actually link to the talk, which I haven't watched, but presumably actually tries to start answering those questions. Quick search is showing the 2015 keynote that Amin gave, haven't found the 2017 one yet... [1] - 2015 ONS Keynote [this video]