HC2023-K2: Hardware for Deep Learning
Why Hacker News recommends it
HN points to it for an insider view of why NVIDIA's AI gains came from design rather than process, and for data-movement power costs.
What it is
Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.
Recommended by 2 HN commenters across 2 different months, 2023 to 2025.
What Hacker News said
Strong rec Footpost · 2025-06 · on “Neuromorphic computing”
Neuromorphic computation has been hyped up for ~ 20 year by now. So far it has dramatically underperformed, at least vis-a-vis the hype. The article does not distinguish between training and inference. Google Edge TPUs [LINK] each one is capable of performing 4 trillion operations per second (4 TOPS), using 2 watts of power—that's 2 TOPS per watt. So inference is already cheaper than the 20 watts the paper attributes to the brain. To be sure, LLM training is expensive, but so is raising a child for 20 years. Unlike the child, LLMs can share weights, and amortise the energy cost of training. Another core problem with neuromorphic computation is that we currently have no meaningful idea how th
Recommends tverbeure · 2023-10 · on “Nvidia, one of tech's hottest companies, is fine with remote work”
I think your biggest mistake is ignoring the amount of software that runs on top of the hardware. There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA. My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you stand out. Bill Daly gave a great presentation at Hot Chips a few weeks ago that explains how Nvidia’s AI performance was only in small parts due to process improvement. The rest was doing things smarter: matrix sparsity, different number representations and so forth. [this video]
Cites gdiamos · 2024-09 · on “3D-Stacked CMOS Takes Moore's Law to New Heights”
Says the hotchips keynote - [this video] - data that is moved is about 100x higher power than data that is local You have a good point about intermediate data. This would not apply to all algorithms.