Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start
Why Hacker News recommends it
HN commenters use it as the standard introductory talk on the AI goal-alignment problem and recommend it to people asking why a superintelligent AI might not share human goals.
Recommended by 3 people across 5 different months, 2017 to 2026.
What it is
Eliezer Yudkowsky of MIRI gives a Stanford Symbolic Systems Distinguished Speaker talk on why aligning advanced AI with human values is hard. He covers the orthogonality thesis, goal specification problems, and early research directions in alignment, with corrections added after the fact.
What people said
Strong rec colorincorrect · 2019-10 · Hacker News · on “Experts who say we shouldn't worry about superintelligent AI are wrong?”
probably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on the question of this "alignment problem". Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: [this video]
Recommends gonehome · 2017-12 · Hacker News · on “The Merge”
The friendly AI goal alignment problem is a pretty interesting part of this and there’s interesting work going on there. The basic idea is how do you effectively align a general intelligence’s goal with humans so that it uses its intelligence to solve problems in a way that aligns with our own human utility functions. If humans figure out general AI before goal alignment it may have outcomes we don’t want. [LINK] [this video]
Recommends gonehome · 2021-01 · Hacker News · on “Still alive”
AGI = Artificial General Intelligence, watch this for the main idea around the goal alignment problem: [this video] They're explicitly not political, lesswrong is a website/community and rationality is about trying to think better by being aware of normal cognitive biases and correcting for them. Also trying to make better predictions and understand things better by applying Bayes' theorem when possible to account for new evidence: [LINK] (and being willing to change your mind when the evidence changes). It's about trying to understand and accept what's true no matter what political tribe it could potentially align with. See: [LINK] For more reading about AGI: Books: - Superintelligence (I find h
Recommends gonehome · 2021-04 · Hacker News · on “Book Review: “A Thousand Brains” by Jeff Hawkins”
I think this is objectively wrong even on a human level. I could see some part of it if you're using empathy in a general enough sense to only mean modeling other minds without caring about their goals except as a way to pursue your own (which doesn't sound like what you're saying). It sounds more like you're putting intelligence in some reference class where you're just stating it's not intelligence until it's already aligned with humans (which is not helpful). For some reason people tend to think that general intelligence would generate all these other positive human-like qualities, but a lot of those are not super well aligned even in humans and they are tied to our multi-billion year evo
Recommends conmod278 · 2026-09 · Hacker News · on “Aligned to whom?”
Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start [this video] Eliezer talked about these ideas way before everyone else.
Cites colorincorrect · 2017-04 · Hacker News · on “Ask HN: AI winter coming in next few years for AGI research?”
Its very hard to say as no one can really predict when a research drought will come unless you actually do the research. Though I must say that most of the ML researchers I've come across aren't directly concerned with AGI. (either they outright dont care, or they think its a step by step thing) If AI winter comes it's probably because of disappointment of progress due to misunderstanding/misalignment with the general public / rich investors, so now you know who to blame. The state of cognitive science right now is basically a bunch of scientists saying: "yeah to be honest I have no idea really how the mind does this, but if you ignore the edge cases then this model I have here works pretty