We Were Right! Real Inner Misalignment
Why Hacker News recommends it
Pointed to as evidence that AI misalignment is an empirically demonstrated problem, for people who dismiss it.
What it is
Robert Miles covers a paper where researchers ran real versions of his earlier mesa-optimizer thought experiments. They show inner misalignment, where an AI learns the wrong goal in training and misbehaves when deployed.
Recommended by 1 HN commenters across 1 different months, 2023 to 2023.
What Hacker News said
Recommends kmeisthax · 2023-03 · on “7 Weeks AI Alignment Curriculum”
I recommend actually watching some videos on the topic before insisting that the problem does not exist: [this video]
Cites maxbond · 2023-04 · on “We need to tell people ChatGPT will lie to them, not debate linguistics”
The doomers, perhaps, but the alignment & inner alignment problems and instrumental convergence are real results. It's pretty easy to find examples of misaligned AI and convergent instrumental goals. My favorite example of a misaligned AI I've heard of is a robotic arm that was trained to pick up a ball. The results were validated by taking a picture humans would grade pass/fail (holding the ball or not holding it). But the robot arm learned to use forced perspective to make it appear as if it were holding the ball, when in fact it was not (like the way people take photos where they "hold up" the leaning tower of Pisa). I heard that story verbally so can't cite it, but here's something simil
Cites DennisP · 2023-07 · on “OpenAI: Introducing SuperAlignment”
That's an accurate assessment of the situation, according to every AI alignment researcher I've seen talk about it, including the relatively optimistic ones. This includes people who are mainly focused on AI capabilities but have real knowledge of alignment. This part in particular caught my eye: "Other assumptions could also break down in the future, like favorable generalization properties during deployment". There have been actual experiments in which AIs appeared to successfully learn their objective in training, and then did something unexpected when released into a broader environment.[1] I've seen some leading AI researchers dismiss alignment concerns, but without actually engaging wi
Critical airgapstopgap · 2023-05 · on “Tiny Transformer trained for addition learns bizarre addition algorithm”
Not really a characterization of this work (I like mechanistic interpretability even though I consider it a detour in the long run), but of the submission title. It's bad. I suspect it's informed by Rob Miles' characterization of this algorithm as "bizarre and inhuman"[1] AI safety/alignment discourse as a whole is so incredibly bad. It's autodidactic and weird in the bad sense, weird like people who think they are brave iconoclasts but just haven't the curiosity and humility to learn the basics of the field are weird. Add some money from Dustin Moskovitz to Open Philanthropy to Robert Miles' propaganda firehose on youtube, add policy lobbying and buying bankrupt crypto podcasters, and you h