Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence

What Hacker News said

Cites lexandstuff · 2023-04 · on “Behind the curtain: what it feels like to work in AI right now”

In this interview [1] with Ilya Sutskever, he indicates that they aren't even close to tapping out of data. [1] [this video]

Cites machdiamonds · 2023-04 · on “OpenAI’s CEO Says the Age of Giant AI Models Is Already Over”

Ilya Sutskever (OpenAI Chief Scientist): "Yeah, I would say the data situation is still quite good. There's still lots to go" - [this video] There was a rumor that they were going to use Whisper to transcribe YouTube videos and use that for training. Since it's multimodal, incorporating video frames alongside the transcriptions could significantly enhance its performance.

Cites jstx1 · 2023-04 · on “OpenAI’s CEO Says the Age of Giant AI Models Is Already Over”

Ilya Sutskever from OpenAI saying that the data situation is good and there's more data to train on - [this video]

Cites godelski · 2025-05 · on “Chomsky on what ChatGPT is good for (2023)”

I think many people are missing the core of what Chomsky is saying. It is often easy to miscommunicate and I think this is primarily what is happening. I think the analogy he gives here really helps emphasize what he's trying to say. If you're only going to read one part, I think it is this: | I mentioned insect navigation, which is an astonishing achievement. Insect scientists have made much progress in studying how it is achieved, though the neurophysiology, a very difficult matter, remains elusive, along with evolution of the systems. The same is true of the amazing feats of birds and sea turtles that travel thousands of miles and unerringly return to the place of origin. | Suppose Tom Jo

Cites godelski · 2025-06 · on “How can AI researchers save energy? By going backward”

> What does it mean for computation to have a direction? Actually, all computation has a directionality! This is actually a subject that I get really excited about ^__^ Think about this, we have a function f, with input x, and output y. f(x) -> y We'd even use that notation! This is our direction. Now, the reverse actually gets a bit tricky. If the reverse is straightforward, our function has an inverse. But, it might not always. So our function f(x)=mx+b is invertible, because we can write x = (f(x)-b)/m (well... m can't be 0), which provides a unique solution. Every x corresponds to a unique f(x). But if instead, we have the function f(x) = x^2, this is not true! x = sqrt(f(x)), and here e

Cites godelski · 2025-08 · on “Persona vectors: Monitoring and controlling character traits in language models”

Thanks for that good summary. > I see how Ilya was right There are still some things Ilya[0] (and Hinton[1]). The parts I'm quoting here are an example of "that reddit comment" that sounds right but is very wrong, and something we know is wrong (and have known it is wrong for hundreds of years!). Yet, it is also something we keep having to learn. It's both obvious and not obvious, but you can make models that are good at predicting things without understanding them. Let me break this down for some clarity. I'm using "model" in a broad and general sense. Not just ML models, any mathematical model, or even any mental model. By "being good at predicting things" I mean that it can make accurate

Critical godelski · 2025-05 · on “The "AI 2027" Scenario: How realistic is it?”

I do say the same think about JKR, btw. And for the same things, because the content she writes. I think you focused on the fanfic part and not the part where I'm criticizing where they say their stuff is the most important to keep humanity alive and they're charging money for it. Meanwhile, you may notice in academia we publish papers to make them freely available, like on arXiv. If it is that important that people need to know, you make it available. The second person, Hinton, is not as good of an authority as you'd expect. Though I do understand why people take him seriously. Fwiw, his Nobel was wildly controversial. Be careful, prizes often have political components. I have degrees in bo

Critical godelski · 2025-05 · on “Chomsky on what ChatGPT is good for (2023)”

> The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. I think they are really excited by this. There seems no deficiency of linguists using these machines. But I think it is important to distinguish the ability to understand language and translate it. Enough that you yourself put quotes around "understanding". This can often be a challenge for many translators, not knowing how to properly translate something because of underlying context. Our communication runs far deeper than the words we speak or write on a page. This is much of what linguistics is about, this depth. (Or at least that's what they've told me, since