Training an unbeatable AI in Trackmania
Why Hacker News recommends it
Recommended as an approachable explanation of how reinforcement learning works in practice, and cited for the constraints needed to stop exploits.
What it is
Yosh trains a reinforcement learning agent to drive Trackmania until he can no longer beat it. The video explains how the training works and the design choices involved.
Recommended by 1 HN commenters across 1 different months, 2024 to 2025.
What Hacker News said
Strong rec smokel · 2024-07 · on “The History of Machine Learning in Trackmania”
For those who only read the comments, be sure to check out the videos by Yosh. They're amazing, and do a great job explaining how reinforcement learning works in practice: [this video] [LINK]
Cites emporas · 2024-01 · on “Chess-GPT's Internal World Model”
>The entire point of this is to not encode the board state! I am not sure about this. From the article "The 50M parameter model played at 1300 ELO with 99.8% of its moves being legal within one day of training." I thought that the experiment was how well the model will perform, given that it's reward function is to predict text, rather than checkmate. Leela, Alpha0 their reward function is to win the game, checkmate or capture pieces. Also it goes without saying that Leela, Alpha0 cannot make illegal moves. The experiment does not need to include the whole board position if that's a problem, if that's an important point of interest. It could encode more information about squares covered by e
Cites fragmede · 2024-10 · on “OmniParser for Pure Vision Based GUI Agent”
Depends what you consider fun, and how far you take it. Some people enjoy programming more than repetitive clicking in a GUI. For a clicker game, writing a bot lets you iterate on strategies easier - is it faster to get to level 2 if I buy the upgrade for A or B first? For Trackmania, it lets you get a world record and a YouTube video with 14M views. [this video]
Cites szvsw · 2025-02 · on “AI-designed chips are so weird that 'humans cannot understand them'”
There’s a few very cool examples where someone recently used RL to solve trackmania, and ends up having to add all sorts of constraints/penalties to prevent extremely strange exploits/glitches that are discovered IIRC… been a while since I watched. [this video] [LINK]