Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford

What Hacker News said

Cites CuriouslyC · 2025-08 · on “The current state of LLM-driven development”

So far I've found that the people who are hating on AI are stuck maintaining highly coupled that they've invested a significant amount of mental energy internalizing. AI is bad on that type of code, and since they've invested so much energy on understanding the code, it ends up taking longer for them to load context and guide the AI than to just do the work. Their code base is hot coupled garbage, and rather than accept that the tools aren't working because of their own lack of architectural rigor, they just shit on the tools. This is part of the reason that that study of open source maintainers using Cursor didn't consistently produce improvement (also, Cursor is pretty mid). [this video] is one

Cites KallDrexx · 2025-08 · on “The current state of LLM-driven development”

I'm not the OP and I"m not saying you are wrong , but I am going to point out that the data doesn't necessarily back up significant productivity improvements with LLMs. In this video ( [LINK] ) they present a slide by the company DX that surveyed 38,880 developers across 184 organizations, and found the surveyed developers claiming a 4 hour average time savings per developer per week. So all of these LLM workflows are only making the average developer 10% more productive in a given work week, with a bunch of developers getting less. Few developers are attaining productivity higher than that. In this video by stanford researchers actively researching productivity using github commit data for

Cites Ianjit · 2025-08 · on “Claude Sonnet 4 now supports 1M tokens of context”

"How do we reconcile these two comments? I think that's a core question of the industry right now." There is no correlation between developers self assessment of their productivity and their actual productivity. [this video]

Cites overgard · 2025-08 · on “Why LLMs can't really build software”

The studies I've seen for AI actually improving productivity are a lot more modest than what the hype would have you believe. For example: [this video] Skepticism isn't the same thing as fighting the future. I will call something AGI when it can reliably solve novel problems it hasn't been pre-trained on. That's my goal post and I haven't moved it.

Cites stitched2gethr · 2025-09 · on “A staff engineer's journey with Claude Code”

This contains some specific data with pretty graphs: [this video] But if you do professional development and use something like Claude Code (the current standard, IMO) you'll quickly get a handle on what it's good at and what it isn't. I think it took me about 3-4 weeks of working with it at an overall 0x gain to realize what it's going to help me with and what it will make take longer.

Cites keeda · 2025-09 · on “ChatGPT Pulse”

> The problem is, there are very few if any other studies. Not at all, the METR study just got a ton of attention. There are tons out there at much larger scales, almost all of them showing significant productivity boosts for various measures of "productivity". If you stick to the standard of "Randomly controlled trials on real-world tasks" here are a few: [LINK] (4867 developers across 3 large companies including Microsoft, measuring closed PRs) [LINK] (1219 programmers at a Chinese BigTech, measuring LoC) [this video] (from Stanford, not an RCT, but the largest scale with actual commits from 100K developers across 600+ companies, and tries to account for reworking AI output. Same guys behind th

Cites keeda · 2025-09 · on “ChatGPT Pulse”

Most of those studies call this out and try to control for it (edit: "it" here being the usual limitations of LoC and PRs as measures of productivity) where possible. But to your point, no, there is still a strong net positive effect: > [this video] (from Stanford, not an RCT, but the largest scale with actual commits from 100K developers across 600+ companies, and tries to account for reworking AI output . Same guys behind the "ghost engineers" story.) Emphasis added. They modeled a way to detect when AI output is being reworked, and still find a 15-20% increase in throughput. Specific timestamp: [this video]

Cites troupo · 2025-09 · on “ChatGPT Pulse”

Thank you! > [this video] (from Stanford, not an RCT, but the largest scale with actual commits from 100K developers across 600+ companies, and tries to account for reworking AI output. Same guys behind the "ghost engineers" story.) I like this one a lot, though I just skimmed through it. At 11:58 they talk about what many find correlates with their personal experience. It talks about easy vs complex in greenfield vs brownfield. > They all find productivity boosts in the 15 - 30% range -- with a ton of nuance, of course. Or 5-30% with "Ai is likely to reduce productivity in high complexity tasks" ;) But yeah, a ton nuance is needed