this post was submitted on 12 Jul 2025
359 points (95.4% liked)
Programming
21711 readers
398 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Reading the paper, AI did a lot better than I would expect. It showed experienced devs working on a familiar code base got 19% slower. It's telling that they thought they had been more productive, but the result was not that bad tbh.
I wish we had similar research for experienced devs on unfamiliar code bases, or for inexperienced devs, but those would probably be much harder to measure.
I don't understand your point. How is it good that the developers thought they were faster? Does that imply anything at all in LLMs' favour? IMO that makes the situation worse because we're not only fighting inefficiency, but delusion.
20% slower is substantial. Imagine the effect on the economy if 20% of all output was discarded (or more accurately, spent using electricity).
I'm not saying it's good, I'm saying I expected it to be even worse.
Yes it suggest lower cognitive load.
1% slowdown is pretty bad. You'd still do better just not using it. 19% is huge!
Using a tool that lowers your productivity by 1/5 instead of not using it at all is "not that bad" to you? A tool that costs an awful lot to run, requires heavy security compromises, too?
On the other parts… experienced devs on unfamiliar code base is a common task. They just get familiar with the code base in a few weeks. And they can base their familiarity on actual code, not hallucinated summaries which would be meaningless.
Inexperienced devs are not going to improve with something else doing the job for them, and LLM without guidance is not able to do any substantial work without deviating into garbage first, then broken garbage second. We already have data on that too.