Posts / ai
The boring plumbing is the interesting part
A post on r/singularity caught my eye this morning. The headline said Google researchers had shown an AI that improves its own software, remembers what worked, and keeps building on that even when you swap the underlying model from Claude to Gemini. The word “improves” is doing a lot of lifting there. So is “its own”.
The comments sorted themselves into the usual camps within about four minutes. One camp said recursive self-improvement had arrived. Another said it was a nothingburger. A third wanted to know, on a scale of one to ten, how big a deal it was. That last one is a fair question, and I’ve asked it myself about plenty of things I don’t understand.
Someone then explained what the paper actually does, and it was more useful than the headline. The model’s weights don’t change. What gets rewritten is everything around it: the prompts, the tools, the scripts, the memory. The new part is a log kept by the thing doing the rewriting. It records every edit, why it was made, what the system expected to happen, and what did happen. Swap the model or the benchmark and the optimiser can look back and see which fixes were for which problem, instead of inheriting a pile of unexplained tweaks.
The gain was about 2.36 points on the benchmark across ten coding runs. Small, but apparently statistically solid. One commenter corrected someone for calling it 2.36 per cent, which is exactly the sort of pedantry I’m here for.
I work in DevOps. When I read “a log of every change, with the reason and the expected outcome,” I didn’t think about superintelligence. I thought about the commit history on a repo I inherited three jobs ago, where the messages said things like “fix” and “fix again” and “actually fix”. Somewhere in that repo was a config line that nobody dared remove because nobody remembered what it was protecting us from. We left it there for four years. It was probably load-bearing. Possibly it was a typo.
That is the real problem the paper is chewing on. Gains reset when the context is lost, and losing context is the most ordinary failure in all of engineering. Teams lose it when someone leaves. Agents lose it when the model gets upgraded. Writing down the why is dull work, and it’s the work that lets improvements stack. A commenter called it boring plumbing, and I agree. I also think boring plumbing has a better track record of changing the world than the dramatic stuff.
Others in the thread pushed back, and I think they were right to. One said the missing ingredient for real recursive self-improvement is research taste, the knack of picking which idea is worth trying. Nothing here measures that. Another said this is an obvious idea people have had since 2020, and the surprise is that anyone thought it worth a paper. Both can be true. Obvious ideas still need someone to run them properly and report the numbers. I’d rather have a modest result with a p-value than a confident one with a vibe.
Then there’s the part I can’t resolve. I’m fascinated by this stuff. Watching the tooling get better week to week is the most interesting thing in my working life since I first got a container running. I also know every efficiency gain in an AI pipeline gets spent on doing more, not on using less power. Cheaper improvement means more improvement, and more improvement means more compute. I don’t know how those two facts add up. I suspect nobody does yet, and anyone who says they do is selling something.
Further down the same day’s feed was a video of a humanoid robot cooking a burger, sped up four times. The comments were mostly jokes, then mostly darker. It was teleoperated, so a person somewhere was piloting it, and the footage is probably being collected to train a model that won’t need the person. Several commenters pointed out that the pilot is likely on a low wage in another country. Some of the jokes about that were not good. The underlying point isn’t a joke, though. A lot of what gets called automation is a person you can’t see, doing a job you haven’t thought about, producing the data that makes their job unnecessary. I’m not sure what the right response is. I’m fairly sure “isn’t it funny” isn’t it.
So the harness paper doesn’t scare me, and it doesn’t thrill me either. It’s a small, honest improvement in how machines keep notes. I find that reassuring in a way I didn’t expect. The notes are legible, and a human can read them and see why a change was made. If this is what the road to self-improving systems looks like, I’d like the logs to stay readable all the way down.
I’ll go and write a proper commit message now. It won’t say “fix”.