the magicians

The next “AI thinkpiece”, six months after the last one, which is here.

The LLMs continue to get better and better; I have not seen Opus 5.5 make a mistake in some time. Where does this leave us?

The code is going to appear faster than ever before, which is sort of crazy. We still don’t seem to have reached “peak code”, so the Jevons Paradox appears to be holding — and lord knows there are a lot of terrible pieces of software out there that could be improved.

Things other than code, like writing, should still not be done with LLMs. Research, perhaps; organization of raw data, perhaps — but sending a 2,000 word doc of slop to someone is rude and useless. Code review is … well, something’s going to happen to it, it can’t survive as is.

The hosting is now possible locally, but slower. I can run QWEN 9B on my Mac Mini; it’s not fast, but it runs, and the installation was three lines.

But, as of right now, there’s still a problem. As I said last time, “we should be careful to take care of our minds, as they’re all we’ve got”. I do not see a way that produces code with LLMs that also exercises our minds. There’s a very strange bad outcome where the frontier labs go down at the same time that a complicated incident happens at <Company X> … and then people at <Company X> don’t know how to analyze their systems anymore. This was always a bit possible; I’ve said many times that my career was enabled by Stack Overflow, and of course there’s an XKCD.

I probably don’t need to know how certain parts of my stack work — networking, say, has never been my forte. But at some point you need to go down a layer — and if the only way to do that is to ask an LLM to do it, we probably won’t remember how to do it. How will that impact our adorable wetware brains? Probably not in a good way.