Banning Superintelligence Is Easy. Defining It Isn't.

I’ve had a browser tab open for two days now with a Reddit thread from r/LocalLLaMA titled “Bernie Sanders proposes to ban AI.” I keep coming back to it, not because I think the bill is going anywhere (it isn’t), but because the argument underneath it is one I genuinely don’t know how to resolve.

The bill, as summarised in the thread, would ban AI systems that exceed human cognitive abilities, with a 20 year prison sentence attached. Straight away you can see the problem, and about four hundred people in that thread saw it faster than I did: what counts as “exceeding human cognitive abilities”? A calculator exceeds my arithmetic. A spreadsheet exceeds my ability to track a thousand rows of data. My phone’s autocomplete probably exceeds my spelling on a bad day. If the bar is “outperforms a human at some cognitive task,” we banned that technology decades ago and nobody noticed, because nobody enforced it.

One commenter put it well: this is a messaging bill. It’s not meant to become law, it’s meant to signal that someone in Washington is worried about the pace of this stuff and is willing to say so loudly. I get the impulse. I don’t think Sanders is wrong to be worried. I just think “ban anything smarter than a person” is the policy equivalent of trying to regulate the weather by banning clouds.

Here’s the bit that actually sat with me, though. A few people in that thread, including some who clearly like Sanders and consider themselves cautiously pro-AI, said something close to: true superintelligence should be treated like nuclear weapons, handled through international treaties, taken dead seriously, but we are nowhere near that yet, and laws written in a panic about a thing that doesn’t exist tend to land squarely on the thing that does exist. Which in this case is a bloke in his garage running a 30 billion parameter open model on a couple of second-hand GPUs, not Skynet.

I run local models on my own machine for exactly this reason, among others. Not because I think I’m building anything dangerous, I promise you my Qwen setup is not plotting world domination, it’s mostly summarising PDFs and helping me debug bash scripts I should be able to debug myself. I like that the box on my desk doesn’t send my data anywhere. I like that nobody can switch it off remotely because a subscription lapsed or a company got acquired. There’s something almost old-fashioned about it, owning your own tools outright. But laws written broadly enough to catch a hypothetical superintelligence have a habit of also catching the garage tinkerer, because the garage tinkerer is easy to find and the actual frontier labs have lawyers.

This is where I land most days when this topic comes up, and it’s not a comfortable landing: I think the concentration of AI power in three or four American companies is a genuine problem, and I also think the response to that problem shouldn’t be “make it a crime to own the tools that let ordinary people push back on that concentration.” Those two views sit next to each other uneasily and I haven’t found a way to make them stop.

There’s also the practical point, which someone in the thread made better than I could: even if you write the perfect law, vague enough to catch what you mean and precise enough not to catch a TI-83 calculator, who enforces it, and how? We’ve watched agencies get their rule-making powers gutted by the courts over the last couple of years. A regulatory approach that depends on a well-staffed, well-funded, expert agency with the authority to write and update rules as the technology moves is a good idea in a world where that agency is allowed to exist and do its job. I’m not sure we’re in that world anymore, in the US at least. Here in Australia we’ve got our own version of this fight brewing, slower and quieter, but the same tension applies: write the law too specifically and it’s obsolete in eighteen months, write it too broadly and you’ve banned Excel.

None of this means we shouldn’t try. I’d rather a government that’s fumbling towards an answer than one that’s not asking the question at all, and Sanders asking a clumsy question loudly is still better than the alternative, which is nobody in power treating this as urgent until it’s a genuine emergency. I just don’t think the answer is a length-of-a-tweet definition and a two-decade prison term. The actual work, defining what we’re scared of clearly enough to regulate it without accidentally criminalising a hobbyist with a couple of GPUs and a Sunday afternoon, is slower and less satisfying than a headline. But it’s the only version of this that doesn’t end with the wrong people going to prison and the right problem left completely untouched.