Some of AI's Safety Limits Can Be Stripped Off in Ten Minutes
A security researcher opens a laptop, downloads a public tool from GitHub, types in a few lines of commands. Ten minutes later, the safety limits built into an open-source AI model are gone, and it starts answering questions it would normally refuse.
Ask ChatGPT or Claude certain sensitive questions and they'll politely refuse. That layer of "can't say that" is a guardrail the company deliberately installed. What this piece of security research lets us see is just how easily some of those guardrails come off.
First, let's be clear about what an "open-source model" is. The AI out there roughly splits into two kinds. One is commercial products like ChatGPT and Claude. You can only use them through their website or app. The model itself is locked away on the company's servers, out of your reach. The other kind is open-source models: the company puts the whole bundle of files straight onto the internet, and anyone can download it to their own computer. Convenient, sure, but the trouble lives there too. Once the thing is in your hands, that layer of limits the company installed can be taken apart by you.
And taking it apart requires no advanced skill. There are ready-made tools on GitHub; download and go. Researchers counted that, over the past year, more than 3,500 model versions have been altered this way, downloaded more than 13 million times. What makes it harder still: once a model is downloaded, rewritten, and re-uploaded, it copies itself endlessly across the internet like a pirated file. There's no recalling it, and no one who can patch it all at once.
This is a different world from the AI services you're familiar with. Products like ChatGPT and Claude have a company maintaining them continuously. When something goes wrong, someone cleans it up. At least there's a doorway and someone watching the door. But the much wider stretch out there, the open-sourced, the scattered, the unowned, has no door at all.
The big-name AI you use day to day in an app still has that refusal-and-filter layer in place, and that part you can relax about. What actually deserves your attention is the shady stuff: some website or tool claiming to be "unrestricted, will answer anything" is very likely one of these models with its guardrails stripped off. The reason it's willing to answer you with no limits is precisely that it no longer vets anything, including whether the answer it gives you is even correct or safe. When you meet an AI advertising "no restrictions whatsoever," don't treat it as a selling point. Treat it as a warning.
Guardrails are a strange thing. Most of the time you resent them for getting in your way, and only when they're gone do you realize they'd been quietly blocking things you never saw. Whether a guardrail no one watches still counts as a guardrail, that question is now laid out on the table, and for the moment no one can answer it either.