Open Weights Are Not Frontier Compute

Why the garage-hacker theory of open-weight AI skips over every difficult part of the problem.

At the Clinton Global Initiative 2026, former Secretary of State Hillary Clinton and Reid Hoffman discussed whether open-weight large language models pose additional risks to the public through cyberattacks. Clinton later described the discussion by saying that while we are used to thinking of open-source models as an “unadulterated good,” AI models can pose additional dangers. Hoffman gave an example of what he thought one of those dangers looked like. With an open-weight model, he said:

This sounds very scary. It also doesn't make much sense. There are basically two different claims buried in this hypothetical: first, that someone can take a locally runnable open-weight model and retrain it until it is capable of discovering entirely new cyberattacks, and second, that this person can then deploy that capability at the scale demonstrated by frontier AI labs. Both skip over the same problem: having the weights is not the same thing as having the model capability or the compute that produced the scary demonstrations in the first place.

I say this as someone who loves open-weight models. I run them locally every day. Qwen and Z.ai are doing incredible work, and the fact that I can run models like Qwen3.8-27B or GLM locally on hardware I own is astonishing. But there is a universe of difference between a very good 27-billion-parameter model sitting under my desk and the frontier systems OpenAI is running across data centers.

That distinction seems to be getting lost.

Retraining With What?

The strangest part of Hoffman's example is the idea that he would “retrain” the model on attack vectors that haven't been tried before so that nobody knows they're coming. Retrain it on what? Fine-tuning requires training data. If these attacks have genuinely never been tried before, you do not have examples of them sitting around waiting to be added to your training set. Somebody has to invent them first.

If Reid Hoffman is sitting in his garage personally writing tens of thousands of novel exploits, testing them, determining which ones work and then assembling them into a high-quality training dataset, then I am no longer particularly concerned about his open-weight model. I am concerned that Reid Hoffman apparently has tens of thousands of undisclosed exploits. He has already done the difficult part.

Maybe the argument is instead that the model itself discovers the novel attacks, and then you retrain it on the attacks it discovered. But that just moves the problem backwards one step. If the model is already capable of autonomously finding novel exploits, then the capability we are supposedly trying to create through retraining already exists. The interesting question is not how you fine-tune the model afterward. It is how the model discovered the exploit in the first place.

There is a real way to train AI systems for cyber work, but it is almost the opposite of what is being described here. You build environments where an agent is given some objective: find a vulnerability, get a file off a server, escalate privileges, complete a capture-the-flag challenge. The agent tries things, most of them don't work, and successful behavior gets rewarded. Over time, the model gets better at the general process of probing a system, finding weaknesses, debugging failed attempts and trying something else. Then, at inference time, you point that model at a system it hasn't seen before and let it search.

That search might result in a novel exploit. But the model was not “trained on an attack nobody had seen before.” It was trained to become better at searching for attacks nobody had seen before. That difference is important because the search itself costs compute. A lot of compute.

And before we even get to that problem, there is another one: fine-tuning a small model for cybersecurity does not magically give it the reasoning ability of a frontier model. Open weights give you control over the model. You can remove refusals. You can change its behavior. You can specialize it. You cannot jailbreak capability into it. If a model gets confused halfway through a moderately complicated coding task, removing its refusals does not suddenly make it able to autonomously reason through a novel exploit chain for six hours.

This is something that is very obvious when you actually run open models locally. They are incredibly useful. They are also not magic. I can give Qwen every permission imaginable. It still occasionally writes a BeautifulSoup scraper and then forgets what HTML element it was looking for three tool calls later. No disrespect to the Qwen team. I love Qwen. That's why I use it. But “the model will do anything you ask it to do” and “the model is capable of doing anything” are two completely different statements.

“Deploy at Scale” With What?

This is where the hypothetical becomes even stranger. The cyber capabilities coming out of OpenAI and other frontier labs aren't simply the result of having a very intelligent model. They are running huge numbers of agents. Those agents fan out: one investigates an API, another looks at package dependencies, another tries an upload endpoint, another enumerates services, another tries a completely different approach. Most attempts fail. That's okay. If you can afford thousands of attempts at the same time, you don't need every agent to succeed. You need one of them to find something useful, share it with the others, and let the swarm continue from there.

Hugging Face's reconstruction of the OpenAI incident recovered about 17,600 attacker actions. Hugging Face's own conclusion was that many of the individual weaknesses were fairly ordinary. What changed the defensive problem was the volume. The system could test huge numbers of paths, abandon failures and keep going at machine speed. This is the part of AI capability discussions that I think gets badly underestimated. The model matters. The amount of inference you can throw behind the model also matters.

We got an unusually good example of the scale involved when OpenAI published details about its solution to the Navier-Stokes Millennium Prize Problem. Navier-Stokes obviously isn't a cyberattack, and I am not claiming that hacking a company requires 130 billion tokens. What makes the example useful is that OpenAI actually told us how much inference it used. The group that produced the solution involved on the order of 10,000 concurrent agents. Those agents could execute code and access a cached copy of the internet. OpenAI split them into groups exploring different approaches and later cross-pollinated useful findings between those groups. The Navier-Stokes effort generated 2.7 million agent messages and approximately 130 billion output tokens, and the result arrived about 88 hours after the first agents were launched. And this wasn't Astra: OpenAI says the internal model used for the result was significantly more capable than GPT-6 Astra.

That is what “at scale” looks like. So let's make the garage-hacker argument as favorable as we possibly can. Forget the capability difference. Pretend that tomorrow China releases an open-weight model that is exactly as intelligent as OpenAI's internal model. Not close. Not competitive on benchmarks. Exactly as good. You download it. Now what? You still have to run the thing.

At 60 output tokens per second, generating 130 billion output tokens would take about 69 years. Obviously you wouldn't run an agent swarm using one inference stream, so let's add concurrency. Two DGX Sparks can be purchased for around $10,000, and highly optimized community benchmarks with GLM-5.3-Flash have demonstrated roughly 200 aggregate output tokens per second across concurrent generations. At around 200 tokens per second, producing 130 billion output tokens takes roughly 20 years. Fine. Let's get ridiculous. Spend luxury-car money on professional Blackwell GPUs and push aggregate generation toward 800 tokens per second. Now we're down to roughly five years. OpenAI did it in days.

Again, 130 billion tokens is not some universal unit of intelligence, and it is not the minimum cost of a cyberattack. Different problems require radically different amounts of search. The point is the scale difference. Even after we grant the garage hacker the absolutely absurd assumption that his locally runnable open-weight model is as intelligent as OpenAI's unreleased frontier research model, there is still another enormous bottleneck. OpenAI can run something on the order of 10,000 agents at the same time. A guy in his garage cannot. That isn't a software licensing issue. It's an infrastructure issue.

This is why I think focusing entirely on whether someone can remove the refusals from a model misses what is actually producing these results. An unrestricted model is not necessarily a frontier model. And a frontier model running on one workstation is not the same system as a frontier model backed by thousands of simultaneous inference workers. The impressive results increasingly come from both. You need a model smart enough to make useful progress on the problem, and then you need enough test-time compute to search a huge number of possible solutions. Publishing the weights does not give you either one automatically.

Bottom Line

There are legitimate security concerns with open-weight models. An open model can be run privately. Nobody can revoke your API key. Nobody can inspect what you're asking it to do. You can remove its refusals, fine-tune it and run it continuously. Those are real differences, and they are worth talking about, but they are differences in control, not magical increases in intelligence.

There is also a much more reasonable cyber-risk argument here: open models can make existing cybercrime cheaper. A local model doesn't need to discover a zero-day to help somebody write phishing emails, modify existing malware, automate reconnaissance or exploit vulnerabilities that are already known. That threat is real. It is also very different from the claim being made in the garage hypothetical. The idea that somebody downloads a relatively small open model, removes its safeguards, retrains it on attacks that somehow haven't been invented yet, and then reproduces the massive autonomous cyber operations being demonstrated by frontier labs skips over essentially every difficult part of the process. Where did the novel attacks come from? How capable is the underlying model? How much compute was required to discover them? And where exactly is the person in the garage getting thousands of simultaneous inference workers?

Ironically, the Hugging Face incident also demonstrated why I want open models to exist. When Hugging Face investigated the OpenAI intrusion, it says the closed models it initially tried to use repeatedly refused parts of the forensic analysis because their safety systems interpreted reverse-engineering the attack as offensive cyber work. Hugging Face eventually ran the open-weight GLM-5.2 on its own infrastructure and used it to help reconstruct the attack and decode payloads. That is the tradeoff. Open weights mean the model developer no longer decides what you're allowed to do with the model. Sometimes that benefits an attacker. Sometimes it means the company being attacked is actually allowed to investigate its own attack.

But none of this changes the underlying capability of the model. Qwen without refusals is still Qwen. And even if tomorrow's Qwen is somehow as capable as OpenAI's best internal model, the person downloading it has only solved half the problem. They still have to run it. The question isn't just “do you have an open-weight model?” It's “how powerful is the model, and how many data centers do you own?”