You might remember that back in what feels like the prehistoric caveman era of January 2025 (with how fast things are moving in the AI world) DeepSeek shipped a model that scored in the same range as OpenAI's best. All while trained for a GPU-rental bill that might as well have amounted to a rounding error on OpenAI's budget. And add on top of that they then handed the weights out for free. At the time some people disregared it as a fluke, a one-off from a clever lab getting lucky under strict sanctions. Eighteen months later though we're getting a repeat, but this time it's Moonshot's new Kimi K3, a 2.8 trillion parameter open-weight model just took the top spot on Frontend Code Arena. That is ahead of the infamous matter of national security Claude Fable 5 by the way. After that came Alibaba within a few days with Qwen 3.8 at 2.4 trillion parameters, also open. All of that is to say that it's clear this isn't a one-off situation. And it's making the people inside closed labs nervous.
Think about it this way, a frontier training run costs somewhere between $100M and $500M in compute alone! On clusters of tens of thousands of GPUs running for months. Before you count the research salaries, the failed runs, and the data deals. The whole closed-model business rests on the idea that this spend buys you something other people can't have for quite a while or at least long enough that you can sell access to it at a nice fat margin. When a Chinese lab under export controls matches most of that capability and gives the weights away, the margin is what gets threatened. But how are these open labs actually able to pull this off?
Distillation
The first thing people like to trow around is distillation. And it is real somewhat but it's just not the whole thing. Distillation trains a small student model to copy the outputs of a large teacher, matching not only the teacher's final answers but the full probability distribution over its next tokens. So the student inherits a lot of the teacher's learned structure without having to pay to discover it. DeepSeek did exactly this inside their own family, distilling the long chain-of-thought reasoning from their R1 model back into V3, and the technique transfers reasoning behaviour into a cheaper package at a fraction of the original cost. If you can query a frontier model through an API, you can in principle generate training data that carries a chunk of its intelligence, and that's the version of the story that lets you say 95% of the capability got copied for pennies.
The problem is that distillation doesn't quite explain the fact that you have models matching models from frontier labs. In agentic coding sessions for example these models felt on par with the best public models. You simply can't distill a capability that the teacher won't show you at the fidelity you need. Agentic coding runs are long, stateful, and full of tool calls, and copying a few thousand API responses doesn't hand you a model that holds together across a 1 million token context. Distillation gets a fast-follower most of the way to a known target on narrow tasks. It doesn't get you to the top of a blind coding arena. Something else is doing the heavy work.
Architecture - Constraints as the drivers of progress
Export controls were supposed to kneecap Chinese labs by denying them the fastest GPUs, and instead they functioned as a push for efficiency and smarter approaches because a lab that can't buy its way to more FLOPs has to earn them back in the design. This is the part of the story that closed labs least want to be true, since it means the gap isn't purely about who has more compute.
Start with the mixture-of-experts trick, which is now the industry default and which the Chinese labs push harder than anyone. A dense model runs every parameter for every token, so a 2.8 trillion parameter dense model would be unusable. Kimi K3 splits its network into 896 experts and activates only 16 per token, roughly 1.8% of the model at any moment, which means it carries the knowledge capacity of a 2.8 trillion parameter model while paying the compute of something far smaller on each forward pass. K2 activated about 2% of a one trillion parameter model, so K3 grew the total size while cutting the activation ratio, scaling capacity and holding the per-token cost roughly flat. The 880 experts sitting idle on any given token are cheap to store and expensive only when the router picks them, and getting that router to pick sensibly from 896 options without collapsing onto a handful of favourites is its own hard problem that Kimi solved with a percentile-based balancing scheme rather than the usual bias nudging.
Then there's attention, which is where the recent gains concentrate. Standard attention costs scale with the square of the context length, so a 1 million token window under vanilla attention is financially absurd, and the Chinese labs have been the most aggressive at replacing it. DeepSeek's Multi-head Latent Attention compresses the key-value cache into a low-rank latent vector, which is the memory that dominates inference cost at long context. Kimi K3 goes further with a linear-attention variant they call Kimi Delta Attention, interleaved three-to-one with full attention layers, which gives the model a recurrent-style memory that decays and updates channel by channel instead of recomputing attention over the whole history, and the reported result is six times the decoding throughput without a drop in quality. FP8 training, latent compression, sparse routing, linear attention, none of these is a marketing gimmick, they're the difference between a model that's economically viable to train and serve under a compute ceiling and one that isn't, and they were driven into production by labs that had no choice.
There's a fast-follower advantage layered on top of the architecture, and it's more so structural than clever. The lab that gets somewhere first pays an exploration tax, it runs the dead-end experiments, trains the models that don't work, and discovers by spending which directions are real. The second lab knows the destination exists before it starts walking, which collapses the search space, and knowing that a 2.8 trillion parameter MoE with linear attention can hit frontier coding scores is worth an enormous amount when you sit down to build one. This is why the lag between a closed capability appearing and an open model matching it keeps shrinking toward the three-to-six month range, and it's a lag that compounds against whoever's paying to go first.
But still how is it so cheap?
The framing that a $100M model gets copied for a few million is nice, and it survives first contact with the details but only if don't look closely. DeepSeek V3's widely quoted $5.5M is 2.788 million H800 GPU-hours at two dollars an hour, and DeepSeek said plainly in their own report that this figure excludes the capital cost of the hardware, the prior research, the ablations, and the runs on architectures that didn't ship. The marginal cost of the final successful run was cheap. The all-in cost of being a lab capable of producing that run was not, and the two get conflated every time someone repeats the number.
Serving is where the cheap story gets thinner. Kimi K3 was popular enough on release that Moonshot's own GPUs ran out and they started turning away paying customers, with the paid tiers sold out, which is not what unlimited cheap intelligence looks like in practice. Dean Ball noted the model felt token-hungry in use, and independent measurement backed the concern, artificial analysis clocked a 51% hallucination rate and a tendency to emit far more tokens than the task needed, so a model that's cheaper per token can still cost you more per finished task while being wrong half the time on the things it makes up. The weights being free doesn't make the model free to run, you still need a rack of data-center GPUs to hold a 2.8 trillion parameter model even quantized, and the benchmark numbers themselves came partly from Moonshot's own coding harness while competitors ran in different ones, which is the sort of asymmetry that flatters the home team. The capability is real and the discount is real. The discount is smaller and messier than the headline, and the headline is the version doing the political work.
What the closed labs are actually defending
Once you see that the open models are cheap-ish, fast-following, and genuinely good, the closed-lab reaction reads as a business response dressed up as a safety argument. Dean Ball over on twitter stirred up quite a drama by calling open-weight models "inherently decelerationist" because they "deter further capital expenditure", which is another way of saying they wreck the return on a $500M training run by making an approximate copy free within months, and he laid out plainly what he would like to be done about it: don't ban open source, which is politically dumb and legally hard, just have federal agencies issue enough soft warnings about backdoors in Chinese models that every regulated enterprise quietly backs away. Manufacture regulatory risk, let the fear do the gatekeeping. Prediction markets put the odds of an outright US ban on Chinese models around 29%, so the FUD route is the more likely one precisely because it's deniable.
You now have this situation where at the World AI Conference the Chinese state positioned itself as the loudest advocate for open and free AI, while Silicon Valley argued for regulation and control. Now it must be said that both positions are self-interested rather than principled. China open-sources partly because its labs are a step behind the frontier and nobody pays for a sub-frontier proprietary model, partly because export controls left them short on the inference compute you'd need to run a big paid API anyway, so giving the weights away turns a weakness into distribution and mindshare.
Where the closed labs still have ground is the part that isn't the raw model. As the price of a given level of intelligence falls toward zero, static weights stop being defensible on their own, and the moat moves to things open weights don't hand you: deep integration with a company's private data, agentic loops that own multi-step execution across real tools, and low-latency distribution built into products people already use. That's the actual bet OpenAI and Anthropic are making, that the model becomes a commodity and it's looking more and more like a reasonable bet. It's also a retreat from the position they held two years ago, when the model itself was the product and the moat, and the reason for the retreat is a stack of open-weight models from labs that were supposed to be too constrained to build them.
The honest summary is that open weight models didn't win by being cheaper, they won by being free, and improving on a schedule that makes the closed premium hard to justify for a growing share of uses.
