This spring I said we're one hardware and one software generation away from having Opus 4.6-like LLM/AI tools at developers’ local computers, and that it will happen between 12 and 18 months from then. Looks like we're pretty much on schedule - and enthusiasts / bleeding edge addicts will be there sooner.
Here's what's pretty much known about how that will look like. From the hardware side:
A minimum of 128 GiB of RAM dedicated to the LLM, preferably 192 GiB or 256 GiB
Because of that, it will most likely be a unified memory architecture (UMA) system, like AMD's “Halo" series or Apple Silicon.
A minimum of 200 GB/s of memory bandwidth, preferably 500 GB/s
Because of that, it will most likely be a memory controller at least 256-bit wide, and preferably 384-bit or 512-bit. Again, AMD or Apple silicon.
On the software side, it will be:
Token generation rate of at least 60 tok/s and, more importantly, prompt processing rate of at least 500 tok/s. If I had to say which is more important to improve, I'd say prompt processing.
Because of that it will be a MoE LLM with between 120 B and 200 B total parameters and between 10B and 15B active parameters
A model with different levels of thinking effort, so it can be used for low-effort and high-effort tasks.
A model with image processing (vision).
At this time, the very minimum / lower ends of these requirements are actually possible to achieve with off-the-shelf component! I'd say it's still not worth the buy-in unless you really need the bleedi g edge in local model use, but it's possible.
On the hardware side, AMD's Strix Halo APU with 128 GB of LPDDR5X RAM will do it. On the software side - between America's decline and Europe's suicidal wish to regulate itself to death, Chinese companies have picked up the baton and we have multiple options.
At this time, the tech-preview model Qwen3.8-Flash-Next, released today, is a strong choice, but GLM and DeepSeek models are also in the game, especially if they manage a similar size shrink.
Their benchmark results are probably benchmaxxed (I.e. slightly unrealistic), but if this is the direction they're going, version 4 will be golden.
This time next year, AMD should have released Medusa Halo APUs, which will probably reach even the higher performance estimates, and that, ladies and gentlemen, is the local-model grail.
