Edge0 Ships 145B and 120B Open-Weight Models Claiming Single Consumer-GPU Operation via SSD Streaming
The latent-MoE models derived from Qwen3.8-Flash-Next stream Q3_K weights from SSD to fit a 24 GB RTX 5090, using a scheduling engine and 1.26B MTP speculative decoding; all performance figures are vendor-reported.
Edge0 has released Edge0-145B and Edge0-120B, latent mixture-of-experts models derived from Qwen3.8-Flash-Next, with weights on Hugging Face and an Apache-2.0 framework repository on GitHub. The headline claim is that the 145B model runs on a single consumer NVIDIA GPU: a Q3_K variant with 40.81 GiB of weights is streamed from SSD on demand, keeping peak memory within a 24 GB RTX 5090 D.
The approach builds on the streaming-MoE framework Edge0 has published since September, which offloads experts to SSD and predicts routing to pre-fetch them. A scheduling engine named edge0-strata coordinates CPU, GPU and PCIe traffic, while a 1.26 billion parameter MTP head provides speculative decoding. The models ship with a 256K context window and int8 KV cache.
If the claims hold up, the work challenges the assumption that large parameter counts require large memory footprints, since peak memory is bounded by the active expert set rather than total parameters. That would change the calculus for on-premises inference in offices with modest hardware. Independent benchmarks do not yet exist, and throughput streaming from SSD will be sensitive to disk speed, so the figures should be treated as unverified vendor claims for now.