Nothing leaves your PC.

🔥 hot experts on the card  →  🌡️ warm in RAM  →  ❄️ cold on the SSD

A fork of Strata by Niko1221 and contributors (MIT). Strata keeps Qwen’s experts in RAM and reads from the SSD when RAM is short; its docs call that path much slower. This fork works on that path: it spreads the experts over the card, RAM and the SSD and guesses ahead which ones to read, so the 78 GB IQ3_S runs on 30 GB of RAM at 50-70 tokens/s.

Can Qwen3.8-Flash-Next run on an RTX 3060 12 GB with low RAM?

Yes. On one RTX 3060 12 GB with 30 GB of RAM and an NVMe SSD, this fork runs Qwen3.8-Flash-Next (Qwen 3.8 Flash Next, 125B) as the 78 GB IQ3_S at 50-70 tokens/s (131K context; 262K fits). The experts that do not fit in RAM are read from the SSD, and a look-ahead guesses ahead which ones. Measured on one PC: see How fast and the “Not tested” list at the bottom.

How much RAM does it need? This fork was measured on 30 GB. Strata’s setup recommends 64 GB for IQ3_S and says that with less RAM it reads the rest from the SSD, much slower.

Does it need a cloud service?…