A project published on GitHub claims it can run an 80-billion parameter language model on a 32 GB laptop, a machine whose memory is smaller than the model's own 48 GB weight file. The tool, called Janas-LLM, reports 37 tokens per second from Qwen3-Next-80B when a draft model helps guess upcoming text, and 24 tokens per second without drafts. On the same laptop, the llama.cpp engine managed 2.8 tokens per second when writing. The figures come from the developer's own tests, so they should be read as claims until other people reproduce them.
How Janas-LLM keeps a huge model in memory
According to the project's GitHub page, Janas is a family of artificial intelligence tools written in C, with no dependencies beyond the C library. Its first component, Janas-LLM, is built to run large mixture-of-experts language models on ordinary computers, with or without a graphics processor.
The design is the key to the claim. The weights stay on disk. The expert sub-networks that each token needs are streamed from an NVMe solid-state drive, and the ones that keep being requested stay cached in memory. Mixture-of-experts models split their capacity into many specialist sub-networks and activate only a handful for any given token, which is what makes this kind of streaming practical rather than hopeless.
The benchmark figures
The headline numbers compare Janas-LLM with llama.cpp on the same machine. The project reports the following for Qwen3-Next-80B:
- Writing with drafts: 37 tokens per second, against 2.8 for llama.cpp.
- Writing without drafts: 24 tokens per second.
- Reading a prompt: 62 tokens per second, against 16 for llama.cpp.
Janas-LLM was faster than llama.cpp on all 13 models compared in the Q4_K_M quantization, with one exception. The three-bit version of Qwen3-30B reads a prompt more slowly than llama.cpp, because its Q3_K weights do not yet go to the GPU's fast prompt path. Even so, the project says that model writes three times as fast.
The llama.cpp result required workarounds. With its usual build, llama.cpp ran out of memory as soon as the GPU took part, even with the experts kept on the CPU, because its backends copy weights while repacking them. The developer reports that llama.cpp only ran the 80B model after a rebuild without repacking and two extra options that leave expert data in the file and read it through the page cache. Janas-LLM ran the same model with no special options, keeping about 19 GB of frequently used experts in a cache and reading the rest from disk.
Test conditions and their limits
The Linux tests ran between 9 and 11 October 2026 on a laptop with Debian 13, an Intel Core Ultra 9 185H processor, its integrated Arc GPU, 32 GB of memory and an NVMe SSD. llama.cpp was build 2b18470 from 18 September 2026, using its Vulkan backend on the same GPU. For most of the comparison, the project's benchmark tool, janas-bench, reports the mean of three rounds, while the Qwen3-Next-80B figures are single rounds.
The developer also notes that the two tools do not measure in exactly the same way. llama-bench writes without first processing a prompt, whereas janas-bench writes after 1,024 tokens. The comparison covers one machine only.
Conditions on the machine matter too. The expert cache takes whatever memory is free when the model opens, and the rest is read from disk while the model answers. In one test with a browser and an editor open, the same model kept about three quarters of its speed but waited five times longer on the disk. The project advises benchmarking on an idle machine.
Windows results and what remains untested
The project also reports results from the same laptop under Windows 11, measured on 12 October 2026 with Intel's own graphics driver. There, Janas-LLM wrote as fast as it did on Linux, at 25.6 tokens per second on Qwen3-4B and 34.2 on Qwen3-30B-A3B. Prompt reading reached 132 tokens per second on the mixture-of-experts Qwen3-30B-A3B. On the dense Qwen3-4B, whose whole blocks go to the GPU, reading reached 346 tokens per second, which the project puts at 1.8 times the 193 figure it cites for the same model.
The developer lists several setups that have not been tried: other CPUs, especially those without AVX-VNNI or made by AMD, discrete GPUs, slower disks, and machines with more or less memory. The engine chooses its thread counts and settings by measuring the machine, so results on other hardware may differ.
Getting started and what comes next
Janas-LLM is free software under the GNU General Public License, version 3 or later. It runs on Linux x86-64 (Ubuntu 22.04, Debian 12 and later) and on Windows 10 (1903 or later) and Windows 11 on x86-64, where users can download a zip file and start a chat from a terminal. Models come already converted from Hugging Face.
The practical takeaway is that running a model larger than installed memory now depends more on disk speed and cache size than on RAM alone. Whether these speeds hold on other machines is still open. The developer, Maurizio Cammalleri, asks users with different hardware to send reports from janas-bench, which will show how far the figures travel beyond one laptop.
Related: Open-Source n8n Bot Calculates Czech Tax on Broker Investments · Software License Renewal Tracker Sends Timely Reminders
Comments (0)