Betelgeuse, 2x8GB DDR4 듀얼 채널 환경에서 Qwen 3.8 Flash Next IQ4_XS 모델 초당 24토큰 달성 (RTX 3060 12GB 사용)

Why switching from 1x 16GB to 2x 8GB jumped IQ4_XS from 14 tok/s $\to$ 24 tok/s:

Because Betelgeuse uses an Exclusive Host RAM Victim Cache + Async Hardware DMA (MOE_DMA=1) between the VRAM slot pool and the NVMe drive:

• When an expert gets evicted from VRAM, it lands in the host RAM victim cache (while MOE_DROP_MMAP drops duplicate GGUF pages from the OS page cache so VRAM and RAM never hold duplicates).

• On a single 16GB stick, single-channel DDR4 caps out around ~17–19 GB/s, so the CPU and the PCIe H2D DMA engine fight over one 64-bit memory channel (14 tok/s on IQ4_XS).

• On 2x 8GB dual-channel DDR4 (still 16GB total RAM, which is what 95% of normal 16GB gaming PCs actually have), host RAM bandwidth doubles, letting the GPU's hardware DMA engine pull victim-cache hits across PCIe at full speed while the upgraded oracle pre-stages cold NVMe misses in the background—jumping IQ4_XS straight to 24 tok/s.

Links & What's Next

• GitHub (Betelgeuse Engine — bel-cli & bel-server): [https://github.com/Betelgeuse-AI/Betelgeuse]

• HuggingFace (Pre-packed Alpha-Orionis-XS 3.2GB Dense + Experts bundle): https://huggingface.co/Betelgeuse-AI/Alpha-Orionis-XS

• Upstream llama.cpp PR: [https://github.com/ggml-org/llama.cpp/pull/31469]

What I'm working on next:
We're currently bringing this exact streaming + victim-cache architecture over to DeepSeek V4.1 Flash (1017)—early testing under development is already sitting around ~25 tok/s on the same hardware class.

Happy to answer any questions or have people poke holes in the benchmarks! Leave us a Star :)

나도 16GB VRAM에 32GB RAM인데.