185 tok/s MoE decode at parity with llama.cpp — 30B-class quality at 3B-class speed.
PARAMETERS
30B (3B active)
ARCHITECTURE
MoE
QUANT
Q4_K_M
VRAM
18 GB full-resident
Qwen3 30B-A3B activates ~3B parameters per token across 128 experts, delivering 30B-class quality at small-model decode speeds. It is the most popular locally-run MoE family.
IronWorks runs it fully GPU-resident with device-side routing, fused expert GEMVs (gate+up+SwiGLU fused; expert kernels measured at the ~670 GB/s bandwidth ceiling), and whole-decode-loop graph capture — measured at decode parity with llama.cpp (0.99×) in same-session A/B.
# CLI (CUDA build)
cargo install ironworks-cli --features cuda
# or build from source
git clone https://github.com/nervosys/IronWorks && cd IronWorks
cargo build --release -p ironworks-cli --features cudaiwx complete -m Qwen3-30B-A3B-Q4_K_M.gguf --gpu \
-p "Compare Rust and C++ for systems programming." -n 512
iwx serve -m Qwen3-30B-A3B-Q4_K_M.gguf --port 8080iwx serve -m Qwen3-30B-A3B-Q4_K_M.gguf --port 8080 \
--kv-cache-dir ~/.cache/ironworks/kv| METRIC | MEASURED | NOTES |
|---|---|---|
| Decode (tg128) | 185.4 ± 0.6 tok/s | llama.cpp same-session: 187.0 ± 11 (0.99× = parity, tighter variance) |
| Prefill (pp512) | ~475–506 tok/s | llama.cpp leads prefill; MoE-prefill megakernel is the known remaining lever |
RTX 3090 Ti, single GPU, greedy tg128, same-session A/B vs llama.cpp CUDA build, clean-GPU protocol, 2026-06-08.