Hybrid Mamba-2 + MoE on GPU — including a larger-than-VRAM mode that stays deterministic.
PARAMETERS
30B (3B active)
ARCHITECTURE
Hybrid SSM
QUANT
Q4_K_M / TurboHybrid 4.06 bpe
VRAM
18–24 GB (any budget via unified memory)
Cascade-2 mixes Mamba-2 state-space layers with sparse MoE — the architecture class powering the fastest agentic decoders. IronWorks runs the full hybrid natively on GPU: warp-level SSM scan kernels, fused multi-expert GEMVs, and graph-captured decode.
It is also the showcase for unified memory: the 24.7 GB Q4_K_M file does not fit a 24 GB card. Where OS paging gives erratic 2–3 tok/s, IronWorks' expert LRU cache with learned pinning delivers deterministic throughput at 98–99% expert hit rates — same weights, exact outputs, explicit budget.
# CLI (CUDA build)
cargo install ironworks-cli --features cuda
# or build from source
git clone https://github.com/nervosys/IronWorks && cd IronWorks
cargo build --release -p ironworks-cli --features cudaIRONWORKS_MOE_CACHE_GB=12 \
iwx complete -m Nemotron-Cascade2-30B-A3B-Q4_K_M.gguf --gpu \
-p "Hi" -n 400iwx complete -m Nemotron-Cascade2-30B-A3B-Q4_K_M.gguf --gpu \
--vram-budget 20G -p "Hi" -n 400| METRIC | MEASURED | NOTES |
|---|---|---|
| Decode (Q4_K_M, n=400) | ~107 tok/s | llama.cpp same-session tg96: 190.7 — kernel-efficiency gap being closed |
| TurboHybrid mode (4.06 bpe, 17 GB) | 45.2 tok/s end-to-end | IronWorks-only quantization mode; not comparable to Q4_K_M |
| Oversubscribed (24.7 GB on 24 GB) | 6.56 tok/s deterministic | vs 2.2–3.1 erratic under OS paging; expert cache 98–99% hits, prewarmed |
RTX 3090 Ti, single GPU, greedy, same-session A/Bs 2026-06-04 (head-to-head) and 2026-07-20 (unified memory).
Expert usage statistics persist to <model>.iwx_usage; second and later runs start pre-warmed (+40% measured over cold cache).