The 8-GB-card class leader: 26B-class quality with ~4B active parameters per token.
PARAMETERS
26B (3.8B active)
ARCHITECTURE
MoE
QUANT
QAT Q4_K_XL (13.2 GB)
VRAM
8 GB+ with expert offload
Gemma 4 26B-A4B routes each token through a small subset of experts (~3.8B active), and Google's QAT quantization keeps the 4-bit file at 13.2 GB with minimal quality loss — the community-standard pick for 8 GB GPUs.
IronWorks supports it with a hybrid path (CPU prefill + GPU decode) and a learned MoE expert cache: expert routing counts persist to a sidecar file, so the hottest experts are pre-warmed into VRAM on load (measured +40% over a cold cache on our hybrid-MoE models, 98–99% hit rates).
# CLI (CUDA build)
cargo install ironworks-cli --features cuda
# or build from source
git clone https://github.com/nervosys/IronWorks && cd IronWorks
cargo build --release -p ironworks-cli --features cudaIRONWORKS_MOE_CACHE_GB=6 \
iwx complete -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --gpu \
--vram-budget 7G \
-p "Plan a two-week Japan itinerary." -n 512Routing statistics persist to <model>.iwx_usage automatically when the expert cache is enabled; subsequent loads pre-warm the hottest experts.
# First run learns the routing distribution; later runs start warm.
IRONWORKS_MOE_CACHE_GB=6 iwx chat -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --gpu| METRIC | MEASURED | NOTES |
|---|---|---|
| Status | Preview | coherent output via hybrid CPU-prefill + GPU-decode; expert-cache fast path in active development |
| Competitive bar (llama.cpp -cmoe) | 20 tok/s on 8 GB RTX 4060 | community-measured; the target this recipe is being tuned against |
Expert-cache mechanics measured on hybrid-MoE (Cascade-2) 2026-07-20; 26B-A4B throughput pending.
Marked PREVIEW: decode-side expert-cache acceleration for this family is in active development; published throughput numbers will follow the same same-session A/B standard as every other recipe.