The small-model showcase: measured 202 tok/s, +29% over llama.cpp at its peak.
PARAMETERS
4.6B (effective 2B)
ARCHITECTURE
Dense
QUANT
Q4_K_M
VRAM
4 GB+
Gemma 4 E2B is Google's efficient small model — per-layer embeddings give it 2B-class memory traffic with larger-model quality.
IronWorks' E2B path demonstrated the largest measured lead over llama.cpp of any model in the suite (+29%, 2026-05). A mid-2026 regression from shared-path changes is being bisected; the recipe below is current best practice.
# CLI (CUDA build)
cargo install ironworks-cli --features cuda
# or build from source
git clone https://github.com/nervosys/IronWorks && cd IronWorks
cargo build --release -p ironworks-cli --features cudaiwx complete -m gemma-4-E2B-it-Q4_K_M.gguf --gpu \
-p "Summarize the attention mechanism." -n 256| METRIC | MEASURED | NOTES |
|---|---|---|
| Decode (tg128, 2026-05-28) | 202.5 ± 1.3 tok/s | llama.cpp same-session: 157.2 ± 3.7 (+28.8%) |
| Decode (2026-07 re-bench) | 136.9 tok/s | regression under active bisection; llama.cpp has also improved to 242 |
RTX 3090 Ti, single GPU, greedy tg128, same-session A/Bs, 2026-05-28 and 2026-07-20.
Both engines' numbers moved between 2026-05 and 2026-07; we publish both measurements rather than the flattering one. Bisection is in progress.