RUN IT RIGHT

Tested deployment recipes for every model IronWorks supports: hardware requirements, exact commands, tuning flags, and measured performance.

Every number is a real measurement with hardware and date — same weights, same session, same GPU as the engine we compare against. No projections.

Llama 3.2 1B Instruct

1B · Dense · 2 GB+

513 tok/s single-stream decode on one consumer GPU — ahead of llama.cpp with tighter variance.

3B · Dense · 4 GB+

262 tok/s marginal decode via a parallel-KV flash-decode kernel — the workhorse small model.

4.6B (effective 2B) · Dense · 4 GB+

The small-model showcase: measured 202 tok/s, +29% over llama.cpp at its peak.

7B · Dense · 6 GB+

143 tok/s decode at parity with llama.cpp — the stable mid-size baseline.

31B · Dense · 24 GB (or any budget via unified memory)

Dense 31B on one 24 GB card — with lossless multi-token-prediction self-speculation (+18%).

26B (3.8B active) · MoE · 8 GB+ with expert offload

The 8-GB-card class leader: 26B-class quality with ~4B active parameters per token.

30B (3B active) · MoE · 18 GB full-resident

185 tok/s MoE decode at parity with llama.cpp — 30B-class quality at 3B-class speed.

30B (3B active) · Hybrid SSM · 18–24 GB (any budget via unified memory)

Hybrid Mamba-2 + MoE on GPU — including a larger-than-VRAM mode that stays deterministic.

8B · Hybrid SSM · 8 GB+

85 tok/s native GPU Mamba-2 decode — hybrid SSM without CPU fallbacks.