Pyromind open-sources PyroDash, a cost-aware token-level small-large model collaborative inference framework

Pyromind (火思动力) has open-sourced PyroDash, a cost-aware token-level small-large collaborative inference framework. A trainable 4B model streams tokens first; when stuck it emits a τ_off token, and a frozen large model continues the same reasoning trace—at most one large-model call per request, with no large-model logits needed. Across five math benchmarks, cost mode averaged one large call per 100 problems, lowering cost from $49.36 to $1.78; accuracy mode hit 64.04% against 57.68% for the large model alone, at ~20% lower cost. The team says it outperformed RouteLLM and GlimpRouter. Licence not stated.

Original sources (Chinese)