Block a user
grenade
created branch feat/allow-on-demand-mirroring in grenade/rustingface
2026-08-31 12:08:40 +00:00
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape
Concurrency × depth at the decided operating point
The one interaction left unmeasured, and the one that governs what beast can actually promise. Balanced 24-layer offload (38.7 GB weights), `l…
Move 27B residency to benjy, so beast can be dedicated to flash-next
Operator decision: benjy becomes 27B-dedicated
If benjy can run the 27B at all, it is not expected to run anything else alongside. The 8B is either dropped or moved to quadbrat.
That…
Move 27B residency to benjy, so beast can be dedicated to flash-next
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape
Concurrent serving envelope — and a correction about why it collapses
Follow-on to #309, which measured single-stream only. Same host, same
build, same IQ4_XS model. llama-server, 8 slots ×…
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape
Operator decision: residency and the target to build against
Recorded so it survives context loss.
Acceptance target for the port is long context, not peak decode. ~30 tok/s is fine; more…
qwen4_exp C: host- or disk-resident PLE table with a device-side gather
Re-scope: this issue's premise did not survive #309
Do not start this as written. The issue is built on "~121 B of routed experts and ~53 B of PLE tables cannot live in 63.7 GB of…