Block a user
qwen4_exp F: use the shipped MTP head as a draft model
Retracting the comment above: llama.cpp does do this, and has numbers
My previous comment concluded llama.cpp is not a reference for this head. That was read off the spike checkout on beast,…
qwen4_exp F: use the shipped MTP head as a draft model
Correction: llama.cpp is not a reference for this head
#309's gate comment says "llama.cpp merged it (#27742/#27793) and
already ships --spec-type draft-mtp, so **#313 has a working
reference…
A total-duration request_timeout makes large-blob fetches fail deterministically in proxy mode
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape
New child: #319 — tensor parallelism
Filed to close the last untracked gap. qwen4_exp is absent from
TP_SUPPORTED_MODEL_TYPES, so the model currently sees one 32.6 GB
card where every…
qwen4_exp J: tensor parallelism — without it the model sees one card, and every #309 number assumes two
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape
New child: #318 — host-resident experts, and measured placement
Filed because the re-scope of #310 retired this work on the strength of
the ncmoe=4 measurement, which the operator decision…
qwen4_exp I: host-resident experts — and a placement decision measured per machine, not chosen once
qwen4_exp C: host- or disk-resident PLE table with a device-side gather
Prior art: FreeToken has solved this twice, and neither answer is one this issue contemplates
FlashML-org/FreeToken is an edge-native MoE serving engine aimed at
exactly our problem —…