grenade
0 Followers · 0 Following
  • Joined on 2026-02-17
Repositories
8
 Projects Packages Public Activity Starred Repositories
grenade pushed to main at helexa/helexa 2026-09-02 05:34:55 +00:00
671a8e8659 Merge branch 'feat/qwen4-exp-qsa-oracle'
a34c86ebc7 test(qwen4_exp): gate 5 — the batched selection against a naive one
Compare 2 commits »
grenade pushed to feat/qwen4-exp-qsa-oracle at helexa/helexa 2026-09-02 04:54:02 +00:00
a34c86ebc7 test(qwen4_exp): gate 5 — the batched selection against a naive one
grenade pushed to feat/qwen4-exp-qsa-oracle at helexa/helexa 2026-09-02 04:53:00 +00:00
57a9c1fd69 test(qwen4_exp): gate 5 — the batched selection against a naive one
grenade created branch feat/qwen4-exp-qsa-oracle in helexa/helexa 2026-09-02 04:52:59 +00:00
grenade pushed to main at helexa/helexa 2026-09-02 04:50:23 +00:00
29c0ce06ba Merge branch 'feat/qwen4-exp-model'
fb79fae5e0 feat(qwen4_exp): the model, and a checkpoint small enough to test it on
cc114cc781 feat(qwen4_exp): the n-gram gather, behind a seam #310 can move
Compare 3 commits »
grenade pushed to main at helexa/helexa 2026-09-01 17:29:33 +00:00
2fb863f8c8 Merge branch 'feat/qwen4-exp-decoder'
d6e4ac3c5f feat(qwen4_exp): the decoder layer, where four streams replace the norms
f876af5e04 Merge branch 'main' into feat/qwen4-exp-decoder
Compare 3 commits »
grenade commented on issue helexa/helexa#307 2026-09-01 16:44:47 +00:00
Epic: serve Qwen3.8-Flash-Next (qwen4_exp) — a 180B sparse MoE whose architecture is built for beast's shape

The text path is implemented end to end

Qwen4ExpForCausalLM loads a checkpoint and returns logits. On main or in review across seven branches: the config, hyper-connections, PLE (addressing…

grenade pushed to feat/qwen4-exp-model at helexa/helexa 2026-09-01 16:44:23 +00:00
fb79fae5e0 feat(qwen4_exp): the model, and a checkpoint small enough to test it on
cc114cc781 feat(qwen4_exp): the n-gram gather, behind a seam #310 can move
Compare 2 commits »
grenade created branch feat/qwen4-exp-model in helexa/helexa 2026-09-01 16:44:22 +00:00
grenade pushed to main at helexa/helexa 2026-09-01 16:13:46 +00:00
61e2c9b50d Merge branch 'feat/qwen4-exp-linear-attn'
a890f61b42 feat(qwen4_exp): 36 layers reused whole, and the two settings that differ
Compare 2 commits »
grenade created branch feat/qwen4-exp-decoder in helexa/helexa 2026-09-01 16:12:50 +00:00
grenade pushed to feat/qwen4-exp-decoder at helexa/helexa 2026-09-01 16:12:50 +00:00
d6e4ac3c5f feat(qwen4_exp): the decoder layer, where four streams replace the norms
f876af5e04 Merge branch 'main' into feat/qwen4-exp-decoder
Compare 2 commits »
grenade pushed to main at helexa/helexa 2026-09-01 16:08:54 +00:00
e5df6ebe48 Merge branch 'feat/qwen4-exp-moe'
049eaf7a6d feat(qwen4_exp): the MoE, loaded from two tensors instead of 512 modules
Compare 2 commits »
grenade pushed to feat/qwen4-exp-linear-attn at helexa/helexa 2026-09-01 15:43:22 +00:00
a890f61b42 feat(qwen4_exp): 36 layers reused whole, and the two settings that differ
grenade created branch feat/qwen4-exp-linear-attn in helexa/helexa 2026-09-01 15:43:21 +00:00
grenade pushed to main at helexa/helexa 2026-09-01 15:38:31 +00:00
ca68d0c9b0 Merge branch 'feat/qwen4-exp-full-attn'
f0a3563879 feat(qwen4_exp): the full-attention layer, and a mask the kernel cannot lie about
Compare 2 commits »
grenade pushed to feat/qwen4-exp-moe at helexa/helexa 2026-09-01 15:35:17 +00:00
049eaf7a6d feat(qwen4_exp): the MoE, loaded from two tensors instead of 512 modules
grenade created branch feat/qwen4-exp-moe in helexa/helexa 2026-09-01 15:35:16 +00:00
grenade pushed to main at helexa/helexa 2026-09-01 15:31:58 +00:00
9d0e7717f4 Merge branch 'feat/qwen4-exp-qsa-indexer'
74c8647ec5 feat(qwen4_exp): the QSA mask, and where it must never be sent
5745708e89 feat(qwen4_exp): the QSA indexer, and the second KV cache nobody counted
Compare 3 commits »
grenade pushed to feat/qwen4-exp-full-attn at helexa/helexa 2026-09-01 15:17:01 +00:00
f0a3563879 feat(qwen4_exp): the full-attention layer, and a mask the kernel cannot lie about