fix(neuron): route the fused MoE block in the activation dtype (#92) #111

Merged

grenade merged 1 commits from fix/92-fused-router-dtype into main 2026-07-02 04:34:41 +00:00

Conversation 0 Commits 1 Files Changed 1 +7 -2

grenade commented 2026-07-02 04:34:31 +00:00

Owner

Copy Link

Part of #92. The fused path fed f32 activations into the replicated bf16 router — a dtype-mismatched matmul that failed the first live fused request on the 80B and (correctly) tripped the poison/auto-recovery machinery with a clean fast-reject. Route in the original activation dtype like route_scatter does; only the softmax and downstream routing weights are f32 (which the grouped-GEMM kernel requires for topk_weights anyway).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

Part of #92. The fused path fed f32 activations into the replicated bf16 router — a dtype-mismatched matmul that failed the first live fused request on the 80B and (correctly) tripped the poison/auto-recovery machinery with a clean fast-reject. Route in the original activation dtype like `route_scatter` does; only the softmax and downstream routing weights are f32 (which the grouped-GEMM kernel requires for `topk_weights` anyway). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

grenade added 1 commit 2026-07-02 04:34:32 +00:00

fix(neuron): route the fused MoE block in the activation dtype (#92) ... 40fecb2cf7

The fused path fed f32 activations into the replicated bf16 router —
a dtype-mismatched matmul that failed the first live fused request on
the 80B and (correctly) tripped the poison/auto-recovery machinery.
Route in the original activation dtype like route_scatter does; only
the softmax and downstream weights are f32 (which the grouped-GEMM
kernel requires for topk_weights anyway).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TczcGF7JSjJs8r15RSSGpx

grenade merged commit 35041f11de into main 2026-07-02 04:34:41 +00:00

grenade deleted branch fix/92-fused-router-dtype 2026-07-02 04:34:41 +00:00

grenade referenced this issue from a commit 2026-07-02 04:34:42 +00:00

Merge pull request 'fix(neuron): route the fused MoE block in the activation dtype (#92)' (#111) from fix/92-fused-router-dtype into main

grenade referenced this pull request 2026-07-02 05:06:30 +00:00

F1 — neuron: MoE FFN into the qwen3_5 hybrid layer stack + TP expert sharding #92

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

out-of-scope

Outside helexa's constraining principles: near-frontier models on consumer hardware, served transparently and predictably

p1-now

Active pain or unblocking work — pick up first

p2-next

Engine levers — start once P1 baselines exist

p3-later

Dependent on earlier work, or demand-driven

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

grenade

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: helexa/helexa#111