grenade
0 Followers · 0 Following
  • Joined on 2026-02-17
Repositories
6
 Projects Packages Public Activity Starred Repositories
grenade created branch chore/cudarc-bump-63327a2 in helexa/helexa 2026-06-08 13:33:18 +00:00
grenade pushed to chore/cudarc-bump-63327a2 at helexa/helexa 2026-06-08 13:33:18 +00:00
60f5598542 build(neuron): bump cudarc fork to 63327a2 (idempotent abort + Comm Send+Sync)
grenade pushed to main at helexa/helexa 2026-06-08 12:06:05 +00:00
7945240646 chore: re-trigger deploy (#17 Stage 2, attempt 3)
grenade pushed to main at helexa/helexa 2026-06-08 11:45:18 +00:00
0c74d89d15 chore: re-trigger deploy (#17 Stage 2)
grenade pushed to main at helexa/helexa 2026-06-08 11:24:02 +00:00
c94a2ae755 fix(neuron): correct nccl_state path on WorkerPool.leader_comm (#17 S2)
99920dd322 feat(neuron): TP step watchdog aborts wedged collectives (#17 Stage 2)
c4f239ceb9 build(neuron): patch cudarc to expose Comm::abort/get_async_error (#17 Stage 2)
Compare 3 commits »
grenade pushed to feat/neuron-17-stage2 at helexa/helexa 2026-06-08 11:21:47 +00:00
c94a2ae755 fix(neuron): correct nccl_state path on WorkerPool.leader_comm (#17 S2)
grenade commented on issue helexa/helexa#17 2026-06-08 11:18:21 +00:00
TP-rank failure should abort the leader's NCCL collective instead of hanging the daemon

Implemented (branch feat/neuron-17-stage2) — verify from real-world journals

Now addressed by the #17 fault-recovery work, in two stages.

Stage 1 (merged, verified) — error-faults: a…

grenade commented on issue helexa/helexa#17 2026-06-08 11:17:55 +00:00
TP-rank failure should abort the leader's NCCL collective instead of hanging the daemon

test

grenade pushed to feat/neuron-17-stage2 at helexa/helexa 2026-06-08 11:15:32 +00:00
99920dd322 feat(neuron): TP step watchdog aborts wedged collectives (#17 Stage 2)
grenade created branch feat/neuron-17-stage2 in helexa/helexa 2026-06-08 10:50:58 +00:00
grenade pushed to feat/neuron-17-stage2 at helexa/helexa 2026-06-08 10:50:58 +00:00
c4f239ceb9 build(neuron): patch cudarc to expose Comm::abort/get_async_error (#17 Stage 2)
grenade opened issue helexa/helexa#20 2026-06-08 10:31:19 +00:00
neuron: keep auto-recovering models listed as recovering in /models (so cortex holds the route)
grenade opened issue helexa/helexa#19 2026-06-08 09:44:49 +00:00
neuron: startup preflight for NVIDIA driver/library mismatch (fail loud, not cryptic NCCL)
grenade pushed to main at helexa/helexa 2026-06-08 06:34:25 +00:00
ac445c1569 chore: re-trigger deploy (#17 Stage 1)
grenade pushed to main at helexa/helexa 2026-06-08 06:13:17 +00:00
abc6e605b8 test(neuron): NEURON_DEBUG_POISON hook to verify auto-recovery (#17)
4f2957af9e feat(neuron): auto-recover poisoned models (#17 Stage 1c)
Compare 2 commits »
grenade created branch feat/neuron-17-fault-recovery in helexa/helexa 2026-06-08 06:08:44 +00:00
grenade pushed to feat/neuron-17-fault-recovery at helexa/helexa 2026-06-08 06:08:44 +00:00
abc6e605b8 test(neuron): NEURON_DEBUG_POISON hook to verify auto-recovery (#17)
4f2957af9e feat(neuron): auto-recover poisoned models (#17 Stage 1c)
Compare 2 commits »
grenade pushed to main at helexa/helexa 2026-06-04 20:30:52 +00:00
75cd088b61 fix(neuron): cap vision max_pixels to the pos_embed patch budget (#14)
grenade pushed to main at helexa/helexa 2026-06-04 19:54:58 +00:00
d311c8ca7a feat(neuron): operator pixel-budget env override + doc cleanup (#14 C5)
c97a8654f5 feat(neuron): dynamic-resolution images via Qwen smart_resize (#14)
Compare 2 commits »
grenade pushed to feat/neuron-14-dynamic-resolution at helexa/helexa 2026-06-04 19:50:07 +00:00
d311c8ca7a feat(neuron): operator pixel-budget env override + doc cleanup (#14 C5)