rob thijssen
8d3194f992
Some checks failed
build-prerelease / Resolve version stamps (push) Successful in 35s
CI / Format (push) Successful in 38s
CI / Clippy (push) Successful in 2m16s
build-prerelease / Build neuron-blackwell (push) Failing after 3m19s
CI / Test (push) Successful in 4m26s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
build-prerelease / Build cortex binary (push) Successful in 4m22s
build-prerelease / Package cortex RPM (push) Successful in 1m23s
build-prerelease / Build neuron-ampere (push) Failing after 4m58s
build-prerelease / Build neuron-ada (push) Failing after 4m53s
build-prerelease / Package helexa-neuron-ada RPM (push) Has been skipped
build-prerelease / Package helexa-neuron-ampere RPM (push) Has been skipped
build-prerelease / Package helexa-neuron-blackwell RPM (push) Has been skipped
build-prerelease / Publish to rpm.lair.cafe (unstable) (push) Has been skipped
Ports the canonical
candle-examples/examples/llama_multiprocess/model.rs pattern into
the harness. Two new files, one deletion:
- harness/tp/all_reduce.rs — AllReduce wraps Arc<cudarc::nccl::Comm>
and implements candle's CustomOp1 trait. cuda_fwd extracts the
rank's CudaSlice<dtype> from a CudaStorage, asserts the input is
contiguous (a strided activation hitting all_reduce is almost
always a model construction bug), allocates an output CudaSlice
on the same device, calls Comm::all_reduce(Sum), and wraps the
result back as a CudaStorage. Handles BF16, F16, F32. NcclError
surfaces via {e:?} (no Display impl in cudarc 0.19.x). Send/Sync
hand-impl'd with the same NCCL-thread-safety caveat candle's
example documents.
- harness/tp/tp_linear.rs — ColumnParallelLinear and
RowParallelLinear, both built on candle's ShardedVarBuilder +
Shard hints. `vb.get_with_hints((), "weight", shard(dim, rank, ws))`
reads JUST the rank's slice from the safetensors view; no full-
tensor host materialisation. ColumnParallel.forward is a plain
local matmul (output is naturally sharded). RowParallel.forward =
local matmul + apply_op1_no_bwd(&self.all_reduce). On CPU /
world_size == 1, the AllReduce is skipped and the partial output
is returned as-is. Both layers are no-bias — every Qwen3-family
target sets attention_bias=false; bias-aware sharding is a
future-model concern.
- Deletes harness/tp/sharded_linear.rs from 7b-ii. That commit's
hand-rolled "load full + narrow" approach was useful exploration
but candle's ShardedVarBuilder does the same work without
materialising the full tensor on host. The 5 unit tests there
verified the slicing math against an unsharded reference; that
math now lives inside candle and is covered by candle's own tests.
Next (7b-iii 2/2): TpQwen3Attention + TpQwen3MLP composing the
column/row pair, then a TpQwen3Model that runs the full forward.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>