rob thijssen a70f317729 feat(stage-8c): scaffold qwen3_5 (Qwen3.6) — dispatch + stubs + TP gate ...

Lays the wiring for the top-priority TP-2 target without doing the
substantive architecture work yet. After this commit, attempting to
load a Qwen3.6 (`model_type = "qwen3_5"`) model:
- Passes config.json parse — the real upstream shape (text_config
  wrapper, layer_types, attn_output_gate, head_dim=256, etc.) round-
  trips through a typed Config (unit test included).
- Constructs a placeholder Qwen3_5ForCausalLM, attaches it to a
  ModelArch::Qwen3_5Dense variant, registers it in the loaded set.
- Fails on the first inference forward with a clear "Qwen3-Next
  forward not implemented yet (Stage 8c, TP-2 motivator)" — the
  point where the real architecture work begins.

New layout:
- `harness/arch/` for custom architectures candle-transformers doesn't
  ship. Each architecture is one module: Config + ForCausalLM + impl.
- `harness/arch/qwen3_5.rs` — the scaffold. Heavy doc comments on the
  open work: layer_types dispatch (full_attention vs linear_attention,
  the latter being the hard part with no candle precedent),
  attn_output_gate, text_config nesting, recurrent state lifecycle.
- DENSE_SUPPORTED_MODEL_TYPES adds "qwen3_5"; load_arch_dense gains a
  branch that constructs the stub.

TP-side gate:
- New `check_tp_arch_supported`: even though Llama / Qwen3 MoE pass
  the single-GPU dense check (DENSE_SUPPORTED_MODEL_TYPES), the
  worker pool's `load_dense_shard` reconstructs the config as Qwen3
  on every rank — silently misrouting a non-Qwen3 dense load through
  it would surface as a cryptic per-rank deserialise error.
- TP_SUPPORTED_MODEL_TYPES = ["qwen3"] (cuda-gated). Anything else
  bails *before* the worker pool spawns and NCCL handshake costs are
  paid, with a marker pointing at the `tp_<family>.rs` module a
  contributor would need to add. qwen3_5 specifically lands here
  until its architecture is real.

The naming choice: keep "qwen3_5" from the model's own config.json
rather than mistralrs's "qwen3_next" — the latter ages poorly the
moment Qwen ship another architecture revision.

Unit tests: 2 new for qwen3_5 (config deserialise + dispatch gate);
the previously-rejecting test for qwen3_5 swapped to a fictional
arch so it stays meaningful as the supported set grows. 26 lib tests
pass; cargo clippy CPU + --features cuda both clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-20 08:58:01 +03:00

..

src

feat(stage-8c): scaffold qwen3_5 (Qwen3.6) — dispatch + stubs + TP gate

2026-05-20 08:58:01 +03:00

tests

Stage 7a-ii: real NCCL handshake behind the worker pool

2026-05-19 16:40:01 +03:00

Cargo.toml

fix(tp): add half dep + drop double-wrapped .w() on CudaDevice::alloc

2026-05-19 19:11:59 +03:00