Carry image content through the TP forward path so every rank encodes
and splices locally (replicated tower, no embedding broadcast).
- rpc.rs: new WorkerRequest::GenerateStepWithImages carrying the source
image data URIs + image_token_id for the single-shot vision prefill;
worker still replies GenerateStepOk. Round-trip test added.
- tp_qwen3_5.rs: TpQwen3_5ForCausalLM::forward_with_images — encode each
preprocessed image through the rank's replicated tower, cat, splice,
forward. Shared by leader and worker so every rank runs identical work.
- tp/mod.rs: TpLeaderModel::forward_with_images and
WorkerPool::generate_step_with_images (mirrors generate_step: fan out
GenerateStepWithImages to subprocess ranks, run the leader's image
forward on its device worker thread, drain, combine).
- worker.rs: WorkerModel::forward_with_images + handle_generate_step_with_images
— each subprocess rank preprocesses the same data URIs via the shared
deterministic preprocess_data_uri, encodes, splices, forwards.
- device_worker: Job::TpForwardLogitsWithImages + tp_forward_logits_with_images
dispatch handler + DeviceWorkerHandle::tp_forward_logits_with_images.
Determinism: every rank runs the same preprocess on the same source
URIs through the same replicated tower, so the spliced hidden state
matches across ranks — preserving the replicated-hidden-state invariant
the row-parallel AllReduce relies on, with no NCCL broadcast.
No caller yet — Stage 3 wires the TP chat/stream entry points to invoke
generate_step_with_images for image prefill. cuda-gated plumbing covered
by CI's CUDA type-check; rpc/route/forward_with_images compile on the
non-cuda build.
Refs TP-vision plan Stage 2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rob thijssen
4994b94c84
