helexa/helexa
1
0
Fork 0
Code Issues 143 Pull Requests Actions Packages Projects Releases Wiki Activity

Vision: deploy on Qwen3.6-27B (production validation) #13

New Issue
Closed
opened 2026-06-01 13:18:18 +00:00 by grenade · 1 comment
grenade commented 2026-06-01 13:18:18 +00:00
Owner
Copy Link
Copy Source

Context

Deferred during planning of the initial vision capability (umbrella:
#3). Stage A–C of the vision plan develop against a smaller Qwen-VL
iteration target to keep cycle time tractable; this issue tracks
the deploy-on-target step. Refs:
~/.claude/plans/foamy-twirling-catmull.md.

Problem

Iterating directly against Qwen3.6-27B costs full model load time
(minutes) per test cycle and burns a TP setup for each attempt.
Stage A nominates a smaller variant (Stage A0 investigation
identifies which — most likely a Qwen3-VL family member if released,
else Qwen2-VL-2B-Instruct as architecture-adjacent fallback). Once
Stages A–C work end-to-end against the iteration model, this issue
covers what's needed to put the same Rust code in front of the
real 27B in production.

Scope

  • Load actual Qwen3.6-27B vision weights from
    /archive3/llm-cache/models--Qwen--Qwen3.6-27B/... (or wherever
    the production cache lives on beast/benjy/quadbrat).
  • Reconcile any architecture mismatches between the iteration model
    and the 27B: vision-tower depth, hidden size, patch size,
    projector layout. If the iteration model's vision.rs doesn't
    generalise cleanly, factor it.
  • Validate end-to-end on beast (single-GPU ISQ first, then TP via
    the TP-vision issue) and benjy and quadbrat.
  • Update models.example.toml to mark Qwen3.6-27B as vision-capable
    and document the deployment.

Acceptance

  • The issue #3 curl repro against beast (or hanzalova → beast)
    with Qwen/Qwen3.6-27B returns coherent image-grounded text
    with image-token-bearing prompt_tokens.
  • Quality benchmark from Stage D passes its threshold on the 27B.
  • Operators can flip capabilities in their /v1/models and see
    Qwen3.6-27B advertised as vision-capable without any code change.

Blocked by

Stage A–C of the vision plan. The TP-vision issue is the natural
co-traveller for full-quality 27B serving on the existing fleet.

References

## Context Deferred during planning of the initial vision capability (umbrella: #3). Stage A–C of the vision plan develop against a smaller Qwen-VL iteration target to keep cycle time tractable; this issue tracks the deploy-on-target step. Refs: `~/.claude/plans/foamy-twirling-catmull.md`. ## Problem Iterating directly against Qwen3.6-27B costs full model load time (minutes) per test cycle and burns a TP setup for each attempt. Stage A nominates a smaller variant (Stage A0 investigation identifies which — most likely a Qwen3-VL family member if released, else Qwen2-VL-2B-Instruct as architecture-adjacent fallback). Once Stages A–C work end-to-end against the iteration model, this issue covers what's needed to put the same Rust code in front of the real 27B in production. ## Scope - Load actual Qwen3.6-27B vision weights from `/archive3/llm-cache/models--Qwen--Qwen3.6-27B/...` (or wherever the production cache lives on beast/benjy/quadbrat). - Reconcile any architecture mismatches between the iteration model and the 27B: vision-tower depth, hidden size, patch size, projector layout. If the iteration model's `vision.rs` doesn't generalise cleanly, factor it. - Validate end-to-end on beast (single-GPU ISQ first, then TP via the TP-vision issue) and benjy and quadbrat. - Update `models.example.toml` to mark Qwen3.6-27B as vision-capable and document the deployment. ## Acceptance - The issue #3 curl repro against beast (or hanzalova → beast) with `Qwen/Qwen3.6-27B` returns coherent image-grounded text with image-token-bearing `prompt_tokens`. - Quality benchmark from Stage D passes its threshold on the 27B. - Operators can flip `capabilities` in their `/v1/models` and see Qwen3.6-27B advertised as vision-capable without any code change. ## Blocked by Stage A–C of the vision plan. The TP-vision issue is the natural co-traveller for full-quality 27B serving on the existing fleet. ## References - Plan: `~/.claude/plans/foamy-twirling-catmull.md` - Umbrella: #3 - Stage A0 investigation note about iteration target selection
grenade commented 2026-06-12 08:57:23 +00:00
Author
Owner
Copy Link
Copy Source

Closing as completed. The code is in front of the real 27B in production: vision requests against Qwen/Qwen3.6-27B on beast (TP=2) were verified end-to-end through the gateway on 2026-06-04 (see #3's verification comment — correct frame descriptions, image tokens counted). Production hardening that followed (chunked prefill fa01350, VRAM pre-flight guard, #17 fault recovery) was driven by real 27B vision traffic, which is this issue's definition of done.

Closing as completed. The code is in front of the real 27B in production: vision requests against `Qwen/Qwen3.6-27B` on beast (TP=2) were verified end-to-end through the gateway on 2026-06-04 (see #3's verification comment — correct frame descriptions, image tokens counted). Production hardening that followed (chunked prefill `fa01350`, VRAM pre-flight guard, #17 fault recovery) was driven by real 27B vision traffic, which is this issue's definition of done.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
out-of-scope
Outside helexa's constraining principles: near-frontier models on consumer hardware, served transparently and predictably
 p1-now
Active pain or unblocking work — pick up first
 p2-next
Engine levers — start once P1 baselines exist
 p3-later
Dependent on earlier work, or demand-driven
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
grenade
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: helexa/helexa#13