feat(neuron): q8_0 ISQ for the Z-Image DiT + allocator-pool hygiene (#204) #210

Merged

grenade merged 3 commits from feat/204-z-image-quant into main 2026-07-29 17:10:39 +00:00

Conversation 0 Commits 3 Files Changed 9 +128 -49

grenade commented 2026-07-29 17:10:31 +00:00

Owner

Copy Link

Z7 (scope: quantized DiT per the operator decision). quant = "q8_0" on an image load quantizes the DiT in situ from the same dense safetensors (fork-side QuantizedZImageTransformer2DModel: QLinear/QMatMul port with verbatim forward logic; f32 activations cast to bf16 around flash-attn). Also trims the cudarc pool after every image job — mixed-resolution sequences otherwise fragment it into OOM — and adds quadbrat's image config (768² ceiling).

Live-validated:

• beast (5090, flash): q8 resident 9.4 GB (vs 14.6 BF16), 1024² denoise 2.8 s, 512→2048→1024 sequence passes, fixed-seed output visually identical to BF16.
• quadbrat (3060, 12 GB): q8 resident 8.7 GB, 768² passes (5.31 units); 1024² OOMs on naive f32 attention → ceiling 768 until #95 extends flash-attn to ampere.

🤖 Generated with Claude Code

https://claude.ai/code/session_0165r11RzqkMqWWXfJE8tAVU

Z7 (scope: quantized DiT per the operator decision). `quant = "q8_0"` on an image load quantizes the DiT in situ from the same dense safetensors (fork-side `QuantizedZImageTransformer2DModel`: QLinear/QMatMul port with verbatim forward logic; f32 activations cast to bf16 around flash-attn). Also trims the cudarc pool after every image job — mixed-resolution sequences otherwise fragment it into OOM — and adds quadbrat's image config (768² ceiling). Live-validated: - beast (5090, flash): q8 resident **9.4 GB** (vs 14.6 BF16), 1024² denoise 2.8 s, 512→2048→1024 sequence passes, fixed-seed output visually identical to BF16. - quadbrat (3060, 12 GB): q8 resident **8.7 GB**, 768² passes (5.31 units); 1024² OOMs on naive f32 attention → ceiling 768 until #95 extends flash-attn to ampere. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_0165r11RzqkMqWWXfJE8tAVU

grenade added 3 commits 2026-07-29 17:10:32 +00:00

feat(neuron): q8_0 ISQ for the Z-Image DiT (#204) ... 5171f72310

quant = "q8_0" on an image-model load now quantizes the DiT in situ
from the same dense safetensors (fork-side
QuantizedZImageTransformer2DModel: QLinear/QMatMul port of the dense
model with identical forward logic; norms/biases/embedders stay
float). Resident DiT drops ~12 GB -> ~6.4 GB, opening the 12 GB
(3060) tier; the pipeline runs f32 activations when quantized.
Preflight passes quant through for diffusers repos; the loader
rejects dtypes other than q8_0 with the accepted list. candle pin
bumped to grenade/candle@cd51258.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165r11RzqkMqWWXfJE8tAVU

fix(neuron): trim the allocator pool after image loads and generations (#204 #199) ... 1c57fc5e00

Beast E2E: generations at mixed resolutions (2048² then 1024²)
fragment the cudarc stream-ordered pool until a generation that would
fit a clean slate OOMs. Trim after every image job — milliseconds
against a multi-second generation. Also bumps the candle pin to
f8fa85fd: the quantized attention now casts its f32 activations to
bf16 around flash-attn (which is f16/bf16-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165r11RzqkMqWWXfJE8tAVU

feat(fleet): quadbrat image config — q8_0 tier, 768 ceiling (#204) ... 6985457871

12 GB validation: q8_0 DiT resident 8.7 GB, 768 generations pass
(5.31 units), 1024 OOMs on naive f32 attention transients. Ceiling
documented at 768 until #95 extends flash-attn to ampere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165r11RzqkMqWWXfJE8tAVU

grenade merged commit fd8b0c7e33 into main 2026-07-29 17:10:39 +00:00

grenade referenced this issue from a commit 2026-07-29 17:10:40 +00:00

Merge pull request 'feat(neuron): q8_0 ISQ for the Z-Image DiT + allocator-pool hygiene (#204)' (#210) from feat/204-z-image-quant into main

grenade referenced this pull request 2026-07-29 17:10:45 +00:00

Z7 — quantized DiT + model variants (base / Edit / fine-tunes) — contingency/demand-driven #204

grenade referenced this pull request 2026-07-29 22:35:37 +00:00

[epic] text-to-image serving: Z-Image-Turbo on neuron + /v1/images surface #197

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

out-of-scope

Outside helexa's constraining principles: near-frontier models on consumer hardware, served transparently and predictably

p1-now

Active pain or unblocking work — pick up first

p2-next

Engine levers — start once P1 baselines exist

p3-later

Dependent on earlier work, or demand-driven

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

grenade

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: helexa/helexa#210