helexa/helexa
1
0
Fork 0
Code Issues 138 Pull Requests Actions Packages Projects Releases Wiki Activity

HeartMuLa H5: memory discipline — LM/codec placement and the 19 GiB peak #393

New Issue
Closed
opened 2026-10-15 11:58:22 +00:00 by grenade · 0 comments
grenade commented 2026-10-15 11:58:22 +00:00
Owner
Copy Link
Copy Source

Part of #379. Needs #391 and #392, and reuses #385's per-component placement.

In the beta, HeartMuLa peaked at about 19 GiB on the 4090 for 60 s of audio. The LM and codec co-resident in their working dtypes OOMed, until the LM was moved to the CPU during codec decode. That makes it the hardest music model to place. This child brings the footprint down to something the fleet can co-host.

Scope

  • Stage-exclusive residency: the LM and HeartCodec never need the device at the same time. Options, in order of preference:
    1. LM on device 0 and codec on device 1. heartlib supports this officially with --mula_device and --codec_device, and only token frames cross. This is the multi-card shape #380 wants.
    2. Single device, LM q8 plus codec fp32, both resident if the budget allows.
    3. Single device, park the LM to host RAM during decode, as the beta does. Swaps on the 4090 cost 0.4–6.5 s.
  • q8 LM (from #391) plus fp32 codec. Measure peak VRAM per duration (30/60/120/240 s) for each option.
  • Cold-swap integration: the ranker's vram_mb must reflect the peak, not the resident size, or the evictor will under-evict.

Acceptance

  • A peak-VRAM and latency table per option and duration.
  • One single-device configuration that fits under 24 GiB with margin for 240 s, and a two-device configuration documented for #380.
Part of #379. Needs #391 and #392, and reuses #385's per-component placement. In the beta, HeartMuLa peaked at about **19 GiB** on the 4090 for 60 s of audio. The LM and codec co-resident in their working dtypes OOMed, until the LM was moved to the CPU during codec decode. That makes it the hardest music model to place. This child brings the footprint down to something the fleet can co-host. ## Scope - **Stage-exclusive residency**: the LM and HeartCodec never need the device at the same time. Options, in order of preference: 1. LM on device 0 and codec on device 1. heartlib supports this officially with `--mula_device` and `--codec_device`, and only token frames cross. This is the multi-card shape #380 wants. 2. Single device, LM q8 plus codec fp32, both resident if the budget allows. 3. Single device, park the LM to host RAM during decode, as the beta does. Swaps on the 4090 cost 0.4–6.5 s. - q8 LM (from #391) plus fp32 codec. Measure peak VRAM per duration (30/60/120/240 s) for each option. - Cold-swap integration: the ranker's `vram_mb` must reflect the *peak*, not the resident size, or the evictor will under-evict. ## Acceptance - A peak-VRAM and latency table per option and duration. - One single-device configuration that fits under 24 GiB with margin for 240 s, and a two-device configuration documented for #380.
grenade added this to the Music generation milestone 2026-10-15 11:58:22 +00:00
grenade added the p3-later label 2026-10-15 12:01:47 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
out-of-scope
Outside helexa's constraining principles: near-frontier models on consumer hardware, served transparently and predictably
 p1-now
Active pain or unblocking work — pick up first
 p2-next
Engine levers — start once P1 baselines exist
 p3-later
Dependent on earlier work, or demand-driven
No labels p3-later
Milestone
No items
No Milestone Music generation
Projects
Clear projects
No projects
Assignees
Clear assignees
grenade
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: helexa/helexa#393