Commit Graph
4 Commits
Author SHA1 Message Date
Benjamin Carlsson 84cc4b3fb5 Keep voice RTP timestamps aligned to 20 ms packets (#48824)
## Why

Capture jitter and mute transitions can shift RTP timestamps off the 20 ms source grid, causing receivers that assemble fixed-duration frames to reject subsequent audio.

## What changed

Advance elapsed gaps in whole 20 ms packets and keep the track clock on that grid. Bias empty gap durations by 1 ns to avoid losing an RTP tick to floating-point truncation.

## Testing

Update jitter and mute tests to assert exact clock advancement, including a 580 ms gap, and verify that resumed RTP timestamps stay aligned to 960-sample packets.

GitOrigin-RevId: 1511c739b77ee5871f72723fd34cb67aa9a48c95
2026-09-28 00:43:04 +00:00
Benjamin Carlsson 28f43b0417 Keep voice sessions alive through mute and audio backlog (#44671)
## Why

Audio queue saturation and processing delays could terminate voice sessions. Muted sessions also need outgoing audio to keep the peer alive.

## What changed

- Drop stale or excess incoming, capture, render-reference, and playback audio so fresh audio can resume. Reset affected processing state while preserving queue bounds and mute generations.
- Send paced, generated Opus silence while muted, without using device or processing buffers.
- Add fixed helper exit stages and session failure categories to default TUI logs without logging raw native errors, SDP, or device error text. Preserve observed helper exit status for shutdown.

## Testing

Extend tests to cover decoded mute silence and RTP sequence continuity, recovery after stale audio and queue saturation, and shutdown after observing helper output closure. Verify the helper's exit stage when the parent pipe closes during blocked startup.

GitOrigin-RevId: 17d580a72acd4423fb0b4f5107d9b541b964d069
2026-09-10 23:25:10 +00:00
Benjamin Carlsson a9896da3fe Connect voice-host RTP audio to speaker playback (#43248)
## Why

The voice host drained incoming RTP packets without playing them. Route received audio through decoding and speaker output while preserving speaker suppression boundaries.

## What changed

- Add a GStreamer pipeline with RTP jitter buffering, Opus decoding, conversion, and resampling into the native speaker sink.
- Rebuild playout when the speaker is re-enabled, reject packets received before the new pipeline started, and discard queued audio and echo references on speaker transitions while preserving capture history.
- Limit each receive pass to 64 packets so incoming media cannot indefinitely delay audio controls.
- Prevent retired sink failures from failing a new speaker generation, and retain the last coherent playback delay estimate during callback contention.

## Testing

Add tests for real Opus playback and stale packet rejection, combined microphone mute and speaker suppression, retired sink failures, delay contention, and echo-reference reset without losing capture history.

GitOrigin-RevId: c91c94a2ba2347ac98af4e7fd1dc331ae766cc45
2026-09-06 16:40:33 +00:00
Benjamin Carlsson 7dc7c7a756 Send processed microphone audio over RTP in voice-host (#43090)
## Why

`voice-host` previously drained captured audio locally without sending it to the peer. Connect capture to the outgoing media track while preserving mute boundaries and limiting stale audio.

## What changed

- Resample capture and render references to 48 kHz with Rubato, apply Sonora echo cancellation, noise suppression, and gain control, and encode capture as 20 ms mono Opus packets.
- Attach an Opus track to the WebRTC peer and advance RTP timestamps across mute gaps without introducing sequence-number gaps.
- Prioritize queued controls and shutdown between sends. Clear pending audio and processing history on microphone mute transitions, reject delayed pre-unmute buffers, and reset partial history after capture gaps.
- Size bounded capture and render queues for device rates, validate callback timing, enforce a 500 ms processing deadline, and time out stalled sends after 100 ms.

Receive/decode and TUI integration remain unimplemented. The bundled Opus encoder build requires CMake and a C compiler.

## Testing

Add synthetic-audio tests for Opus decoding at 44.1 and 48 kHz input rates, echo reduction, capture gaps, stale audio, queue capacity, and control priority. A loopback WebRTC test verifies RTP delivery, decoded signal, mute isolation, and timestamp and sequence continuity across mute gaps.

GitOrigin-RevId: 38b8438577cfadb8124f8aeb426dffbab4845964
2026-09-05 20:02:55 +00:00