Commit Graph

20 Commits

Author SHA1 Message Date
Benjamin Carlsson
9daf7d22ca Add bounded GStreamer playback components to the voice host (#43244)
## What changed

- Add a `GstAudioSink` subclass backed by a bounded mono `F32LE` playback writer, with partial writes, cancellation on speaker epoch changes, and delay accounting for queued samples and pending device output.
- Recreate the CPAL output stream and discard buffered audio when speaker controls change the epoch, preventing stale audio from surviving suppression.
- Add GStreamer dependencies, adjust Windows Bazel native-tool selection, and allow the helper handshake time to load startup-linked libraries.
- Separate installed-client tests from protocol lifecycle tests and supply native startup libraries in test packages.

## Testing

Add unit coverage for partial writes, sample accounting, suppression cancellation, stale writers, device delay, invalid samples, and stalled consumption.

GitOrigin-RevId: c48c48d2fee00659cf6f8a4039a6745385cb77cd
2026-09-06 16:02:27 +00:00
Benjamin Carlsson
1c40ffe427 Add Windows MSVC Bazel targets for native voice libraries (#43144)
## What changed

- Add explicit x64 and ARM64 targets for native builds, runtime preparation, and linking using the existing voice recipes. Each target requires native Windows execution of the matching architecture.
- Declare compiler, SDK, Python, and CMake inputs; require an explicitly supplied Cygwin/pkgconf tool tree and a fixed `SystemRoot`. Validate installed tool selections against the manifest and declared files.
- Pair Windows DLLs with SDK import libraries, preserve plugin and receipt runfiles, and omit Unix runtime-search flags. Use Python for portable payload copying.
- Correct the MSVC ARM64 tool directory casing to `HostArm64` and document provisioning and build commands.

The generic Rust-consumer aliases remain separate; native Windows Bazel execution and consumer validation are still needed to establish complete Windows voice support.

## Testing

Add eight unit tests covering tool selection, path anchoring, invalid inputs, and DLL/import-library copying, plus x64 and ARM64 link smoke targets that reference `gst_version`.

GitOrigin-RevId: fb2b4998e9ffb5b64ca830ebf4bb4633fc959eb9
2026-09-06 02:14:05 +00:00
Benjamin Carlsson
aa4a870e06 Add explicit Windows tool selection for native voice builds (#43125)
## Why

Windows libffi builds invoke tools by name, and Cygwin provides a different `link.exe` from MSVC. Inherited search paths can select unintended tools or SDK inputs.

## What changed

Add optional `--windows-build-inputs <json>` to `third_party/voice/build_native.py` to select installed tools and SDK directories for Windows MSVC targets. Validate the target, tool paths, architecture-specific assembler, SDK directories, and agreement with CLI tool arguments.

Build the search path from the selection with MSVC ahead of Cygwin, reject shadowed tools, replace inherited `INCLUDE` and `LIB`, and remove `LIBPATH`. Retain the selection in build receipts. Builds without the option continue using the normal Visual Studio environment.

## Testing

Add tests for linker precedence and subprocess selection, invalid or mismatched inputs, shadowed SDK tools, environment replacement, and receipt recording.

GitOrigin-RevId: c80a6d9eb29556157ba1316d62c9b5083f4038b9
2026-09-05 23:49:23 +00:00
Benjamin Carlsson
e01f38c388 Require a prepared runtime when assembling voice helper packages (#43121)
## Why

The helper's native bindings require shared libraries before entering `main`, including for lifecycle calls. Helper-only packages cannot satisfy that requirement.

## What changed

Make `--runtime` mandatory in `third_party/voice/assemble_package.py` and require the `runtime` argument in `assemble()`. Always validate and include the prepared runtime, and update the voice host documentation to describe the requirement.

## Testing

Update assembly tests to supply prepared runtimes and verify that package manifests include runtime library hashes and `runtime.json` alongside the helper.

GitOrigin-RevId: ab5abf80ae935b4ace833fa59b74f670ba62c632
2026-09-05 23:09:26 +00:00
Benjamin Carlsson
f5a71ff40a Link Unix Bazel bindings against the prepared voice runtime (#43117)
## What changed

- Export `native_sdk` from the same inspected build as `native_runtime`.
- Configure GLib and GStreamer Rust build scripts to use the declared `pkg-config` executable and SDK, preserving upstream version checks while directing linking to prepared libraries.
- Add `native_link` to expose canonical libraries, development aliases, transitive dependencies, plugins, and the runtime manifest through `CcInfo` and runfiles. Replace GStreamer SDK absolute rpaths with relative Bazel runpaths.

## Testing

Extend the macOS archive preparation test to cover absent and empty SDK output directories, verify SDK file hashes, and check that SDK and runtime manifests share the same source commit and target.

GitOrigin-RevId: 3fc33af06225c69e4c293437f1985290f228dc0a
2026-09-05 22:42:31 +00:00
Benjamin Carlsson
a947db131b Add Bazel preparation for native voice runtimes (#43114)
## What changed

Add the manual `//third_party/voice:native_runtime` target for macOS and GNU Linux. It consumes the native prefix archive, validates the completed build receipt against the target and source manifest, inspects libraries, and prepares verified copies using the existing platform policies.

Record the build commit through Bazel workspace status and include it in runtime receipts. Reject invalid or ambiguous commit metadata and escaping library aliases. Keep macOS preparation local and allow the system `libc++.1.dylib` dependency.

## Testing

Add tests for receipt validation, inspection ordering and failures, invalid commit metadata, and escaping archive links. Add a macOS integration test for preparing real libraries from a sandbox-linked archive with native aliases and either an absent or empty output directory.

GitOrigin-RevId: 23717ab33d1d3da5fb0042c1cc8e8efcd333c386
2026-09-05 22:27:22 +00:00
Benjamin Carlsson
a31c18ab7a Add a Bazel target for native voice dependencies (#43111)
## What changed

Add `//third_party/voice:native_prefix` to run the existing native build recipe with Bazel-declared archives and toolchain inputs on matching macOS and Linux GNU 2.28 targets for x64 and ARM64. The manual target exports `prefix.tar` and a `built.json` receipt through the `receipt` output group.

Keep macOS actions local to access Apple tools, use Apple's linker for libffi partial links, and preserve compiler runtime flags through libtool on Linux. Surface upstream build failure logs in the Bazel action output.

Document the build's host utility requirements and the distinction between the raw prefix and a relocatable SDK or runtime package.

GitOrigin-RevId: b4917943f621dec07bb994392619e8a42e496b8b
2026-09-05 22:07:21 +00:00
Benjamin Carlsson
e67a8ae6c9 Add explicit toolchain inputs for native voice builds (#43109)
## Why

Native voice builds ignore ambient compiler flags, but need a way to accept declared compiler, linker, and archive-tool inputs.

## What changed

- Add repeated `--c-flag`, `--cxx-flag`, and `--link-flag` options plus optional `--ar` and `--ranlib` paths for Unix builds. Record flags and pass them through CMake, Meson, and libffi, including Objective-C flags on macOS.
- Preserve literal libffi compiler arguments with response files. Reject flags containing whitespace, libffi paths requiring shell quoting, and Unix toolchain overrides on Windows.
- Add a pinned Bazel `pkg-config` tool built with LLVM archive tools and native macOS configure checks. Register the pinned Make toolchain and patch its bootstrap to pass preprocessor flags and enable cross-compilation configuration.

## Testing

Extend native-build tests to verify compiler and linker flags, explicit archive tools, Meson and Objective-C inputs, literal definitions through shell and recursive Make expansion, and rejection of unsupported overrides.

GitOrigin-RevId: e5ea541ec76abfa170f6b0411b0f14cd698c07e7
2026-09-05 21:47:17 +00:00
Benjamin Carlsson
47ca4619be Include GIO in voice SDKs and native runtimes (#43102)
## Why

The Rust GStreamer bindings link GLib's GIO library even when none of the selected plugins imports it. Following only plugin dependencies can omit a library the bindings need.

## What changed

- Export `gio-2.0` pkg-config metadata in the voice SDK.
- Require GIO as a runtime dependency root on Linux, macOS, and Windows, including its transitive dependencies under the existing import checks.
- Reject runtime packages that omit the required GIO library.

## Testing

Extend fixtures to cover GIO inclusion and missing-library rejection across platforms, loading GIO and its transitive dependency after moving a Linux runtime, and resolving `gio-2.0` through pkg-config after moving an SDK.

GitOrigin-RevId: d12e45d387d56c54d94d788cf07d82bb22c2b4a3
2026-09-05 21:18:44 +00:00
Benjamin Carlsson
5d35805715 Add receipt-verified native voice SDK export (#43099)
## What changed

Add `third_party/voice/sdk.py` to export headers, development libraries, and selected pkg-config metadata from an inspected native build. Verify shared libraries against inspection receipts and record the target, source commit, source manifest hash, and exported file hashes in `sdk.json`.

Enable relocatable Meson pkg-config metadata and materialize development library aliases. Reject mismatched receipts, paths escaping the build prefix, nonrelocatable metadata, and unsafe output locations; remove partial output on copy failure. Document SDK consumption as build inputs separate from packaged runtime files.

## Testing

Add tests for exported hashes and pkg-config resolution after moving the SDK and deleting the original prefix, alias materialization, invalid inputs, and cleanup after copy failure.

GitOrigin-RevId: 6d1575e659a02cd1f56e465e7830a296e8eaf486
2026-09-05 20:54:13 +00:00
Benjamin Carlsson
0683368584 Supply Bazel-managed CMake and Ninja for the bundled Opus build (#43083)
## Why

The bundled Opus encoder uses CMake and should not depend on tools available on a build worker's `PATH`.

## What changed

- Add `rules_foreign_cc` and provide its pinned CMake and Ninja tools and data to the `opusic-sys` build script.
- Add a CMake toolchain file that selects the declared Ninja executable, resolves the archiver path, and sets the target system and processor from `TARGET`, while retaining Bazel's compiler, SDK, and linker inputs.

GitOrigin-RevId: 48580c506d088d8c76cce958589ff94da458bd79
2026-09-05 19:31:24 +00:00
Benjamin Carlsson
dc0dc4f15d Package prepared runtimes with the voice host (#42332)
## What changed

- Add an optional `--runtime` input to `assemble_package.py`.
- Validate the runtime receipt, target, source manifest, plugin inventory, paths,
  and file hashes before copying only the declared runtime files.
- Preserve each platform's runtime layout and record the copied files and
  `runtime.json` in the package manifest.
- Recheck hashes after copying and remove the new output if assembly fails.

## Testing

Add package assembly tests for Linux, macOS, and Windows layouts, invalid
receipts and paths, modified inputs, symlinked directories, and copy-time
changes.

GitOrigin-RevId: 8c9609af8ced406926707b2b0e1dcd6d2251f94b
2026-09-02 19:13:32 +00:00
Benjamin Carlsson
eb10d91e48 Add Windows voice runtime preparation (#42209)
## What changed

- Add an x64 and ARM64 MSVC runtime preparer that uses `dumpbin` to validate
  PE32+ DLL metadata, selects the declared GStreamer plugins and dependency
  closure, and copies them unchanged into a private `bin/` directory.
- Reject malformed or unsupported PE metadata, path-bearing imports, delayed
  imports, managed DLLs, forwarded exports, undeclared dependencies, and
  case-insensitive DLL identity conflicts.
- Keep `third_party/voice/sources.json` line endings stable across Windows
  checkouts so native build receipts remain valid.

## Testing

Add native Windows tests for relocated DLL loading with a restricted search
path, receipt and digest failures, duplicate identities, malformed PE headers,
unsupported loader features, and cleanup after failed preparation.

GitOrigin-RevId: 5375f21fc54f5d597f3d4e5f8fbca3df307fa2a8
2026-09-02 05:23:24 +00:00
Benjamin Carlsson
8d01cd42fa Add GNU Linux voice runtime preparation (#42208)
## Why

Native voice libraries need package-relative loader paths so a prepared runtime
can be moved without retaining references to its build prefix.

## What changed

- Configure CMake and GNU Linux Meson builds with relative runtime paths, while
  also setting relocatable install names and paths for CMake libraries on macOS.
- Add a GNU Linux runtime preparer for x64 and ARM64 that validates bounded ELF64
  metadata, selects the declared GStreamer plugins and dependency closure, and
  preserves the `lib/gstreamer-1.0/` layout.
- Reject malformed ELF metadata, path-bearing imports, unsupported loader
  dependencies, and native outputs that still contain incompatible runtime
  paths.

## Testing

Add native tests for relative build paths, relocated library loading, dependency
and digest failures, malformed ELF inputs, and cleanup after failed preparation.

GitOrigin-RevId: ed94819b07d20214b56363446b64f765e08d75fa
2026-09-02 05:14:37 +00:00
Benjamin Carlsson
8813bd4b00 Add macOS voice runtime projection (#42204)
## What changed

- Add a shared runtime preparer that validates native build receipts, source and
  file digests, dependency closure, and output containment before copying the
  required voice plugins and libraries.
- Add macOS-specific Mach-O inspection and relocation that removes build-time
  runpaths, rewrites non-system dependencies relative to each loader, applies
  development ad-hoc signatures, and records the resulting payload in
  `runtime.json`.
- Close the state runtime explicitly in the clear-memories CLI test.

## Testing

- Add macOS tests covering relocation after the source prefix is removed and
  fail-closed handling of invalid receipts, changed inputs, unsafe paths,
  dependency conflicts, malformed Mach-O metadata, and failed transforms.

GitOrigin-RevId: 3be7a6fabf0135a57036cbb0a9a3e597096930fd
2026-09-02 04:52:32 +00:00
Charlie Marsh
84aa75204a Format Python scripts across the repository (#42109)
## Why

The root Python formatting pass only covered `scripts/`, leaving Python utilities elsewhere in the repository outside `just fmt` and `just fmt-check`.

## What changed

- Run the scripts Ruff formatter from the repository root.
- Exclude `sdk/python`, which has its own formatter pass, and `codex-rs/vendor`.
- Apply Ruff formatting to the newly covered Python files and update the formatter coverage test.

GitOrigin-RevId: 73ac82b8bb9c713d589724f6c4f926399febb513
2026-09-01 17:47:29 +00:00
Benjamin Carlsson
13bc770eaf Add installed voice host lifecycle support (#41902)
## What changed

- Add `VoiceHost` to resolve the packaged voice helper, launch it with an
  allowlisted environment, perform the protocol handshake, and enforce bounded
  shutdown and process cleanup.
- Preserve native executable path encoding in the pipe process APIs.
- Add `third_party/voice/assemble_package.py` to create a fresh package copy
  containing a target-compatible helper and a provenance manifest with file
  hashes.

## Testing

- Cover installed helper lifecycle, build matching, missing and symlinked
  helpers, non-UTF-8 package paths, environment filtering, package validation,
  target pairing, and failure cleanup.

GitOrigin-RevId: f893074b36ae6bb9bcedc00fbe7af6bb72745f4c
2026-08-31 19:44:44 +00:00
Benjamin Carlsson
65237aeca0 Fix Windows native voice dependency builds (#41894)
## Why

Windows builds combine Cygwin build tools with native MSVC outputs. On ARM64,
Cygwin can run under x64 emulation, so inferred host details and untranslated
paths can select the wrong target or leak POSIX paths into native metadata.

## What changed

- Convert libffi source, prefix, and shell paths with `cygpath`, preserve
  `USERPROFILE`, and pass explicit build and host targets.
- Configure libffi's MSVC and libtool environment to produce and install its
  shared library and import library, including across recursive make calls.
- Reject libffi pkg-config output containing `/cygdrive/` paths and enable
  Opus NEON support for `aarch64-pc-windows-msvc`.
- Document the additional Windows build prerequisites.

## Testing

- Extend `test_build_native.py` with Windows path conversion, x64 and ARM64
  target configuration, recursive make flag propagation, pkg-config, and Opus
  coverage.

GitOrigin-RevId: 12eb4ff83e141aa069abd68af4839f06e4c45b8f
2026-08-31 19:02:31 +00:00
Benjamin Carlsson
fc7d34ad67 Add native voice dependency build recipe (#41890)
## Why

The pinned voice source inputs can be prepared but do not yet provide compiled
native libraries for downstream integration.

## What changed

- Add `build_native.py` to build a shared-library prefix for the supported voice
  dependencies on native x64 and ARM64 GNU/Linux, macOS, and Windows MSVC hosts.
- Require explicit toolchain inputs, isolate dependency discovery to the output
  prefix, and record command logs and build state for provenance and failures.
- Expose the recipe and source inputs through `//third_party/voice:build_inputs`
  and document its prerequisites, outputs, and integration boundaries.

## Testing

- `PYTHONSAFEPATH=1 python3 -m unittest discover -s third_party/voice -p 'test_build_native.py'`

GitOrigin-RevId: 132d93561a8e1bb178518f1248cf87d031060aae
2026-08-31 18:05:19 +00:00
Benjamin Carlsson
379d50be35 Add pinned native voice source preparation (#41884)
## What changed

- Pin the GStreamer, codec, native library, and build-tool archives needed by
  the native voice runtime, including their SHA-256 digests.
- Add lazy Bazel repositories and a manual `//third_party/voice:sources` target
  for fetching and unpacking the pinned archives.
- Add an offline Python adapter that verifies archive identity and extraction
  bounds, refuses existing output directories, cleans up failed preparations,
  and records a completion receipt.
- Document that this stage prepares source inputs only; it does not compile or
  link the libraries or enable voice.

## Testing

- Add unit coverage for digest failures, unsafe links, expansion limits,
  cleanup after extraction failures, existing-output preservation, and systems
  where archive links cannot be created.

GitOrigin-RevId: 42a71d7b6b91d49b3954a40932033c44f9861f04
2026-08-31 17:25:50 +00:00