Commit Graph
15 Commits
Author SHA1 Message Date
Benjamin Carlsson a947db131b Add Bazel preparation for native voice runtimes (#43114)
## What changed

Add the manual `//third_party/voice:native_runtime` target for macOS and GNU Linux. It consumes the native prefix archive, validates the completed build receipt against the target and source manifest, inspects libraries, and prepares verified copies using the existing platform policies.

Record the build commit through Bazel workspace status and include it in runtime receipts. Reject invalid or ambiguous commit metadata and escaping library aliases. Keep macOS preparation local and allow the system `libc++.1.dylib` dependency.

## Testing

Add tests for receipt validation, inspection ordering and failures, invalid commit metadata, and escaping archive links. Add a macOS integration test for preparing real libraries from a sandbox-linked archive with native aliases and either an absent or empty output directory.

GitOrigin-RevId: 23717ab33d1d3da5fb0042c1cc8e8efcd333c386
2026-09-05 22:27:22 +00:00
Benjamin Carlsson a31c18ab7a Add a Bazel target for native voice dependencies (#43111)
## What changed

Add `//third_party/voice:native_prefix` to run the existing native build recipe with Bazel-declared archives and toolchain inputs on matching macOS and Linux GNU 2.28 targets for x64 and ARM64. The manual target exports `prefix.tar` and a `built.json` receipt through the `receipt` output group.

Keep macOS actions local to access Apple tools, use Apple's linker for libffi partial links, and preserve compiler runtime flags through libtool on Linux. Surface upstream build failure logs in the Bazel action output.

Document the build's host utility requirements and the distinction between the raw prefix and a relocatable SDK or runtime package.

GitOrigin-RevId: b4917943f621dec07bb994392619e8a42e496b8b
2026-09-05 22:07:21 +00:00
Benjamin Carlsson e67a8ae6c9 Add explicit toolchain inputs for native voice builds (#43109)
## Why

Native voice builds ignore ambient compiler flags, but need a way to accept declared compiler, linker, and archive-tool inputs.

## What changed

- Add repeated `--c-flag`, `--cxx-flag`, and `--link-flag` options plus optional `--ar` and `--ranlib` paths for Unix builds. Record flags and pass them through CMake, Meson, and libffi, including Objective-C flags on macOS.
- Preserve literal libffi compiler arguments with response files. Reject flags containing whitespace, libffi paths requiring shell quoting, and Unix toolchain overrides on Windows.
- Add a pinned Bazel `pkg-config` tool built with LLVM archive tools and native macOS configure checks. Register the pinned Make toolchain and patch its bootstrap to pass preprocessor flags and enable cross-compilation configuration.

## Testing

Extend native-build tests to verify compiler and linker flags, explicit archive tools, Meson and Objective-C inputs, literal definitions through shell and recursive Make expansion, and rejection of unsupported overrides.

GitOrigin-RevId: e5ea541ec76abfa170f6b0411b0f14cd698c07e7
2026-09-05 21:47:17 +00:00
Benjamin Carlsson 47ca4619be Include GIO in voice SDKs and native runtimes (#43102)
## Why

The Rust GStreamer bindings link GLib's GIO library even when none of the selected plugins imports it. Following only plugin dependencies can omit a library the bindings need.

## What changed

- Export `gio-2.0` pkg-config metadata in the voice SDK.
- Require GIO as a runtime dependency root on Linux, macOS, and Windows, including its transitive dependencies under the existing import checks.
- Reject runtime packages that omit the required GIO library.

## Testing

Extend fixtures to cover GIO inclusion and missing-library rejection across platforms, loading GIO and its transitive dependency after moving a Linux runtime, and resolving `gio-2.0` through pkg-config after moving an SDK.

GitOrigin-RevId: d12e45d387d56c54d94d788cf07d82bb22c2b4a3
2026-09-05 21:18:44 +00:00
Benjamin Carlsson 5d35805715 Add receipt-verified native voice SDK export (#43099)
## What changed

Add `third_party/voice/sdk.py` to export headers, development libraries, and selected pkg-config metadata from an inspected native build. Verify shared libraries against inspection receipts and record the target, source commit, source manifest hash, and exported file hashes in `sdk.json`.

Enable relocatable Meson pkg-config metadata and materialize development library aliases. Reject mismatched receipts, paths escaping the build prefix, nonrelocatable metadata, and unsafe output locations; remove partial output on copy failure. Document SDK consumption as build inputs separate from packaged runtime files.

## Testing

Add tests for exported hashes and pkg-config resolution after moving the SDK and deleting the original prefix, alias materialization, invalid inputs, and cleanup after copy failure.

GitOrigin-RevId: 6d1575e659a02cd1f56e465e7830a296e8eaf486
2026-09-05 20:54:13 +00:00
Benjamin Carlsson 0683368584 Supply Bazel-managed CMake and Ninja for the bundled Opus build (#43083)
## Why

The bundled Opus encoder uses CMake and should not depend on tools available on a build worker's `PATH`.

## What changed

- Add `rules_foreign_cc` and provide its pinned CMake and Ninja tools and data to the `opusic-sys` build script.
- Add a CMake toolchain file that selects the declared Ninja executable, resolves the archiver path, and sets the target system and processor from `TARGET`, while retaining Bazel's compiler, SDK, and linker inputs.

GitOrigin-RevId: 48580c506d088d8c76cce958589ff94da458bd79
2026-09-05 19:31:24 +00:00
Benjamin Carlsson dc0dc4f15d Package prepared runtimes with the voice host (#42332)
## What changed

- Add an optional `--runtime` input to `assemble_package.py`.
- Validate the runtime receipt, target, source manifest, plugin inventory, paths,
  and file hashes before copying only the declared runtime files.
- Preserve each platform's runtime layout and record the copied files and
  `runtime.json` in the package manifest.
- Recheck hashes after copying and remove the new output if assembly fails.

## Testing

Add package assembly tests for Linux, macOS, and Windows layouts, invalid
receipts and paths, modified inputs, symlinked directories, and copy-time
changes.

GitOrigin-RevId: 8c9609af8ced406926707b2b0e1dcd6d2251f94b
2026-09-02 19:13:32 +00:00
Benjamin Carlsson eb10d91e48 Add Windows voice runtime preparation (#42209)
## What changed

- Add an x64 and ARM64 MSVC runtime preparer that uses `dumpbin` to validate
  PE32+ DLL metadata, selects the declared GStreamer plugins and dependency
  closure, and copies them unchanged into a private `bin/` directory.
- Reject malformed or unsupported PE metadata, path-bearing imports, delayed
  imports, managed DLLs, forwarded exports, undeclared dependencies, and
  case-insensitive DLL identity conflicts.
- Keep `third_party/voice/sources.json` line endings stable across Windows
  checkouts so native build receipts remain valid.

## Testing

Add native Windows tests for relocated DLL loading with a restricted search
path, receipt and digest failures, duplicate identities, malformed PE headers,
unsupported loader features, and cleanup after failed preparation.

GitOrigin-RevId: 5375f21fc54f5d597f3d4e5f8fbca3df307fa2a8
2026-09-02 05:23:24 +00:00
Benjamin Carlsson 8d01cd42fa Add GNU Linux voice runtime preparation (#42208)
## Why

Native voice libraries need package-relative loader paths so a prepared runtime
can be moved without retaining references to its build prefix.

## What changed

- Configure CMake and GNU Linux Meson builds with relative runtime paths, while
  also setting relocatable install names and paths for CMake libraries on macOS.
- Add a GNU Linux runtime preparer for x64 and ARM64 that validates bounded ELF64
  metadata, selects the declared GStreamer plugins and dependency closure, and
  preserves the `lib/gstreamer-1.0/` layout.
- Reject malformed ELF metadata, path-bearing imports, unsupported loader
  dependencies, and native outputs that still contain incompatible runtime
  paths.

## Testing

Add native tests for relative build paths, relocated library loading, dependency
and digest failures, malformed ELF inputs, and cleanup after failed preparation.

GitOrigin-RevId: ed94819b07d20214b56363446b64f765e08d75fa
2026-09-02 05:14:37 +00:00
Benjamin Carlsson 8813bd4b00 Add macOS voice runtime projection (#42204)
## What changed

- Add a shared runtime preparer that validates native build receipts, source and
  file digests, dependency closure, and output containment before copying the
  required voice plugins and libraries.
- Add macOS-specific Mach-O inspection and relocation that removes build-time
  runpaths, rewrites non-system dependencies relative to each loader, applies
  development ad-hoc signatures, and records the resulting payload in
  `runtime.json`.
- Close the state runtime explicitly in the clear-memories CLI test.

## Testing

- Add macOS tests covering relocation after the source prefix is removed and
  fail-closed handling of invalid receipts, changed inputs, unsafe paths,
  dependency conflicts, malformed Mach-O metadata, and failed transforms.

GitOrigin-RevId: 3be7a6fabf0135a57036cbb0a9a3e597096930fd
2026-09-02 04:52:32 +00:00
Charlie Marsh 84aa75204a Format Python scripts across the repository (#42109)
## Why

The root Python formatting pass only covered `scripts/`, leaving Python utilities elsewhere in the repository outside `just fmt` and `just fmt-check`.

## What changed

- Run the scripts Ruff formatter from the repository root.
- Exclude `sdk/python`, which has its own formatter pass, and `codex-rs/vendor`.
- Apply Ruff formatting to the newly covered Python files and update the formatter coverage test.

GitOrigin-RevId: 73ac82b8bb9c713d589724f6c4f926399febb513
2026-09-01 17:47:29 +00:00
Benjamin Carlsson 13bc770eaf Add installed voice host lifecycle support (#41902)
## What changed

- Add `VoiceHost` to resolve the packaged voice helper, launch it with an
  allowlisted environment, perform the protocol handshake, and enforce bounded
  shutdown and process cleanup.
- Preserve native executable path encoding in the pipe process APIs.
- Add `third_party/voice/assemble_package.py` to create a fresh package copy
  containing a target-compatible helper and a provenance manifest with file
  hashes.

## Testing

- Cover installed helper lifecycle, build matching, missing and symlinked
  helpers, non-UTF-8 package paths, environment filtering, package validation,
  target pairing, and failure cleanup.

GitOrigin-RevId: f893074b36ae6bb9bcedc00fbe7af6bb72745f4c
2026-08-31 19:44:44 +00:00
Benjamin Carlsson 65237aeca0 Fix Windows native voice dependency builds (#41894)
## Why

Windows builds combine Cygwin build tools with native MSVC outputs. On ARM64,
Cygwin can run under x64 emulation, so inferred host details and untranslated
paths can select the wrong target or leak POSIX paths into native metadata.

## What changed

- Convert libffi source, prefix, and shell paths with `cygpath`, preserve
  `USERPROFILE`, and pass explicit build and host targets.
- Configure libffi's MSVC and libtool environment to produce and install its
  shared library and import library, including across recursive make calls.
- Reject libffi pkg-config output containing `/cygdrive/` paths and enable
  Opus NEON support for `aarch64-pc-windows-msvc`.
- Document the additional Windows build prerequisites.

## Testing

- Extend `test_build_native.py` with Windows path conversion, x64 and ARM64
  target configuration, recursive make flag propagation, pkg-config, and Opus
  coverage.

GitOrigin-RevId: 12eb4ff83e141aa069abd68af4839f06e4c45b8f
2026-08-31 19:02:31 +00:00
Benjamin Carlsson fc7d34ad67 Add native voice dependency build recipe (#41890)
## Why

The pinned voice source inputs can be prepared but do not yet provide compiled
native libraries for downstream integration.

## What changed

- Add `build_native.py` to build a shared-library prefix for the supported voice
  dependencies on native x64 and ARM64 GNU/Linux, macOS, and Windows MSVC hosts.
- Require explicit toolchain inputs, isolate dependency discovery to the output
  prefix, and record command logs and build state for provenance and failures.
- Expose the recipe and source inputs through `//third_party/voice:build_inputs`
  and document its prerequisites, outputs, and integration boundaries.

## Testing

- `PYTHONSAFEPATH=1 python3 -m unittest discover -s third_party/voice -p 'test_build_native.py'`

GitOrigin-RevId: 132d93561a8e1bb178518f1248cf87d031060aae
2026-08-31 18:05:19 +00:00
Benjamin Carlsson 379d50be35 Add pinned native voice source preparation (#41884)
## What changed

- Pin the GStreamer, codec, native library, and build-tool archives needed by
  the native voice runtime, including their SHA-256 digests.
- Add lazy Bazel repositories and a manual `//third_party/voice:sources` target
  for fetching and unpacking the pinned archives.
- Add an offline Python adapter that verifies archive identity and extraction
  bounds, refuses existing output directories, cleans up failed preparations,
  and records a completion receipt.
- Document that this stage prepares source inputs only; it does not compile or
  link the libraries or enable voice.

## Testing

- Add unit coverage for digest failures, unsafe links, expansion limits,
  cleanup after extraction failures, existing-output preservation, and systems
  where archive links cannot be created.

GitOrigin-RevId: 42a71d7b6b91d49b3954a40932033c44f9861f04
2026-08-31 17:25:50 +00:00