## What changed
- Add a tag-release workflow gated by `CODEX_PROVISIONED_MACOS_CANDIDATE` for Apple Silicon and Intel macOS candidate artifacts.
- Package the CLI in `CodexCLI.app` with an embedded provisioning profile and a relocatable `bin/codex` launcher. Validate independently supplied profile, certificate, and team expectations before signing.
- Sign and notarize candidates, then verify signatures, entitlements, architecture, stapling, and Gatekeeper acceptance on macOS. Retain verified artifacts after package smoke tests pass.
- Filter code-mode smoke-test requests to `/v1/responses` so analytics requests are excluded from response parsing.
## Testing
Add tests for profile validation, launcher relocation and symlink handling, standard and provisioned signing flows, and rejection of signing, notarization, identity, and entitlement failures. The candidate workflow runs package smoke tests, including sandboxed code mode.
GitOrigin-RevId: f214f6a23dd10df62cd72a2c63c20cb0864fbc9c
## What changed
Add a downstream workflow that builds the Python SDK and runtime from the stable CLI release commit, using the CLI version for both packages and the SDK's exact runtime dependency. Publish and verify the runtime on PyPI before publishing the SDK.
Require a successful CLI `release` job, an unchanged release tag, and complete runtime assets. Skip CLI prereleases and allow publication despite unrelated publisher failures. Support retries by CLI workflow run ID and accept existing PyPI uploads while verifying the complete release.
Document release setup, retry procedures, and independent SDK releases.
## Testing
Add resolver unit tests covering tag resolution, prerelease skipping, partial reruns, pagination, invalid runs, moved tags, missing assets, and equivalent automatic and manual release resolution.
GitOrigin-RevId: 4ec6b2e77c94c851507dbe7200ea420994fce36e
## What changed
- Build the voice host and native runtime with Bazel's `-c opt` configuration.
- Resolve runtime paths before validating their contents in the release workflow.
- Make `runtime.json` owner-writable before sealing the post-signing release receipt.
- Use `scripts` in `PYTHONPATH` so archive creation can import `codex_package.archive`.
GitOrigin-RevId: 02f960573a4151f52cde486e17e1d45b73435306
## Why
The SDK publish job needs to wait for runtime wheels to become available on PyPI, and release reruns need to tolerate SDK files that have already been uploaded.
## What changed
- Require runtime publication and verification before publishing the SDK, and enable `skip-existing` for SDK uploads.
- Share a PyPI verifier between runtime and SDK releases. Require the exact expected artifact set, including the SDK wheel and source distribution.
- Canonicalize versions with `packaging.version.Version` and retry registry errors, malformed responses, and incomplete artifact sets with a bounded retry loop.
## Testing
Add unit tests for transient failures and malformed responses, waiting for complete SDK artifacts, canonical version lookup, and retry exhaustion when runtime wheels are missing. Run them in repository checks.
GitOrigin-RevId: 9cb2ddced4ce6d509ebfd130511ab3f39239dd62
## What changed
- Build, sign, and notarize `codex-voice-host` and its native runtime for Apple Silicon and Intel macOS release packages, granting the helper audio-input access.
- Include voice resources in primary package archives and DMGs, with a root-level `codex` symlink to `bin/codex` so the runtime can be located.
- Seal runtime receipts with post-signing hashes, require matching release versions and source builds, and bundle dependency notices, licenses, and source metadata.
- Keep voice resources out of Python wheels to preserve their older macOS compatibility; the native voice build targets macOS 14.
## Testing
Add packaging tests for alpha, beta, and stable versions, signed-byte preservation, license hashes, receipt validation, tamper detection, and exclusion of unlisted files. Extend release verification to check voice architectures, signatures, build identity, package hashes, and DMG contents.
GitOrigin-RevId: 9f9415a8b2532d655a9a8740bcdf64066ddb7472
## Why
The root Python formatting pass only covered `scripts/`, leaving Python utilities elsewhere in the repository outside `just fmt` and `just fmt-check`.
## What changed
- Run the scripts Ruff formatter from the repository root.
- Exclude `sdk/python`, which has its own formatter pass, and `codex-rs/vendor`.
- Apply Ruff formatting to the newly covered Python files and update the formatter coverage test.
GitOrigin-RevId: 73ac82b8bb9c713d589724f6c4f926399febb513
## Why
Release jobs download the `codex-zsh` manifest before packaging or signing its
bundled binaries. Verify that download against a pinned digest so an unexpected
manifest cannot enter release artifacts.
## What changed
- Pin the SHA-256 digest for the `codex-zsh-v0.1.0` manifest.
- Add a portable verification script that uses `sha256sum` or macOS `shasum`.
- Run verification in both the package archive and macOS helper signing flows.
GitOrigin-RevId: a404bb897c8cc0bf3c0ff38e8a6e1cce059d7fab
## What changed
- Load the Apple issuer ID from the notarization key's `apple-issuer-id`
tag alongside its key ID and pinned version.
- Require the tag to contain a valid UUID before creating the notarization JWT.
- Remove the separate `APPLE_NOTARIZATION_ISSUER_ID` environment variable and
release workflow secret wiring.
## Testing
- Cover valid, missing, empty, and malformed issuer ID tags in the macOS
notarization tests.
GitOrigin-RevId: c42da96a36293cf39312d8238f958f6898247f19
## Why
Keep the App Store Connect private key in Azure Key Vault instead of exporting it to release runners as a base64-encoded `.p8` secret.
## What changed
- Add a notarization client that creates notary-scoped Apple JWTs with a pinned Azure Key Vault key, submits and streams artifacts to Apple's notarization service, polls for completion, and saves developer diagnostics.
- Update the binary and DMG release wrappers to use the new client while continuing to staple accepted DMGs with `rcodesign`.
- Configure release jobs with the notarization vault key name and version instead of private-key material.
## Testing
- Add unit coverage for key validation, JWT signing, uploads, polling, diagnostic logs, and the binary and DMG wrapper contracts.
GitOrigin-RevId: 8f526c5691d774964fd33f847f8420cac0840b05
## Why
R2 publication waited for DotSlash to finish before uploading any release
assets, extending the release pipeline's critical path.
## What changed
- Start an `assets` stage after the GitHub release is created, in parallel with
DotSlash publication. Upload available assets concurrently without publishing
release metadata or channel aliases.
- Run a `finalize` stage after both jobs complete. Verify assets already present
in R2, upload any assets DotSlash added, and then publish metadata, stable
installer aliases, and release channels.
- Validate downloaded assets before upload and retain no-overwrite semantics
across both stages.
GitOrigin-RevId: 1c3e8ca715be4064c2532a5c651f1bbe03ed96a4
## What changed
- Write staged `rusty_v8` checksum manifests with LF line endings on every platform.
- Strip carriage returns when verifying manifests so existing Windows-built releases with CRLF line endings remain usable.
## Testing
- Assert that staged checksum manifests contain no carriage returns.
GitOrigin-RevId: 142855c147a08a60f5e5782c61e2fcd6b2e350d6
## Why
The Rust release workflow accepts alpha versions with a second numeric suffix,
such as `1.2.3-alpha.4.1`, but the R2 publisher rejects them as invalid.
## What changed
- Allow up to two numeric suffix components for alpha versions in
`publish_r2_release.py`.
- Document that its version pattern must remain aligned with release tag
validation in `rust-release.yml`.
GitOrigin-RevId: 8df68ce4f4d3e652367f7394a642827f21979d70
## Why
The macOS release workflow fetched `rg` and zsh while assembling package
archives, after the signing stage. This left the bundled helper executables
outside the workflow's signing and notarization checks.
## What changed
- Fetch, sign, notarize, and upload the pinned macOS `rg` and zsh binaries with
the other release artifacts.
- Build package archives from those signed helpers via `--rg-bin` and the new
`--zsh-bin` override.
- Verify the helpers' architecture, signatures, and absence of entitlements in
the final package.
## Testing
- Cover the prebuilt zsh override and verify that package assembly preserves
the supplied helper binaries.
GitOrigin-RevId: a3865c04fa2f0f4df32e627ee7202bc87bdc3241
## What changed
- Publish `install.sh` and `install.ps1` from stable release assets to the
mutable `codex/install.sh` and `codex/install.ps1` R2 keys.
- Require both installer assets and verify each uploaded object's size,
SHA-256 metadata, and CRC64 checksum before completing publication.
- Leave prerelease publication unchanged.
GitOrigin-RevId: 869f5fcd6c5f5d7fa4dc39c5216b31409d595fc9
## What changed
- Publish the verified `release.json` metadata to `codex/channels/latest` for
stable releases and `codex/channels/prerelease` for prereleases.
- Pass the release classification through the reusable R2 workflow so GitHub
Releases and R2 channels use the same stable-versus-prerelease decision.
- Allow channel objects to be replaced while keeping versioned release objects
immutable, and verify each channel upload before completing the publication.
GitOrigin-RevId: bd5c728c5565befe42b041ee1a032b111daca1c2
## What changed
- Add a reusable workflow that runs after the GitHub Release and DotSlash publishing jobs and mirrors each release asset to a versioned R2 prefix.
- Validate GitHub asset sizes and SHA-256 digests before upload, publish objects without overwriting existing files, and verify their remote size, digest metadata, and CRC64 checksum.
- Generate `release.json` with installer-facing asset URLs and digests, then publish and verify it alongside the release assets.
- Allow an in-progress Rust release workflow to finish instead of cancelling it when another run starts.
GitOrigin-RevId: e6717cd27914994accced6e2c5df696f0e9a558e
## Why
Windows Cargo and Bazel jobs spend significant time in filesystem-heavy
build and cache directories. Route those directories through one CI
build root so Windows can use its Dev Drive and Unix can use a stable
cache root.
## What
- Have `setup-ci` define `CI_BUILD_ROOT`, `CARGO_TARGET_DIR`, Bazel
cache/output paths, and temp paths.
- Require Windows to find or provision a verified Dev Drive instead of
falling back to `C:`.
- Pass the shared Bazel output base to `setup-bazel` so its explicit
`output_base` does not defeat Dev Drive routing.
- Point nextest, release, and V8 source-build paths at the shared
environment contract.
## Benchmark results
One-off cold-cache WPR/ETW traces show the explicit Bazel output-base
routing removes the dominant `C:` traffic:
| sample | `C:\_bazel` | summed `C:` traffic | traced test step |
|---|---:|---:|---:|
| shard 1 before | 62.2 GiB | 85.2 GiB | 16m22s |
| shard 1 updated | 0 | 16.5 GiB | 12m05s |
| shard 3 before | 67.2 GiB | 84.6 GiB | 16m48s |
| shard 3 updated | 0 | 13.5 GiB | 11m08s |
For a cold x64 V8 source build, the retained build-tail sample showed
`D:\cargo-target` at ~1.29 GiB while measured `C:` roots totaled ~0.45
GiB (`C:\Users` ~0.33 GiB, `C:\Program Files` ~0.06 GiB, `C:\Windows`
~0.03 GiB). The full cold build took 2h20m36s.
The Bazel timing improvement is directional because both refreshed
shards failed tests. The V8 trace is a bounded build-tail sample, not
the full build. All final samples had zero lost ETW events; VHDX traffic
was excluded from the optimization ranking.
Runs: [baseline
Bazel](https://github.com/openai/codex/actions/runs/28911908527),
[updated
Bazel](https://github.com/openai/codex/actions/runs/28917133701), [V8
build tail](https://github.com/openai/codex/actions/runs/28933626678).
## Manual validation
- Ran `just fmt`.
- Ran `just test-github-scripts` (35 tests).
- Parsed GitHub Actions YAML with `yq`.
- Ran `git diff --check`.
## Stack
- [#31332](https://github.com/openai/codex/pull/31332) — parameterize
Cargo target paths
- [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025
runner bump
- [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O
routing
## Why
Prepare CI jobs for a later build-output relocation without changing
where they write today.
## What
- Export `CARGO_TARGET_DIR` from `setup-ci` at the existing
`codex-rs/target` path.
- Route nextest, release, artifact, and signing paths through
`CARGO_TARGET_DIR`.
- Require V8 staging callers to pass an explicit target directory while
preserving the existing upstream path.
## Manual validation
- Ran `just test-github-scripts`.
- Parsed GitHub Actions YAML with `yq`.
## Stack
- [#31332](https://github.com/openai/codex/pull/31332) — parameterize
Cargo target paths
- [#31356](https://github.com/openai/codex/pull/31356) — Windows 2025
runner bump
- [#31357](https://github.com/openai/codex/pull/31357) — Dev Drive I/O
routing
## Why
Intel macOS release binaries crash on the first Code Mode tool call
while V8 creates its code range. The x86_64 V8 allocator later makes a
non-`MAP_JIT` reservation executable, which Hardened Runtime rejects
when the signature contains only `com.apple.security.cs.allow-jit`.
Tracks
[SE-8006](https://linear.app/openai/issue/SE-8006/intel-macos-codex-cli-crashes-in-v8-startup-on-gpt-56-sol-tool-calls).
Fixes#28390.
## What
- add an expanded entitlement profile only for x86_64 `codex` and
`codex-app-server`, the release binaries that link V8
- keep arm64 and `codex-responses-api-proxy` on the existing narrower
profile
- share one fail-closed target/binary selector between signing and final
verification
- verify the expected Mach-O architecture and exact entitlement
dictionary for the signed binary, tar.gz, zstd, package, and DMG copies
## Verification
- `just test-github-scripts` (34 tests)
- `UV_CACHE_DIR=/private/tmp/codex-uv-cache just fmt-check`
- `bash -n .github/scripts/macos-signing/select_codex_entitlements.sh`
- `plutil -lint` on both entitlement profiles
- parsed `rust-release.yml` as YAML
- `git diff --check`
- ad-hoc Hardened Runtime signing smoke on an x86_64 Mach-O slice:
strict `codesign` verification passed; the Codex profile contained
exactly both keys and the proxy profile retained exactly `allow-jit`
## Release validation
Run a native Intel smoke of the final Developer ID-signed x86_64 Codex
binary through V8 isolate creation before shipping. PR #30849 is
diagnostic scaffolding, but its non-sandbox release job currently fails
in the harness before V8 starts, so it is not counted as coverage here.
## Why
CI jobs repeat common bootstrap steps, which makes it harder to keep
Bazel and Cargo lanes aligned. Centralizing the lightweight setup gives
us one place for future runner-wide optimizations without adding Rust
toolchain or component installation to the Windows Bazel long poles.
## What
- add zero-input `.github/actions/setup-ci` to set Cargo's git
transport, install DotSlash and `just`, expose DotSlash from stable PATH
locations, and enable Windows Git long paths
- have `setup-bazel-ci` compose the common setup, then remove its
DotSlash/test-prerequisite plumbing
- migrate Bazel, Cargo CI, V8 canary, repo checks, nextest, and Windows
release call sites while keeping Rust toolchain and MSVC setup explicit
- preserve the existing nextest Dev Drive setup unchanged
## Validation
- `just test-github-scripts` (30 tests)
- parsed workflow and composite-action YAML with `yq`
## Summary
- build, strip, sign, and publish `codex-code-mode-host` with the
primary Codex release binaries on Linux, macOS, and Windows
- place the host beside `codex[.exe]` in canonical package archives,
macOS DMGs, and the legacy Linux bundle so the runtime's sibling lookup
succeeds
- preserve and validate the host through standalone installers and
Python runtime wheel staging
- add package-builder coverage for source selection and the resulting
package layout
## Why
The process-owned code-mode client launches `codex-code-mode-host` as a
sibling of the running Codex executable. Release artifacts currently
build and bundle `codex` without that host, so code mode cannot start
from installed packages.
## Why
It's hard to change the set of required jobs when they're managed in the
GitHub UI, and when each workflow is responsible for choosing it's own
scheduling it's easy to end up with skew between what we enforce on PRs
vs. on main.
## What
- add a `blocking-ci` caller workflow, triggered by pull requests and
pushes to `main`, for Bazel, blob size, cargo-deny, Codespell,
`repo-checks`, rust CI, and SDK CI
- add an `always()` terminal job named `CI required` that fails unless
every called workflow succeeds
- add a `postmerge-ci` caller workflow for `rust-ci-full` and
`v8-canary`, with a terminal `Postmerge CI results` job
- centralize V8 relevance detection in `v8_canary_changes.py`; unrelated
PR and postmerge runs execute metadata only and skip the expensive build
matrices
- leave `v8-canary` outside the blocking gate and leave the external
`cla` check independent
## Rollout
A repository admin must replace the existing required GitHub Actions
contexts with `CI required` in the main-branch ruleset. Retain `cla` as
a separate required check. Until that change is coordinated, this PR
cannot satisfy the old standalone check names. In-flight PRs will need
to be rebased after this lands.
## Why
Once #30114 publishes zsh independently, regular Rust releases should
reuse that protected, versioned artifact set instead of rebuilding
identical zsh binaries for every Codex version. Keeping the zsh release
tag explicit in the workflow also makes future artifact upgrades
deliberate and easy to review.
This PR assumes the first standalone artifact release will be published
as `codex-zsh-v0.1.0` before this change lands.
## What changed
- Added `CODEX_ZSH_RELEASE_TAG` near the top of
`.github/workflows/rust-release.yml`, initially pinned to
`codex-zsh-v0.1.0`.
- Download the standalone release’s generated `codex-zsh` DotSlash
manifest before assembling Linux and macOS Codex packages.
- Added a `--zsh-manifest` package-builder override so release packaging
fetches the matching target archive and verifies the size and SHA-256
digest recorded in that manifest.
- Removed the reusable zsh build job from regular Rust releases.
- Stopped copying zsh archives into each Rust release and stopped
regenerating a zsh DotSlash manifest there.
Windows packaging remains unchanged because the patched zsh resource is
only shipped for supported Unix targets.
## Testing
- Added package-helper coverage that supplies a standalone manifest
override and verifies the extracted zsh bytes.
- Ran the `scripts/codex_package` unit test suite.
- Validated `.github/scripts/build-codex-package-archive.sh` with `bash
-n`.
Avoid rebuilding sandboxed Windows MSVC V8 artifacts for unrelated
changes to `codex-rs/Cargo.toml`.
The V8 canary now compares the resolved V8 version between the base and
head commits and only runs the Windows source-build matrix when:
- the resolved V8 crate version changes;
- Windows artifact-production scripts or workflows change; or
- the workflow is manually dispatched.
The existing Bazel V8 matrix is unchanged.
## Why
The Windows MSVC source builds take roughly two to three hours and
currently run whenever any entry in the broad `v8-canary` path filter
changes.
## Summary
- Restore separate release symbol archives for macOS, Linux, and Windows
binaries.
- Build release binaries with `line-tables-only` debuginfo instead of
full debuginfo.
- Strip Unix distribution binaries after extracting symbols, preserve
Windows PDBs, and keep symbol archives available to the release job.
- Strip the packaged Linux `bwrap` binary before hashing it so the
embedded digest matches the distributed bytes.
## Root cause
The first symbol-artifact implementation enabled
`CARGO_PROFILE_RELEASE_DEBUG=full`. In the June 2 release runs, macOS
ARM primary builds reached the 90-minute timeout while still inside
`Cargo build`. After the symbol changes were reverted, the same primary
build completed in about 22 minutes. The archive step itself completed
in tens of seconds when reached.
Rust's `line-tables-only` debuginfo level preserves function names and
source locations for symbolication without emitting the heavier variable
and type information from full debuginfo.
## Validation
- Ran `just fmt` from `codex-rs`.
- Ran `just test-github-scripts` from the repository root: 23 tests
passed.
- Ran `bash -n` and `shellcheck` on
`.github/scripts/archive-release-symbols-and-strip-binaries.sh`.
- Parsed both modified workflows as YAML and ran `git diff --check`.
- Built a macOS release smoke binary with `line-tables-only`, archived
its dSYM through the restored script, stripped the production binary,
and verified that `atos` resolves `symbol_smoke_function` to
`main.rs:2`.
- Ran Linux archive-script control-flow coverage with stubbed `objcopy`
and `strip` commands.
- Ran Windows PDB archive staging coverage and verified
underscore-emitted Rust PDB names are staged under shipped hyphenated
binary names.
## Follow-up
The release workflow only runs for tags or manual dispatches, so CI
cannot dry-run the full release matrix on this PR. The next release run
will verify runner time and memory behavior under `line-tables-only`.
## Why
`just bazel-clippy` ran target discovery with
`--noexperimental_remote_repo_contents_cache`, then ran the build with
the workspace default `--experimental_remote_repo_contents_cache`. Bazel
therefore killed and restarted its server on each transition, slowing
repeated commands and discarding the in-memory analysis cache. An audit
found the same class of startup-option variation in several CI command
sequences.
## What changed
- Keep local lint target-discovery queries on the workspace-default
Bazel server, while making CI target discovery explicitly use the CI
startup options.
- Normalize GitHub Actions launches through the BuildBuddy wrapper to
share `BAZEL_OUTPUT_USER_ROOT` and
`--noexperimental_remote_repo_contents_cache`.
- Route the CI lockfile check and Windows test-shard query through the
same startup configuration.
- Document the startup-option invariant and add wrapper regression
coverage.
## Validation
- Confirmed consecutive local clippy target-discovery runs retained the
same Bazel server PID.
## Why
The public Codex release workflow needs to sign and notarize macOS
binaries and DMGs without placing the Developer ID private key in
GitHub. This moves the private-key operation behind the protected
`codesigning` environment and uses GitHub OIDC with Azure Key Vault
PKCS#11, while preserving the existing external `build_unsigned` /
`promote_signed` fallback.
## What changed
- Add a reusable AKV PKCS11 setup action that authenticates to Azure
with OIDC, downloads pinned signing tools, verifies their SHA-256
digests, and loads the public signing certificate from Key Vault.
- Replace the legacy macOS signing action with scripts that support
AKV-backed `rcodesign`, notarize signed binaries and DMGs, and staple
DMG notarization tickets.
- Restructure `rust-release.yml` so macOS builds produce unsigned
artifacts first, protected jobs perform signing and notarization, macOS
runners package and verify the results, and release publishing waits for
verified artifacts.
- Preserve the manual external-signing handoff flow and make manual-mode
conditions explicit.
- Move the Codex entitlements file alongside the signing scripts and
update CODEOWNERS for the new signing surfaces.
## Verification
- [Live protected signing workflow
run](https://github.com/openai/codex/actions/runs/26903610631) completed
successfully for both macOS architectures, including binary
signing/notarization, DMG signing/notarization, and final artifact
verification.
- Downloaded both signed DMGs and independently verified their checksums
and strict signatures.
- Confirmed `xcrun stapler validate` succeeds and Gatekeeper accepts
both DMGs as `Notarized Developer ID`.
- Mounted both DMGs and confirmed the contained `codex` and
`codex-responses-api-proxy` binaries have valid Developer ID signatures
for the expected architectures.
---------
Co-authored-by: shijie-openai <shijie.rao@openai.com>
## Why
PR #25905 intentionally adds a failing `codex-core` unit test, but its
[Bazel test on Windows
check](https://github.com/openai/codex/actions/runs/26837526950/job/79135369259)
passed. That shows the Bazel configuration introduced by #25156 is not
behaving as expected, so revert it while the configuration can be
investigated separately.
## What changed
Revert #25156 in full, restoring the previous Bazel remote
configuration, CI scripts, workflows, `rusty_v8` handling, and
documentation. This removes the shared BuildBuddy wrapper and its tests.
## Validation
Not run locally; this exact revert was prioritized for a fast rollback.
## Why
Bazel remote configuration was selected in several CI scripts and
workflow steps. That made the BuildBuddy tenant policy easy to duplicate
and harder to audit, especially for fork pull requests that must not use
the OpenAI tenant.
This builds on
[sluongng/buildbuddy-ci-host-routing](https://github.com/openai/codex/compare/main...sluongng:codex:sluongng/buildbuddy-ci-host-routing)
and consolidates the policy in one place.
## What to do if this breaks you
See `codex-rs/docs/bazel.md` for details. TLDR:
1. make a BuildBuddy API key and put it in `~/.bazelrc`
2. if you're an OpenAI employee, add `common
--config=buildbuddy-openai-rbe` to `user.bazelrc` in the repo root
Run `just bazel-test` to ensure it works.
Note that `just bazel-remote-test` no longer exists, you need to select
a remote configuration as documented to use RBE.
## What changed
- Add `.github/scripts/run_bazel_with_buildbuddy.py` as the shared Bazel
wrapper and Python library. It selects the OpenAI host only for trusted
upstream GitHub Actions runs, routes keyed fork runs to the generic
host, and falls back to local Bazel execution when no key is available.
- Move endpoint selection into explicit `.bazelrc` configurations and
update Bazel CI, query helpers, and `rusty_v8` staging to use the shared
policy. Loading-phase target-discovery queries remain local.
- Add wrapper and `rusty_v8` unit coverage, plus `just test-scripts` for
the `.github/scripts` Python tests.
- Document local Bazel usage, `user.bazelrc` setup, BuildBuddy
configurations, and CI behavior in `codex-rs/docs/bazel.md`.
## Validation
- `just test-scripts`
- `bash -n .github/scripts/run-bazel-ci.sh
.github/scripts/run-bazel-query-ci.sh
.github/scripts/run-argument-comment-lint-bazel.sh
scripts/list-bazel-clippy-targets.sh`
- `python3 -m py_compile .github/scripts/run_bazel_with_buildbuddy.py
.github/scripts/test_run_bazel_with_buildbuddy.py
.github/scripts/test_rusty_v8_bazel.py
.github/scripts/rusty_v8_bazel.py`
- `ruff check .github/scripts/run_bazel_with_buildbuddy.py
.github/scripts/test_run_bazel_with_buildbuddy.py
.github/scripts/test_rusty_v8_bazel.py
.github/scripts/rusty_v8_bazel.py`
## Why
Production Codex binaries are stripped for distribution, which leaves
crashes and samples from released builds without the symbols needed for
useful stack traces. Publish symbols as separate release assets so
production artifacts stay small while released builds remain
symbolicateable.
## What changed
- Add `.github/scripts/archive-release-symbols-and-strip-binaries.sh` to
package platform-native symbols into `codex-symbols-<artifact>.tar.gz`
assets while stripping the corresponding Unix binaries before signing.
- Build release binaries with full debug information before producing
distribution artifacts.
- Publish macOS `.dSYM` bundles, Linux `.debug` files with
`.gnu_debuglink`, and Windows `.pdb` files.
- Strip Linux `bwrap` before computing its packaged-resource digest, but
intentionally omit `bwrap` from symbol archives.
- Preserve symbols artifacts in the unsigned macOS promotion flow.
## Verification
- Ran `shellcheck` and `bash -n` on
`.github/scripts/archive-release-symbols-and-strip-binaries.sh`.
- Parsed the modified workflow YAML files and ran `git diff --check`.
- Built a macOS release smoke binary and verified that the archived
`.dSYM` contains DWARF application source information and has the same
UUID as the stripped production binary.
- Built Linux smoke binaries and verified that the symbol archive
contains `codex.debug`, excludes `bwrap.debug`, leaves the expected
`.gnu_debuglink` in `codex`, and does not mutate the separately stripped
`bwrap` digest.
- Staged a Windows smoke archive and verified that it contains the
expected `.pdb` file.
It seems that this was added to allow rustc to load proc macros that had
been compiled with UBSan enabled, which zig does for debug and
`ReleaseSafe` builds. When zig drives the link of the final binary it
knows to include the ubsan runtime, but our zig-built artifacts are
being linked into a binary whose linking rustc drives. This removes the
libubsan workaround we have and replaces it with
`-fno-sanitize=undefined` passed to zig.
The new argument is passed at the end of zig's args so should take
precedence over any earlier arguments from the script's caller.
## Why
Release packaging should be a staging step once release binaries have
already been built and signed. The Windows release job was downloading
and signing `codex-command-runner.exe` and
`codex-windows-sandbox-setup.exe`, but `scripts/build_codex_package.py`
still rebuilt those helpers while creating the package archives.
That makes the package step slower and, more importantly, risks putting
helper binaries in the archive that were produced after the signing
step. Linux had the same shape for package resources: `bwrap` could be
rebuilt by the package builder instead of being passed in as a prebuilt
release artifact.
This builds on #23752, which fixes `.tar.zst` creation when Windows
runners rely on the repository DotSlash `zstd` wrapper.
## What changed
- Add explicit prebuilt resource inputs to the Codex package builder:
- `--bwrap-bin`
- `--codex-command-runner-bin`
- `--codex-windows-sandbox-setup-bin`
- Make `.github/scripts/build-codex-package-archive.sh` pass resource
binaries from the release output directory when they are already
present.
- Build Linux `bwrap` for app-server release jobs too, so app-server
package creation does not invoke Cargo just to supply the package
resource.
- Keep macOS package creation as a no-Cargo path when `--entrypoint-bin`
is provided, since macOS packages have no resource binaries.
- Add unit coverage showing prebuilt macOS, Linux, and Windows package
inputs result in no source-built binaries.
## Verification
- `python3 -m unittest discover -s scripts/codex_package -p 'test_*.py'`
- `python3 -m py_compile scripts/codex_package/*.py`
- `bash -n .github/scripts/build-codex-package-archive.sh`
- Dry-ran Linux and Windows package builds with fake prebuilt resources
and a nonexistent Cargo path to verify the package builder did not
invoke Cargo.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23759).
* #23760
* __->__ #23759
## Summary
DotSlash should resolve the same canonical package archives used by
standalone installers and npm platform packages, rather than continuing
to point at single-binary zstd artifacts or the older Linux bundle
archive.
This updates the Codex CLI and `codex-app-server` DotSlash release
config entries to match `codex-package-<target>.tar.gz` and
`codex-app-server-package-<target>.tar.gz`, with paths that select
`bin/codex` or `bin/codex-app-server` inside the extracted package. The
other helper outputs stay on their existing per-binary artifacts for
now.
## Test plan
- `python3 -m json.tool .github/dotslash-config.json > /dev/null`
- Ran a Python regex smoke test that checked every updated `codex` and
`codex-app-server` platform entry against the archive names emitted by
`.github/scripts/build-codex-package-archive.sh`.
## Why
Release CI already builds the Codex entrypoints before staging
artifacts, and the package builder can now package those prebuilt
binaries directly. The workflow should produce package-shaped sidecar
archives from the same staged entrypoints that downstream distribution
channels will eventually consume, without rebuilding `codex` or
`codex-app-server` inside the packaging step.
This intentionally does **not** publish the new package archives as
GitHub Release assets yet. The archives are kept with workflow artifacts
until npm, Homebrew, `install.sh`, winget, and related consumers are
ready to switch over.
## What changed
- Adds a `Build Codex package archive` step to
`.github/workflows/rust-release.yml` after target artifacts are staged.
- Runs `scripts/build_codex_package.py` for both release bundles:
- `primary` builds `codex-package-${TARGET}.tar.gz` with `--variant
codex`.
- `app-server` builds `codex-app-server-package-${TARGET}.tar.gz` with
`--variant codex-app-server`.
- Passes `--entrypoint-bin target/${TARGET}/release/<entrypoint>` so
packages contain the entrypoint already built by the workflow.
- Deletes both package archive names before the final GitHub Release
upload so they remain workflow artifacts only for now.
## Verification
- Parsed `.github/workflows/rust-release.yml` with Ruby's YAML loader.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23582).
* #23596
* __->__ #23582
## Why
`rust-ci-full` was paying the full Cargo nextest build-and-run cost once
per platform, with Windows ARM64 as the long pole. This change moves the
heavy work into one reusable per-platform flow: build a nextest archive
once, then replay it across four shards so the platform lane spends less
time running tests serially. For Windows ARM64, the archive is
cross-compiled on Windows x64 and replayed on native Windows ARM64
shards so the slow ARM64 machine is used for execution rather than
compilation.
## What changed
- split the `rust-ci-full` nextest matrix into five explicit
per-platform reusable-workflow calls
- add `.github/workflows/rust-ci-full-nextest-platform.yml` to build one
archive, upload timings/helpers, replay four nextest shards, upload
per-shard JUnit, and roll the shard status back up per platform
- add Windows CI helpers for Dev Drive setup and MSVC ARM64 linker
environment export so the Windows ARM64 archive can be produced on
Windows x64
- keep the existing Cargo git CLI fetch hardening inside the reusable
workflow, since caller workflow-level `env` does not flow through
`workflow_call`
- document the archive-backed shard shape in
`.github/workflows/README.md`
- raise the default nextest slow timeout to 30s so the sharded full-CI
path does not treat every >15s test as stuck
## Verification
- validated the archive/shard flow with live GitHub Actions runs on this
PR branch
- Windows ARM64 cross-compile latency on completed runs:
- https://github.com/openai/codex/actions/runs/26118759651: `34m30s`
lane e2e, `17m16s` archive build, `9m55s` shard phase
- https://github.com/openai/codex/actions/runs/26120777976: `30m36s`
lane e2e, `17m21s` archive build, `6m50s` shard phase
- comparable pre-cross-compile sharded Windows ARM64 runs were `55m01s`,
`50m21s`, and `46m42s`, so the completed cross-compile runs improved the
lane by roughly `12m` to `24m` versus the prior range
- latest corrected cross-compile run:
https://github.com/openai/codex/actions/runs/26120777976
- Windows ARM64 archive built successfully on Windows x64
- native Windows ARM64 shards started immediately after the archive
upload
- 3/4 Windows ARM64 shards passed; the failing shard hit the same
existing `code_mode` test failure seen outside this lane
- downloaded failed-shard JUnit XML from the validation runs and
confirmed the remaining red is from known test failures, not
archive/shard wiring
- no local Codex tests run per repo guidance
## Notes
- this PR does not change developers.openai.com documentation
## Summary
Move the rusty_v8 artifact production into hermetic Bazel path and bump
the `v8` crate to `147.4.0`
The new flow builds V8 release artifacts from source for Darwin and
Linux targets, publishes both the current release-compatible artifacts
and sandbox-enabled variants, and keeps Cargo consumers on prebuilt
binaries by continuing to feed the `v8` crate the archive and generated
binding files it already expects.
## Why
We need control over V8 build-time features without giving up prebuilt
artifacts for downstream Cargo builds.
Upstream `rusty_v8` already supports source-only features such as
`v8_enable_sandbox`, but its normal prebuilt release assets do not cover
every feature combination we need. Building the artifacts ourselves lets
us enable settings such as the V8 sandbox and pointer compression at
artifact build time, then publish those outputs so ordinary Cargo builds
can still consume prebuilts instead of compiling V8 locally.
This keeps the fast consumer experience of prebuilt `rusty_v8` archives
while giving us a reproducible path to ship featureful variants that
upstream does not currently publish for us.
## Implementation Notes
The Bazel graph in this PR is not copied wholesale from `rusty_v8`;
`rusty_v8`'s normal source build is still GN/Ninja-based.
Instead, this change starts from upstream V8's Bazel rules and adapts
them to Codex's hermetic toolchains and dependency layout. Where we
intentionally follow `rusty_v8`, we mirror its existing artifact
contract:
- the same `v8` crate version and generated binding expectations
- the same sandbox feature relationship, where sandboxing requires
pointer compression
- the same custom libc++ model expected by Cargo's default
`use_custom_libcxx` feature
- the same release-style archive plus `src_binding` outputs consumed by
the `v8` crate
To preserve that contract, the Bazel release path pins the libc++,
libc++abi, and llvm-libc revisions used by `rusty_v8 v147.4.0`, builds
release artifacts with `--config=rusty-v8-upstream-libcxx`, and folds
the matching runtime objects into the final static archive.
## Windows
Windows is annoyingly handled differently.
Codex's current hermetic Bazel Windows C++ platform is `windows-gnullvm`
/ `x86_64-w64-windows-gnu`, while upstream `rusty_v8` publishes Windows
prebuilts for `*-pc-windows-msvc`. Those are different ABIs, so the
Bazel graph cannot truthfully reproduce the upstream MSVC artifacts
until we add a real MSVC-targeting C++ toolchain.
For now:
- Windows MSVC consumers continue to use upstream `rusty_v8` release
archives.
- Windows GNU targets are built in-tree so they link against a matching
GNU ABI.
- The canary workflow separately exercises upstream `rusty_v8` source
builds for MSVC sandbox artifacts, but MSVC is not yet part of the
Bazel-produced release matrix.
## Validation
This PR is technically self validating through CI. I have already
published it as a release tag so the artifacts from this branch are
published to
https://github.com/openai/codex/releases/tag/rusty-v8-v147.4.0 CI for
this PR should therefore consume our own release targets. I have also
locally tested for linux and darwin.
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
This is the first PR in the V8 in-process sandboxing rollout.
It adds the build-system and Rust feature plumbing needed to support
sandboxed V8 builds, then enables sandboxing by default for the
source-built Bazel V8 path that we control directly. It deliberately
keeps the published `rusty_v8` artifact workflows on their current
non-sandboxed contract so this PR can land and ship independently before
we change any released artifacts.
## Rollout plan
- [x] **PR 1: land sandbox plumbing and default source-built Bazel V8 to
sandboxed mode**
- [ ] **PR 2: publish sandbox-enabled release artifacts and add
compatibility validation**
- Produce sandboxed artifact pairs for every released Cargo target that
does not already use the source-built Bazel path.
- Add CI coverage that consumes those sandboxed artifacts and verifies:
- `codex-v8-poc` reports sandbox enabled
- `codex-code-mode` builds/tests against the sandboxed path
- [ ] **PR 3: switch release consumers to sandboxed artifacts by
default**
- Update released artifact selectors/checksums.
- Enable the Rust `v8_enable_sandbox` feature in the default release
path.
- Make the sandboxed artifact family the normal path for published
builds.
- [ ] **PR 4: remove rollout-only compatibility paths**
- Remove the temporary non-sandbox release compatibility config once the
new default has shipped and baked.
- Keep the invariant tests permanently.
## Why
#20585 moved the Windows Bazel test job to the cross-compile path, but
the Windows Bazel clippy and verify-release-build jobs were still using
the native Windows/MSVC-host fallback. Those two jobs became the slowest
Windows PR legs, even though both are build-only signal and do not need
to execute the resulting binaries.
## What Changed
- Switches the Windows Bazel clippy job from
`--windows-msvc-host-platform` to `--windows-cross-compile`, so clippy
build actions use Linux RBE while still targeting
`x86_64-pc-windows-gnullvm`.
- Switches the Windows Bazel verify-release-build job to
`--windows-cross-compile` as well. This job only compiles
`cfg(not(debug_assertions))` Rust code under `fastbuild`, so it does not
need a native Windows build host.
- Keeps the old `--skip_incompatible_explicit_targets` behavior only for
fork/community PRs without `BUILDBUDDY_API_KEY`, where `run-bazel-ci.sh`
falls back to the local Windows MSVC-host shape.
- Adds `--windows-cross-compile` support to
`.github/scripts/run-bazel-query-ci.sh`, so target-discovery queries
select the same `ci-windows-cross` config as the subsequent build.
- Threads that option through `scripts/list-bazel-clippy-targets.sh` so
the Windows clippy job discovers targets under the same platform shape
as the subsequent clippy build.
## Verification
Local checks:
```shell
bash -n .github/scripts/run-bazel-query-ci.sh
bash -n scripts/list-bazel-clippy-targets.sh
ruby -e 'require "yaml"; YAML.load_file(".github/workflows/bazel.yml"); puts "ok"'
RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh | grep -c -- '-windows-cross-bin$'
RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh --windows-cross-compile | grep -c -- '-windows-cross-bin$'
```
The Linux target-list check reported `0` Windows-cross internal test
binaries, while the Windows cross target-list check reported `47`,
preserving the test-code clippy coverage shape from the existing Windows
job.
## Status
This is the Bazel PR-CI cross-compilation follow-up to #20485. It is
intentionally split from the Cargo/cargo-xwin release-build PoC so
#20485 can stay as the historical release-build exploration. The
unrelated async-utils test cleanup has been moved to #20686, so this PR
is focused on the Windows Bazel CI path.
The intended tradeoff is now explicit in `.github/workflows/bazel.yml`:
pull requests get the fast Windows cross-compiled Bazel test leg, while
post-merge pushes to `main` run both that fast cross leg and a fully
native Windows Bazel test leg. The native main-only job keeps full
V8/code-mode coverage and gets a 40-minute timeout because it is less
latency-sensitive than PR CI. All other Bazel jobs remain at 30 minutes.
## Why
Windows Bazel PR CI currently does the expensive part of the build on
Windows. A native Windows Bazel test job on `main` completed in about
28m12s, leaving very little headroom under the 30-minute job timeout and
making Windows the slowest PR signal.
#20485 showed that Windows cross-compilation can be materially faster
for Cargo release builds, but PR CI needs Bazel because Bazel owns our
test sharding, flaky-test retries, and integration-test layout. This PR
applies the same high-level shape we already use for macOS Bazel CI:
compile with remote Linux execution, then run platform-specific tests on
the platform runner.
The compromise is deliberately signal-aware: code-mode/V8 changes are
rare enough that PR CI can accept losing the direct V8/code-mode
smoke-test signal temporarily, while `main` still runs the native
Windows job post-merge to catch that class of regression. A follow-up PR
should investigate making the cross-built Windows gnullvm V8 archive
pass the direct V8/code-mode tests so this tradeoff can eventually go
away.
## What Changed
- Adds a `ci-windows-cross` Bazel config that targets
`x86_64-pc-windows-gnullvm`, uses Linux RBE for build actions, and keeps
`TestRunner` actions local on the Windows runner.
- Adds explicit Windows platform definitions for
`windows_x86_64_gnullvm`, `windows_x86_64_msvc`, and a bridge toolchain
that lets gnullvm test targets execute under the Windows MSVC host
platform.
- Updates the Windows Bazel PR test leg to opt into the cross-compile
path via `--windows-cross-compile` and `--remote-download-toplevel`.
- Adds a `test-windows-native-main` job that runs only for `push` events
on `refs/heads/main`, uses the native Windows Bazel path, includes
V8/code-mode smoke tests, and has `timeout-minutes: 40`.
- Keeps fork/community PRs without `BUILDBUDDY_API_KEY` on the previous
local Windows MSVC-host fallback, including
`--host_platform=//:local_windows_msvc` and `--jobs=8`.
- Preserves the existing integration-test shape on non-gnullvm
platforms, while generating Windows-cross wrapper targets only for
`windows_gnullvm`.
- Resolves `CARGO_BIN_EXE_*` values from runfiles at test runtime,
avoiding hard-coded Cargo paths and duplicate test runfiles.
- Extends the V8 Bazel patches enough for the
`x86_64-pc-windows-gnullvm` target and Linux remote execution path.
- Makes the Windows sandbox test cwd derive from `INSTA_WORKSPACE_ROOT`
at runtime when Bazel provides it, because cross-compiled binaries may
contain Linux compile-time paths.
- Keeps the direct V8/code-mode unit smoke tests out of the Windows
cross PR path for now while native Windows CI continues to cover them
post-merge.
## Command Shape
The fast Windows PR test leg invokes the normal Bazel CI wrapper like
this:
```shell
./.github/scripts/run-bazel-ci.sh \
--print-failed-action-summary \
--print-failed-test-logs \
--windows-cross-compile \
--remote-download-toplevel \
-- \
test \
--test_tag_filters=-argument-comment-lint \
--test_verbose_timeout_warnings \
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
-- \
//... \
-//third_party/v8:all \
-//codex-rs/code-mode:code-mode-unit-tests \
-//codex-rs/v8-poc:v8-poc-unit-tests
```
With the BuildBuddy secret available on Windows, the wrapper selects
`--config=ci-windows-cross` and appends the important Windows-cross
overrides after rc expansion:
```shell
--host_platform=//:rbe
--shell_executable=/bin/bash
--action_env=PATH=/usr/bin:/bin
--host_action_env=PATH=/usr/bin:/bin
--test_env=PATH=${CODEX_BAZEL_WINDOWS_PATH}
```
The native post-merge Windows job intentionally omits
`--windows-cross-compile` and does not exclude the V8/code-mode unit
targets:
```shell
./.github/scripts/run-bazel-ci.sh \
--print-failed-action-summary \
--print-failed-test-logs \
-- \
test \
--test_tag_filters=-argument-comment-lint \
--test_verbose_timeout_warnings \
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
--build_metadata=TAG_windows_native_main=true \
-- \
//... \
-//third_party/v8:all
```
## Research Notes
The existing macOS Bazel CI config already uses the model we want here:
build actions run remotely with `--strategy=remote`, but `TestRunner`
actions execute on the macOS runner. This PR mirrors that pattern for
Windows with `--strategy=TestRunner=local`.
The important Bazel detail is that `rules_rs` is already targeting
`x86_64-pc-windows-gnullvm` for Windows Bazel PR tests. This PR changes
where the build actions execute; it does not switch the Bazel PR test
target to Cargo, `cargo-nextest`, or the MSVC release target.
Cargo release builds differ from this Bazel path for V8: the normal
Windows Cargo release target is MSVC, and `rusty_v8` publishes prebuilt
Windows MSVC `.lib.gz` archives. The Bazel PR path targets
`windows-gnullvm`; `rusty_v8` does not publish a prebuilt Windows
GNU/gnullvm archive, so this PR builds that archive in-tree. That
Linux-RBE-built gnullvm archive currently crashes in direct V8/code-mode
smoke tests, which is why the workflow keeps native Windows coverage on
`main`.
The less obvious Bazel detail is test wrapper selection. Bazel chooses
the Windows test wrapper (`tw.exe`) from the test action execution
platform, not merely from the Rust target triple. The outer
`workspace_root_test` therefore declares the default test toolchain and
uses the bridge toolchain above so the test action executes on Windows
while its inner Rust binary is built for gnullvm.
The V8 investigation exposed a Windows-client gotcha: even when an
action execution platform is Linux RBE, Bazel can still derive the
genrule shell path from the Windows client. That produced remote
commands trying to run `C:\Program Files\Git\usr\bin\bash.exe` on Linux
workers. The wrapper now passes `--shell_executable=/bin/bash` with
`--host_platform=//:rbe` for the Windows cross path.
The same Windows-client/Linux-RBE boundary also affected
`third_party/v8:binding_cc`: a multiline genrule command can carry CRLF
line endings into Linux remote bash, which failed as `$'\r'`. That
genrule now keeps the `sed` command on one physical shell line while
using an explicit Starlark join so the shell arguments stay readable.
## Verification
Local checks included:
```shell
bash -n .github/scripts/run-bazel-ci.sh
bash -n workspace_root_test_launcher.sh.tpl
ruby -e "require %q{yaml}; YAML.load_file(%q{.github/workflows/bazel.yml}); puts %q{ok}"
RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh
RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh
RUNNER_OS=Linux ./tools/argument-comment-lint/list-bazel-targets.sh
RUNNER_OS=Windows ./tools/argument-comment-lint/list-bazel-targets.sh
```
The Linux clippy and argument-comment target lists contain zero
`*-windows-cross-bin` labels, while the Windows lists still include 47
Windows-cross internal test binaries.
CI evidence:
- Baseline native Windows Bazel test on `main`: success in about 28m12s,
https://github.com/openai/codex/actions/runs/25206257208/job/73907325959
- Green Windows-cross Bazel run on the split PR before adding the
main-only native leg: Windows test 9m16s, Windows release verify 5m10s,
Windows clippy 4m43s,
https://github.com/openai/codex/actions/runs/25231890068
- The latest SHA adds the explicit PR-vs-main tradeoff in `bazel.yml`;
CI is rerunning on that focused diff.
## Follow-Up
A subsequent PR should investigate making a cross-built Windows binary
work with V8/code-mode enabled. Likely options are either making the
Linux-RBE-built `windows-gnullvm` V8 archive correct at runtime, or
evaluating whether a Bazel MSVC target/toolchain can reuse the same
prebuilt MSVC `rusty_v8` archive shape that Cargo release builds already
use.
## Why
A rerun of the Windows Bazel clippy job after
[#19161](https://github.com/openai/codex/pull/19161) had exactly the
cache behavior we wanted in BuildBuddy: zero action-cache misses. Even
so, the GitHub job still took a little over five minutes.
The problem was that the job was paying for two separate Bazel startup
paths:
1. a `bazel query` to discover extra lint targets
2. the real `bazel build --config=clippy ...` invocation
On Windows, that query was bypassing the CI Bazel wrapper, so it did not
reuse the same `--output_user_root`, CI config, or remote-cache setup as
the real build. In practice that meant the rerun could still cold-start
a separate Bazel server before the actual clippy build even began.
## What
- add `.github/scripts/run-bazel-query-ci.sh` to run CI-side Bazel
queries with the same startup and cache-related flags as the main Bazel
command
- switch `scripts/list-bazel-clippy-targets.sh` to use that helper for
manual `rust_test` target discovery
- switch `tools/argument-comment-lint/list-bazel-targets.sh` to use the
same helper
- simplify `.github/scripts/run-argument-comment-lint-bazel.sh` so its
Windows-only query path also goes through the shared helper
This keeps the target-discovery queries aligned with the later
build/test invocation instead of treating them as a separate cold Bazel
session.
## Verification
- `bash -n .github/scripts/run-bazel-query-ci.sh`
- `bash -n scripts/list-bazel-clippy-targets.sh`
- `bash -n tools/argument-comment-lint/list-bazel-targets.sh`
- `bash -n .github/scripts/run-argument-comment-lint-bazel.sh`
- mocked a Windows invocation of `run-bazel-query-ci.sh` and verified it
forwards `--output_user_root`, `--config=ci-windows`, the BuildBuddy
auth header, and the repository cache flags
## Docs
No documentation updates are needed.
## Why
The BuildBuddy runs for PR #19086 and the later `main` build had the
same source tree, but their Windows Bazel action and test cache keys did
not line up. Comparing the downloaded execution logs showed the full
GitHub-hosted Windows runner `PATH` had changed from
`apache-maven-3.9.14` to `apache-maven-3.9.15`.
This repo is not using Maven; the Maven entry was just ambient
hosted-runner state. The problem was that Windows Bazel CI was still
forwarding the whole runner `PATH` into Bazel via `--action_env=PATH`,
`--host_action_env=PATH`, and `--test_env=PATH`, which made otherwise
reusable cache entries sensitive to unrelated runner image churn.
After discussion with the Bazel and BuildBuddy folks, the better shape
for this change was to stop asking Bazel to inherit the ambient Windows
`PATH` and instead compute one explicit cache-stable `PATH` in the
Windows setup action that already prepares the CI toolchain environment.
## What
- remove Windows `PATH` passthrough from `.bazelrc`
- export `CODEX_BAZEL_WINDOWS_PATH` from
`.github/actions/setup-bazel-ci/action.yml`
- move the PATH derivation logic into
`.github/scripts/compute-bazel-windows-path.ps1` so the allow-list is
easier to review and document
- keep only the Windows tool locations these Bazel jobs actually need:
MSVC and SDK paths, Git, PowerShell, Node, DotSlash, and the standard
Windows system directories
- update `.github/scripts/run-bazel-ci.sh` to require that explicit
value and forward it to Bazel action, host action, and test environments
- log the derived `CODEX_BAZEL_WINDOWS_PATH` in the setup step to
simplify cache-key debugging
## Verification
- `bash -n .github/scripts/run-bazel-ci.sh`
- `ruby -e 'require "yaml"; YAML.load_file(ARGV[0])'
.github/actions/setup-bazel-ci/action.yml`
- PowerShell parse check for
`.github/scripts/compute-bazel-windows-path.ps1`
- simulated a representative Windows `PATH` in PowerShell; the
allow-list retained MSVC, Git, PowerShell, Node, Windows, and DotSlash
entries while dropping Maven
## Why
Unused imports in `core/tests/suite/unified_exec.rs` in the Windows
build were not caught by Bazel CI on
https://github.com/openai/codex/pull/18096. I spot-checked
https://github.com/openai/codex/actions/workflows/rust-ci-full.yml?query=branch%3Amain
and noticed that builds were consistently red. This revealed that our
Cargo builds _were_ properly catching these issues, identifying a
Windows-specific coverage hole in the Bazel clippy job.
The Windows Bazel clippy job uses `--skip_incompatible_explicit_targets`
so it can lint a broad target set without failing immediately on targets
that are genuinely incompatible with Windows. However, with the default
Windows host platform, `rust_test` targets such as
`//codex-rs/core:core-all-test` could be skipped before the clippy
aspect reached their integration-test modules. As a result, the imports
in `core/tests/suite/unified_exec.rs` were not being linted by the
Windows Bazel clippy job at all.
The clippy diagnostic that Windows Bazel should have surfaced was:
```text
error: unused import: `codex_config::Constrained`
--> core\tests\suite\unified_exec.rs:8:5
|
8 | use codex_config::Constrained;
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
error: unused import: `codex_protocol::permissions::FileSystemAccessMode`
--> core\tests\suite\unified_exec.rs:11:5
|
11 | use codex_protocol::permissions::FileSystemAccessMode;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemPath`
--> core\tests\suite\unified_exec.rs:12:5
|
12 | use codex_protocol::permissions::FileSystemPath;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemSandboxEntry`
--> core\tests\suite\unified_exec.rs:13:5
|
13 | use codex_protocol::permissions::FileSystemSandboxEntry;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemSandboxPolicy`
--> core\tests\suite\unified_exec.rs:14:5
|
14 | use codex_protocol::permissions::FileSystemSandboxPolicy;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
## What changed
- Run the Windows Bazel clippy job with the MSVC host platform via
`--windows-msvc-host-platform`, matching the Windows Bazel test job.
This keeps `--skip_incompatible_explicit_targets` while ensuring Windows
`rust_test` targets such as `//codex-rs/core:core-all-test` are still
linted.
- Remove the unused imports from `core/tests/suite/unified_exec.rs`.
- Add `--print-failed-action-summary` to
`.github/scripts/run-bazel-ci.sh` so Bazel action failures can be
summarized after the build exits.
## Failure reporting
Once the coverage issue was fixed, an intentionally reintroduced unused
import made the Windows Bazel clippy job fail as expected. That exposed
a separate usability problem: because the job keeps `--keep_going`, the
top-level Bazel output could still end with:
```text
ERROR: Build did NOT complete successfully
FAILED:
```
without the underlying rustc/clippy diagnostic being visible in the
obvious part of the GitHub Actions log.
To keep `--keep_going` while making failures actionable, the wrapper now
scans the captured Bazel console output for failed actions and prints
the matching rustc/clippy diagnostic block. When a diagnostic block is
found, it is emitted both as a GitHub `::error` annotation and as plain
expanded log output, rather than being hidden in a collapsed group.
## Verification
To validate the CI path, I intentionally introduced an unused import in
`core/tests/suite/unified_exec.rs`. The Windows Bazel clippy job failed
as expected, confirming that the integration-test module is now covered
by Bazel clippy. The same failure also verified that the wrapper
surfaces the matching clippy diagnostics directly in the Actions output.
## Summary
- cap the Windows Bazel test lane at `--jobs=8` to reduce local runner
pressure
- keep Linux and macOS Bazel test concurrency unchanged
- make failed-test log tailing resolve `bazel-testlogs` with the same CI
config and Windows host-platform context as the failed invocation
- prefer Bazel-reported `test.log` paths and normalize Windows path
separators before tailing
## Context
The Windows Bazel workflow currently uses `ci-windows`, which does not
inherit the remote executor config. This means the lane runs the `//...`
test suite locally and otherwise falls back to the repo-wide `common
--jobs=30`. The new Windows-only override is intended to reduce local
executor pressure without changing coverage.
## Validation
Not run locally; this is a CI workflow change and the draft PR is
intended to exercise the GitHub Actions lane directly.
---------
Co-authored-by: Codex <noreply@openai.com>
Make sure Bazel logs shows every errors so that we can debug flakes +
fix a small flake on Windows by updating the sleep command to a
`Start-Sleep` instead of a PowerShell nested command (otherwise we had
double nesting which is absurdely slow)
## Summary
- Pin Rust git patch dependencies to immutable revisions and make
cargo-deny reject unknown git and registry sources unless explicitly
allowlisted.
- Add checked-in SHA-256 coverage for the current rusty_v8 release
assets, wire those hashes into Bazel, and verify CI override downloads
before use.
- Add rusty_v8 MODULE.bazel update/check tooling plus a Bazel CI guard
so future V8 bumps cannot drift from the checked-in checksum manifest.
- Pin release/lint cargo installs and all external GitHub Actions refs
to immutable inputs.
## Future V8 bump flow
Run these after updating the resolved `v8` crate version and checksum
manifest:
```bash
python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
```
The update command rewrites the matching `rusty_v8_<crate_version>`
`http_file` SHA-256 values in `MODULE.bazel` from
`third_party/v8/rusty_v8_<crate_version>.sha256`. The check command is
also wired into Bazel CI to block drift.
## Notes
- This intentionally excludes RustSec dependency upgrades and
bubblewrap-related changes per request.
- The branch was rebased onto the latest origin/main before opening the
PR.
## Validation
- cargo fetch --locked
- cargo deny check advisories
- cargo deny check
- cargo deny check sources
- python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
- python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
- python3 -m unittest discover -s .github/scripts -p
'test_rusty_v8_bazel.py'
- python3 -m py_compile .github/scripts/rusty_v8_bazel.py
.github/scripts/rusty_v8_module_bazel.py
.github/scripts/test_rusty_v8_bazel.py
- repo-wide GitHub Actions `uses:` audit: all external action refs are
pinned to 40-character SHAs
- yq eval on touched workflows and local actions
- git diff --check
- just bazel-lock-check
## Hash verification
- Confirmed `MODULE.bazel` hashes match
`third_party/v8/rusty_v8_146_4_0.sha256`.
- Confirmed GitHub release asset digests for denoland/rusty_v8
`v146.4.0` and openai/codex `rusty-v8-v146.4.0` match the checked-in
hashes.
- Streamed and SHA-256 hashed all 10 `MODULE.bazel` rusty_v8 asset URLs
locally; every downloaded byte stream matched both `MODULE.bazel` and
the checked-in manifest.
## Pin verification
- Confirmed signing-action pins match the peeled commits for their tag
comments: `sigstore/cosign-installer@v3.7.0`, `azure/login@v2`, and
`azure/trusted-signing-action@v0`.
- Pinned the remaining tag-based action refs in Bazel CI/setup:
`actions/setup-node@v6`, `facebook/install-dotslash@v2`,
`bazelbuild/setup-bazelisk@v3`, and `actions/cache/restore@v5`.
- Normalized all `bazelbuild/setup-bazelisk@v3` refs to the peeled
commit behind the annotated tag.
- Audited Cargo git dependencies: every manifest git dependency uses
`rev` only, every `Cargo.lock` git source has `?rev=<sha>#<same-sha>`,
and `cargo deny check sources` passes with `required-git-spec = "rev"`.
- Shallow-fetched each distinct git dependency repo at its pinned SHA
and verified Git reports each object as a commit.
