mirror of
https://github.com/openai/codex.git
synced 2026-10-23 18:08:45 +00:00
## Why V8's Abseil and protobuf's shared Abseil can be built for different C++ runtimes. Give V8's copy a distinct inline namespace so both can link into one binary without interposing incompatible functions. ## What changed - Move pinned V8 dependency imports and BUILD overlays into `third_party/v8`, and inject those repositories into V8 through a module-extension bridge with no fallback downloads. - Build V8's Abseil with a V8-specific inline namespace while preserving protobuf's shared Abseil namespace. - Apply V8's C++ runtime settings in the Abseil, Highway, and simdutf overlays, and remove the shared `rules_cc` patch. - Document source dependency ownership and the version, checksum, and overlay review procedure for V8 upgrades. GitOrigin-RevId: 2b8e3deed285559f8dd3324647cb4b81089f2f56
220 lines
11 KiB
Markdown
220 lines
11 KiB
Markdown
# `rusty_v8` Consumer Artifacts
|
|
|
|
This directory wires the `v8` crate to exact-version Bazel inputs.
|
|
Bazel consumer builds use:
|
|
|
|
- Codex-published sandbox archive/binding pairs on Darwin, GNU/musl Linux,
|
|
and Windows MSVC (x64 and arm64), with checksums pinned from the trusted
|
|
release manifests
|
|
- source-built V8 archives on Windows GNU
|
|
|
|
Local Cargo builds still use upstream prebuilt `rusty_v8` archives by default.
|
|
Selected Cargo CI, release, and package builds override
|
|
`RUSTY_V8_ARCHIVE`/`RUSTY_V8_SRC_BINDING_PATH` with Codex release assets. Bazel
|
|
sets those variables independently in `MODULE.bazel`, selecting the pair above
|
|
for its consumers. All Bazel compilation modes use the same published V8
|
|
release archive on supported platforms.
|
|
|
|
The Bazel `v8` crate feature selection enables V8's in-process sandbox.
|
|
Darwin/Linux consumers select prebuilts only when both the V8 sandbox and pointer
|
|
compression settings match the published artifact. For source instrumentation
|
|
or custom V8 C++ settings, use `--//:rusty_v8_from_source=true`; the archive
|
|
and binding then both come from the source path. The published release archive
|
|
cannot incorporate local V8 C++ flags or native debug/sanitizer settings.
|
|
|
|
Current pinned versions:
|
|
|
|
- Rust crate: `v8 = =152.2.0`
|
|
- Embedded upstream V8 source for Bazel-produced release builds: `15.2.124.1`
|
|
|
|
## Updating to a new `v8` release
|
|
|
|
Use this as the maintainer flow for a version bump:
|
|
|
|
1. Bump the `v8` crate version and refresh `codex-rs/Cargo.lock`.
|
|
2. Update the Bazel versioned inputs in `MODULE.bazel` and the source pins
|
|
in `third_party/v8/dependencies.bzl` and `abseil_repository.bzl` using
|
|
[the source dependency update procedure](#updating-source-dependencies).
|
|
Then refresh the matching checksum manifest and generated checksums below.
|
|
3. Publish a release-candidate PR and validate that `v8-canary` passes.
|
|
4. If the canary is green, publish the release tag and release build.
|
|
5. Independently verify the published Codex-built checksum manifests and record
|
|
their SHA-256 digests in
|
|
`third_party/v8/rusty_v8_<version>_release_manifests.sha256`.
|
|
6. Once the release build completes, rerun the build on the candidate branch
|
|
and verify that the final artifact builds and tests pass.
|
|
|
|
When changing the prebuilt `rusty_v8` `http_file` inputs, keep the
|
|
checked-in checksum manifest and `MODULE.bazel` in sync:
|
|
|
|
```bash
|
|
python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
|
|
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
|
|
```
|
|
|
|
For all eight published pairs, verify each published
|
|
`rusty_v8_ptrcomp_sandbox_release_<target>.sha256` against the committed
|
|
`rusty_v8_<version>_release_manifests.sha256` first. Copy the verified archive
|
|
and binding checksums into `rusty_v8_<version>.sha256`, then run these
|
|
commands. They validate every matching `http_file` entry, and CI blocks
|
|
checksum drift.
|
|
|
|
The consumer-facing selectors are:
|
|
|
|
- `//third_party/v8:rusty_v8_archive_for_target`
|
|
- `//third_party/v8:rusty_v8_binding_for_target`
|
|
|
|
Published release assets are expected at the tag:
|
|
|
|
- `rusty-v8-v<crate_version>`
|
|
|
|
with these raw asset names:
|
|
|
|
- `librusty_v8_release_<target>.a.gz`
|
|
- `src_binding_release_<target>.rs`
|
|
|
|
During the sandbox rollout, sandbox-enabled assets are published alongside those
|
|
current assets on the same tag, with the Rust crate's sandbox feature suffix in
|
|
their raw names:
|
|
|
|
- `librusty_v8_ptrcomp_sandbox_release_<target>.a.gz`
|
|
- `rusty_v8_ptrcomp_sandbox_release_<target>.lib.gz` on Windows MSVC
|
|
- `src_binding_ptrcomp_sandbox_release_<target>.rs`
|
|
|
|
The dedicated publishing workflow is `.github/workflows/rusty-v8-release.yml`.
|
|
Tagged runs build release artifacts from the Bazel graph itself:
|
|
|
|
- `//third_party/v8:rusty_v8_release_pair_x86_64_apple_darwin`
|
|
- `//third_party/v8:rusty_v8_release_pair_aarch64_apple_darwin`
|
|
- `//third_party/v8:rusty_v8_release_pair_x86_64_unknown_linux_gnu`
|
|
- `//third_party/v8:rusty_v8_release_pair_aarch64_unknown_linux_gnu`
|
|
- `//third_party/v8:rusty_v8_release_pair_x86_64_unknown_linux_musl`
|
|
- `//third_party/v8:rusty_v8_release_pair_aarch64_unknown_linux_musl`
|
|
|
|
The same run also builds the matching sandbox pair targets:
|
|
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_x86_64_apple_darwin`
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_aarch64_apple_darwin`
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_x86_64_unknown_linux_gnu`
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_aarch64_unknown_linux_gnu`
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_x86_64_unknown_linux_musl`
|
|
- `//third_party/v8:rusty_v8_sandbox_release_pair_aarch64_unknown_linux_musl`
|
|
|
|
The workflow also builds sandbox-enabled
|
|
`x86_64-pc-windows-msvc` and `aarch64-pc-windows-msvc` archive/binding pairs
|
|
from upstream `rusty_v8` source. Those ABI-specific outputs cannot be produced
|
|
by Codex's Bazel Windows GNU toolchain.
|
|
|
|
The Bazel graph pins the same libc++, libc++abi, and llvm-libc source revisions
|
|
used by `rusty_v8 v152.2.0`, compiles published artifact targets with
|
|
`--config=rusty-v8-upstream-libcxx`, and folds the matching runtime objects into
|
|
the final static archive so consumers can link it with the `v8` crate's default
|
|
`use_custom_libcxx` feature. The config keeps the object files and the bundled
|
|
runtime on Chromium's `std::__Cr` ABI namespace instead of mixing those objects
|
|
with the toolchain libc++ default namespace. Bazel consumers use these
|
|
published archives for supported Darwin/Linux platforms, as do Cargo release and
|
|
package builds. On Linux, Bazel decompresses to a private output and weakens
|
|
the ten shared `std::logic_error` and `std::runtime_error` constructors and
|
|
assignment entry points listed in `gnu_libcxx_shared_exception_symbols.txt`.
|
|
libc++ keeps these exception functions in `std::` across inline ABI namespaces;
|
|
the toolchain's definitions then take precedence if both runtimes are linked.
|
|
Constructors accepting Chromium's `std::__Cr::string` remain strong in the V8
|
|
archive. The verified input and the producer's archive are never mutated.
|
|
This depends on libc++'s shared exception-object ABI; updates to either libc++
|
|
revision must keep the native GNU link/runtime checks passing. The Bazel pair targets above still point directly to source
|
|
targets so a new release or canary never depends on an older published copy.
|
|
|
|
MSVC is not part of the Bazel-produced matrix yet. The repository's current
|
|
hermetic Windows C++ platform is `windows-gnullvm`/`x86_64-w64-windows-gnu`, so
|
|
it cannot truthfully reproduce upstream's `*-pc-windows-msvc` archives until we
|
|
add a real MSVC-targeting C++ toolchain to the Bazel graph.
|
|
|
|
Release and CI Cargo builds for Darwin and Linux use `RUSTY_V8_ARCHIVE` plus a
|
|
downloaded `RUSTY_V8_SRC_BINDING_PATH` to point at those `openai/codex` release
|
|
assets directly. We do not use `RUSTY_V8_MIRROR` because the upstream `v8` crate
|
|
hardcodes a `v<crate_version>` tag layout, while our artifacts are published
|
|
under `rusty-v8-v<crate_version>`.
|
|
|
|
Do not mix artifacts across crate versions. The archive and binding must match
|
|
the exact resolved `v8` crate version in `codex-rs/Cargo.lock`.
|
|
|
|
## Source dependency ownership
|
|
|
|
The Darwin, Linux GNU/musl and Windows GNU source targets consume Codex-owned
|
|
Abseil, Highway, simdutf, ICU, fast_float, dragonbox and fp16 repositories.
|
|
`dependencies.bzl` declares the pinned imports; the `*.BUILD.bazel` overlays
|
|
here declare the targets V8 uses. Abseil, Highway and simdutf apply their C++ runtime
|
|
settings through `dependency_config.bzl`; ICU defines its settings in its own
|
|
overlay.
|
|
|
|
V8's module has a small dependency ingress. The root `MODULE.bazel` injects
|
|
the seven Codex repositories there, so V8's `@highway`, `@icu`, etc. resolve to
|
|
the same repositories Codex declares. The bridge has no fallback downloads.
|
|
V8's BUILD patch still wires these targets into its generator and library graph.
|
|
|
|
V8's private Abseil is built from the pinned 20250814.1 sources and uses
|
|
`codex_v8_lts_20250814` for its supported inline-namespace option. Protobuf
|
|
keeps the regular shared Abseil module and its `lts_20250814` namespace.
|
|
Both can link into one binary without interposing functions built for different
|
|
C++ runtimes. No V8 dependency needs a hook in shared `rules_cc` libraries.
|
|
|
|
### Updating source dependencies
|
|
|
|
These are reviewed with each V8 source upgrade. There is no source-pin
|
|
autoupdater: the prebuilt checksum commands above and their CI check validate
|
|
published archive/binding pairs, not these source revisions. Bazel verifies
|
|
the declared download hashes and Git revisions; it does not choose them for us.
|
|
|
|
Use the V8 tag in the root `MODULE.bazel` `archive_override(module_name = "v8")`
|
|
(currently `15.2.124.1`) as the input. Read that tag's **unpatched** `DEPS`
|
|
and `MODULE.bazel`; the fetched `@v8` MODULE has Codex's bridge applied.
|
|
|
|
```bash
|
|
v8_tag=15.2.124.1 # Replace with the V8 source tag selected for this upgrade.
|
|
gh api "repos/v8/v8/contents/DEPS?ref=$v8_tag" \
|
|
-H 'Accept: application/vnd.github.raw+json' |
|
|
rg -n -A 2 "'third_party/(highway/src|icu|dragonbox/src|fp16/src|fast_float/src|simdutf)'"
|
|
gh api "repos/v8/v8/contents/MODULE.bazel?ref=$v8_tag" \
|
|
-H 'Accept: application/vnd.github.raw+json' | rg 'bazel_dep\(name = "abseil-cpp"'
|
|
```
|
|
|
|
- `v8_highway`, `v8_icu`, `v8_dragonbox`, `v8_fp16`: copy the respective
|
|
DEPS commits. For Highway, use that commit in the archive URL and strip prefix.
|
|
- `v8_fast_float`: use the DEPS commit. If selecting a release-tag archive,
|
|
verify the tag resolves to that commit, e.g.
|
|
`gh api repos/fastfloat/fast_float/git/ref/tags/v8.2.10 --jq '.object'`
|
|
(dereference annotated tags). Current v8.2.10 equals DEPS commit `34164f5`.
|
|
- `v8_simdutf`: DEPS points at a Chromium vendoring repo, not the simdutf
|
|
source repo. Read the metadata at that DEPS commit:
|
|
|
|
```bash
|
|
simdutf_chromium=f7356eed293f8208c40b3c1b344a50bd70971983 # from new DEPS
|
|
curl -fsSL "https://chromium.googlesource.com/chromium/src/third_party/simdutf/+/$simdutf_chromium/README.chromium?format=TEXT" |
|
|
base64 -d
|
|
```
|
|
|
|
Match its `Version:` to the upstream release tag and inspect Chromium's
|
|
local changes before accepting the source form. Chromium vendors amalgamated
|
|
files; we build upstream `src/simdutf.cpp`. At the current pin it reports
|
|
7.7.0, but its `Revision: da645ece` predates the v7.7.0 tag. This is a
|
|
version match, not an assertion that the repositories' commits or bytes match.
|
|
- `v8_abseil`: follow `abseil-cpp` in V8's unpatched MODULE, not its Chromium
|
|
DEPS vendor revision. Refresh `abseil_repository.bzl` release archive and
|
|
its guarded inline namespace together; the V8 namespace must stay private.
|
|
|
|
For each `http_archive`/`download_and_extract` source, hash the exact new
|
|
URL, not a different archive of the same commit:
|
|
|
|
```bash
|
|
source_url='https://github.com/google/highway/archive/2607d3b5b0113992fe84d3848859eae13b3b52c1.tar.gz'
|
|
curl -fsSL "$source_url" | shasum -a 256
|
|
# For an integrity= field (Abseil) instead:
|
|
curl -fsSL "$source_url" | openssl dgst -sha256 -binary | openssl base64 -A
|
|
```
|
|
|
|
Record the hex digest in `sha256`, or prefix the base64 digest with `sha256-`
|
|
for `integrity`. Review each matching BUILD overlay against its stated source,
|
|
then build the V8 generator and source/archive targets on supported platforms.
|
|
The regular V8 canary/release checks validate the resulting engine; they do not
|
|
prove upstream-version equivalence for a substituted source.
|