Split the code-mode protocol and client from the V8-backed runtime so core and
app-server no longer link codex-code-mode in production. ThreadManager now
provisions durable code-mode sessions through a shared external host process,
while tests can still inject the in-process provider.
The IPC protocol uses a persistent stdin/stdout transport. Each frame is a
4-byte big-endian length followed by JSON, with a 16 MiB frame limit. Client
requests carry u64 request IDs so create, execute, wait, terminate, and shutdown
operations can be multiplexed over one process. Session IDs isolate durable
stored values. Execute returns an ExecutionStarted response immediately and an
asynchronous InitialResponse when the initial yield or completion is available.
Nested tool calls and notifications travel from the host back to the client as
delegate requests with their own IDs. Delegate responses, cancellation, and
cell-closed lifecycle messages use the same framed channel. Wire operations
encode errors as Result values. A dead connection fails pending operations,
cancels outstanding delegates, and lets the provider spawn a new host for later
sessions.
Build codex-code-mode-host with V8 pointer-compression sandbox support and add
it to canonical primary and app-server packages, legacy Linux and Windows
bundles, signing verification, installers, Python runtime packages, and release
CI for macOS, Linux, and Windows. The host is discovered next to the current
executable, through CODEX_CODE_MODE_HOST_PATH, or on PATH. OS-level seccomp or
seatbelt restrictions remain a follow-up to this cross-platform process split.
Benchmarks were run from release builds on Linux x86_64 with the V8 sandbox
profile and a text('ok') workload. Cold measurements used 30 samples, warm
session provisioning used 200, and warm command execution used 500. Values are
mean/p50/p95 in milliseconds:
- session startup: in-process 0.002/0.002/0.005, IPC 2.623/2.599/2.894
- fresh-session command: in-process 1.831/1.758/1.915, IPC 7.428/7.252/8.306
- warm session provisioning: in-process 0.002/0.002/0.003,
IPC 0.471/0.463/0.581
- warm command: in-process 1.759/1.757/1.940, IPC 2.005/2.001/2.166
The steady-state median command overhead is approximately 0.244 ms. The median
fresh host plus first command cost is 7.252 ms.
Validation:
- 62/62 core code-mode integration tests passed against the external host
- focused protocol, client, runtime, host, tools, and trace tests passed
- Cargo and Bazel real-process host IPC tests passed
- 11/11 package builder tests passed
- Bazel lock verification, scoped Clippy fixes, and repository formatting passed
## Why
The package layout gives Codex a stable place for runtime helpers that
should travel with the entrypoint. `shell_zsh_fork` still required users
to configure `zsh_path` manually, even though we already publish
prebuilt zsh fork artifacts.
This PR builds on #24129 and uses the shared DotSlash artifact fetcher
to include the zsh fork in Codex packages when a matching target
artifact exists. Packaged Codex builds can then discover the bundled
fork automatically; the user/profile `zsh_path` override is removed so
the feature uses the package-managed artifact instead of a legacy path
knob.
## What Changed
- Added `scripts/codex_package/codex-zsh`, a checked-in DotSlash
manifest for the current macOS arm64 and Linux zsh fork artifacts.
- Taught `scripts/build_codex_package.py` to fetch the matching zsh fork
artifact and install it at `codex-resources/zsh/bin/zsh` when available
for the selected target.
- Added package layout validation for the optional bundled zsh resource.
- Added `InstallContext::bundled_zsh_path()` and
`InstallContext::bundled_zsh_bin_dir()` for package-layout resource
discovery.
- Threaded the packaged zsh path through config loading as the runtime
`zsh_path` for packaged installs, and removed the config/profile/CLI
override path.
- Kept the packaged default zsh override typed as `AbsolutePathBuf`
until the existing runtime `Config::zsh_path` boundary.
- Updated app-server zsh-fork integration tests to spawn
`codex-app-server` from a temporary package layout with
`codex-resources/zsh/bin/zsh`, matching the new packaged discovery path
instead of setting `zsh_path` in config.
- Switched package executable copying from metadata-preserving `copy2()`
to `copyfile()` plus explicit executable bits, which avoids macOS
file-flag failures when local smoke tests use system binaries as inputs.
## Testing
To verify that the `zsh` executable from the Codex package is picked up
correctly, first I ran:
```shell
./scripts/build_codex_package.py
```
which created:
```
/private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/
```
so then I ran:
```
/private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/bin/codex exec --enable shell_zsh_fork 'run `echo $0`'
```
which reported the following, as expected:
```
/private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/codex-resources/zsh/bin/zsh
```
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23756).
* #23768
* __->__ #23756
## Why
The package builder should describe the binaries it is actually
packaging, not require callers to restate release metadata out of band.
A caller-provided `--version` flag can drift from the workspace version,
but running the target entrypoint to discover its version breaks
cross-target packages when the produced binary cannot execute on the
build host.
This PR keeps package metadata tied to the repository source of truth by
reading `[workspace.package].version` from `codex-rs/Cargo.toml`. It
also prepares the package layout for `codex-app-server` packages: the
same package structure can now represent either the CLI entrypoint or
the app-server entrypoint while keeping shared sidecars such as `rg`,
`bwrap`, and Windows sandbox helpers in the existing package
directories.
## What changed
- Removes the `--version` CLI flag from
`scripts/build_codex_package.py`.
- Adds Cargo.toml version discovery for `codex-package.json.version` via
`codex-rs/Cargo.toml`.
- Adds `--entrypoint-bin` so callers can package a prebuilt entrypoint
instead of rebuilding it with Cargo.
- Makes `--variant` an explicit choice between `codex` and
`codex-app-server`, and uses it to select the cargo binary and packaged
`bin/` entrypoint name.
- Updates `scripts/codex_package/README.md` to document variants,
prebuilt entrypoints, and Cargo.toml version detection.
## Verification
- Compiled `scripts/build_codex_package.py` and
`scripts/codex_package/*.py` with `PYTHONDONTWRITEBYTECODE=1`.
- Ran `scripts/build_codex_package.py --help` and verified `--version`
is gone while `--variant` and `--entrypoint-bin` are present.
- Verified the package builder reads version `0.0.0` from
`codex-rs/Cargo.toml`.
- Built a fake cross-target `codex-app-server` package using a
non-executable `--entrypoint-bin`; verified metadata records version
`0.0.0`, variant `codex-app-server`, and `bin/codex-app-server` as the
entrypoint.
## Why
The package builder should be easy to run during local iteration.
Requiring callers to provide both a target triple and an output
directory every time makes the common host-package case more awkward
than necessary.
This PR keeps explicit overrides available, but makes the default
invocation useful: build for the current host platform and place the
package in a fresh temporary directory. Because a temp output path is
otherwise easy to lose, the builder continues to print the final package
directory path when it completes.
## What changed
- Makes `--target` optional and maps the host OS/architecture to
supported Codex package target triples.
- Uses GNU Linux target triples for Linux host defaults, while keeping
the musl targets available for release jobs that pass `--target`
explicitly.
- Makes `--package-dir` optional and creates a new `codex-package-*`
temp directory when omitted.
- Documents the new defaults in `scripts/codex_package/README.md`.
## Verification
- Compiled `scripts/build_codex_package.py` and
`scripts/codex_package/*.py` with `PYTHONDONTWRITEBYTECODE=1`.
- Ran `scripts/build_codex_package.py --help` from outside the repo.
- Verified Linux host detection maps `x86_64` and `aarch64` to GNU
target triples.
- Ran a fake-Cargo package build while omitting both `--target` and
`--package-dir`; verified the generated metadata target, expected
package files, and printed temp package path.
- Ran a fake-Cargo package build for `x86_64-unknown-linux-gnu` and
verified `codex`, `bwrap`, and `rg` are assembled into the package.
## Why
The Codex package builder should produce a complete package without
requiring callers to pre-populate `rg` under `codex-cli/vendor` or have
`dotslash` installed on `PATH`. The repo already tracks the
authoritative DotSlash manifest in `codex-cli/bin/rg`, so the builder
can read that metadata directly and fetch the correct ripgrep archive
for the target it is packaging.
## What changed
- Added `scripts/codex_package/ripgrep.py` to parse `codex-cli/bin/rg`
after stripping the shebang, select the target platform entry, download
the configured artifact, and verify the recorded size and SHA-256
digest.
- Added a cache under `$TMPDIR/codex-package/<target>-rg` so verified
archives can be reused without fetching again.
- Extracted `rg`/`rg.exe` from `tar.gz` and `zip` artifacts into the
package-builder cache, then copied that into `codex-path` through the
existing package layout flow.
- Kept `--rg-bin` as an explicit local override for offline tests and
unusual local workflows.
- Documented the default `rg` fetch/cache behavior in
`scripts/codex_package/README.md`.
## Verification
- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran a local manifest fetch test covering shebang-stripped manifest
parsing, `tar.gz` extraction, `zip` extraction, size/SHA-256
verification, and cache reuse after deleting the original source
archives.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts with `--rg-bin`, including an assertion that generated
tar archives contain no duplicate member names.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23526).
* #23541
* __->__ #23526
## Why
Codex CLI packaging is currently split across npm staging, standalone
installers, and release bundle creation, which makes it hard to define
and validate a single valid package directory. This adds the first
standalone package builder so later release paths can converge on the
same canonical layout.
## What changed
- Added `scripts/build_codex_package.py` as the stable executable
wrapper around `scripts/codex_package`.
- Added modules for CLI parsing, target metadata, grouped cargo builds,
package layout validation, and archive writing.
- The builder creates a package directory with `codex-package.json`,
`bin/`, `codex-resources/`, and `codex-path`, and can serialize it as
`.tar.gz`, `.tar.zst`, or `.zip`.
- Source-built artifacts are built by one grouped `cargo build`: `codex`
for all targets, `bwrap` for Linux, and the Windows sandbox helpers for
Windows. `rg` remains an input because it is vendored from upstream
rather than built from this repo.
- Added `scripts/codex_package/README.md` to document the package
layout, source-built artifacts, and cargo profile behavior.
## Verification
- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts, including an assertion that generated tar archives
contain no duplicate member names.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23513).
* #23526
* __->__ #23513
