mirror of
https://github.com/openai/codex.git
synced 2026-08-23 13:09:46 +00:00
main
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Adam Perry @ OpenAI
|
cc2f262033 |
Extend bundled package discovery and expose its version (#37886)
## What changed - Recognize executables under `codex-resources/` as part of a package layout by resolving the sibling `bin/` directory, while requiring that directory to exist. - Add `InstallContext::package_manifest()` to parse the semantic package version from `codex-package.json`. ## Testing - Extend the package-layout test fixture with a complete manifest and verify that version `1.2.3` is returned. GitOrigin-RevId: 034aad3e8b865c6c6ecdff47da7c12c969744404 |
||
|
seanh-oai
|
53d06e24ea |
Prefer the bundled resource for the code mode host (#36264)
## What changed - Resolve the code mode host from the package or standalone `resources` directory before checking beside the Codex executable. - Preserve the existing executable-adjacent fallback when the bundled resource is missing or is not a file. ## Testing - Cover package and standalone layouts, resource precedence, missing legacy binaries, and resource paths that point to directories. GitOrigin-RevId: c9e97c22f35a118377dea715d255899192d1b48a |
||
|
Channing Conger
|
97576b1794 |
Run code mode exclusively through the standalone host (#36217)
## What changed - Move the V8 implementation into a dedicated `codex-code-mode-runtime` crate used by `codex-code-mode-host`, removing the embedded runtime fallback from the Codex process. - Resolve the host executable from the active installation layout and check its availability before selecting tools. - Fall back to direct tools with a one-time warning when optional code mode is unavailable. Keep `code_mode_only` and `disable_in_process_fallback` configurations fail-closed. ## Testing - Cover host discovery for standalone and package layouts, including missing hosts and symlinks. - Verify direct-tool fallback, one-time warnings, and fail-closed code-mode-only behavior. GitOrigin-RevId: 5aa3c6f1db148b2231fc24089a2ee0e2b00dbddb |
||
|
Charlie Marsh
|
e212cc95b0 |
Detect Codex installs managed by pnpm (#31503)
## Why The Codex JavaScript shim currently distinguishes npm and Bun installs, but a global pnpm install falls back to npm. That causes the native CLI, `codex doctor`, and update flows to report or run npm commands even though pnpm owns the installation. pnpm also allows its global package and bin directories to differ, so `PNPM_HOME` does not reliably identify the package owner. ## What changed - detect pnpm-managed installs by finding pnpm's `node_modules/.modules.yaml` metadata from the launched JavaScript entrypoint - pass a mutually exclusive `CODEX_MANAGED_BY_PNPM` marker into the native CLI - represent pnpm in install context, doctor output, version checks, and TUI update actions without changing the existing public `InstallContext::from_exe` signature - recommend `pnpm add -g @openai/codex` for pnpm-managed installs and snapshot the rendered TUI update notice Validated with the 9-test install-context suite, the focused pnpm TUI snapshot test, and Node's syntax check for the launcher. Closes https://github.com/openai/codex/issues/10294. |
||
|
Michael Bolin
|
6bcccb0ee6 |
cli: add package path from install context (#26189)
## Why Codex package installs include helper binaries in `codex-path`, such as the bundled `rg`. Package-layout launches should add that directory before user commands run, but standalone launches were missing it while npm launches only worked because `codex.js` had its own legacy `PATH` rewrite. That made npm and standalone package behavior diverge. Shell snapshot restoration can also reset `PATH` after runtime setup. Any package-owned `PATH` prepend has to be recorded as an explicit runtime override so shells, unified exec, and user-shell commands keep access to `codex-path` after a snapshot is sourced. ## Repro Before this change, a curl-installed package could contain `rg` under `codex-path` but still fail to put it on `PATH`: ```shell mkdir /tmp/test-codex-curl curl -fsSL https://chatgpt.com/codex/install.sh \ | CODEX_HOME=/tmp/test-codex-curl CODEX_NON_INTERACTIVE=1 sh /tmp/test-codex-curl/packages/standalone/current/bin/codex exec \ --skip-git-repo-check 'print `which -a rg`' find /tmp/test-codex-curl -name rg ``` The `which -a rg` output omitted the packaged helper even though `find` showed it under `/tmp/test-codex-curl/packages/standalone/releases/.../codex-path/rg`. The npm install path behaved differently only because `codex-cli/bin/codex.js` had legacy `PATH` rewriting: ```shell mkdir /tmp/test-codex-npm cd /tmp/test-codex-npm npm install @openai/codex ./node_modules/.bin/codex exec --skip-git-repo-check 'print `which -a rg`' ``` That printed the npm package's `vendor/<target>/codex-path/rg` first. This PR moves that behavior into Rust-side package launch setup so curl/standalone and npm/bun launches agree without JS rewriting `PATH`. ## What Changed - `codex-rs/arg0` now uses `InstallContext::current().package_layout.path_dir` to prepend the package helper directory before any threads are created. - Package helper `PATH` setup is independent from the temporary arg0 alias setup, so `codex-path` is still added even if CODEX_HOME tempdir, lock, or symlink setup fails. - `codex-rs/install-context` detects the canonical package layout we ship: `bin/`, `codex-resources/`, and `codex-path/` next to `codex-package.json`. - Shell, local unified exec, and user-shell runtimes now record package `codex-path` prepends in `explicit_env_overrides`, matching the existing zsh-fork behavior so shell snapshots cannot restore over the package helper path. - Remote unified exec requests do not receive the local app-server package path overlay. - `codex-cli/bin/codex.js` no longer computes or overrides `PATH`; it only locates the native binary in the canonical package layout and passes npm/bun management metadata. - Added regression tests for `PATH` ordering, package layout detection, and shell snapshot preservation of package path prepends. ## Verification - `node --check codex-cli/bin/codex.js` - `just test -p codex-install-context -p codex-arg0` - `just test -p codex-core user_shell_snapshot_preserves_package_path_prepend` - `just test -p codex-core tools::runtimes::tests` - `just bazel-lock-update` - `just bazel-lock-check` - `just fix -p codex-install-context -p codex-arg0 -p codex-core` |
||
|
Michael Bolin
|
c7bcb90f9b |
package: include zsh fork in Codex package (#23756)
## Why The package layout gives Codex a stable place for runtime helpers that should travel with the entrypoint. `shell_zsh_fork` still required users to configure `zsh_path` manually, even though we already publish prebuilt zsh fork artifacts. This PR builds on #24129 and uses the shared DotSlash artifact fetcher to include the zsh fork in Codex packages when a matching target artifact exists. Packaged Codex builds can then discover the bundled fork automatically; the user/profile `zsh_path` override is removed so the feature uses the package-managed artifact instead of a legacy path knob. ## What Changed - Added `scripts/codex_package/codex-zsh`, a checked-in DotSlash manifest for the current macOS arm64 and Linux zsh fork artifacts. - Taught `scripts/build_codex_package.py` to fetch the matching zsh fork artifact and install it at `codex-resources/zsh/bin/zsh` when available for the selected target. - Added package layout validation for the optional bundled zsh resource. - Added `InstallContext::bundled_zsh_path()` and `InstallContext::bundled_zsh_bin_dir()` for package-layout resource discovery. - Threaded the packaged zsh path through config loading as the runtime `zsh_path` for packaged installs, and removed the config/profile/CLI override path. - Kept the packaged default zsh override typed as `AbsolutePathBuf` until the existing runtime `Config::zsh_path` boundary. - Updated app-server zsh-fork integration tests to spawn `codex-app-server` from a temporary package layout with `codex-resources/zsh/bin/zsh`, matching the new packaged discovery path instead of setting `zsh_path` in config. - Switched package executable copying from metadata-preserving `copy2()` to `copyfile()` plus explicit executable bits, which avoids macOS file-flag failures when local smoke tests use system binaries as inputs. ## Testing To verify that the `zsh` executable from the Codex package is picked up correctly, first I ran: ```shell ./scripts/build_codex_package.py ``` which created: ``` /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/ ``` so then I ran: ``` /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/bin/codex exec --enable shell_zsh_fork 'run `echo $0`' ``` which reported the following, as expected: ``` /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/codex-resources/zsh/bin/zsh ``` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23756). * #23768 * __->__ #23756 |
||
|
Michael Bolin
|
b0b383bea3 |
runtime: use install context for bundled bwrap (#23634)
## Summary The Linux sandbox should find bundled `bwrap` through the same package-layout abstraction as the rest of the runtime, instead of maintaining a separate standalone-specific lookup path. This adds an `InstallContext` helper for bundled resources and updates `codex-linux-sandbox` to ask the current install context for `codex-resources/bwrap` before falling back to the old executable-relative probes. The tests cover npm-style, standalone, and canonical package layouts so `bwrap` lookup follows the package structure introduced earlier in the stack. ## Test plan - `cargo test -p codex-install-context` - `cargo test -p codex-linux-sandbox --lib` - `just fix -p codex-install-context -p codex-linux-sandbox` - `just bazel-lock-check` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23634). * #23638 * #23637 * #23636 * #23635 * __->__ #23634 |
||
|
Michael Bolin
|
cfa16fcc2e |
runtime: detect Codex package layout (#23596)
## Why The package-builder stack now creates a canonical Codex package directory where the entrypoint lives under `bin/`, bundled helper resources live under `codex-resources/`, and bundled PATH-style tools live under `codex-path/`. That layout is not specific to the standalone installer: npm, brew, install scripts, and manually unpacked artifacts should all be able to use the same package shape. The Rust runtime still only knew about the legacy standalone release layout, where resources sit next to the executable. A packaged binary therefore would not identify its package root or prefer the bundled `rg` from `codex-path/`. ## What changed - Adds `CodexPackageLayout` to `codex-install-context` and detects it from an executable path shaped like `<package>/bin/<entrypoint>` when `<package>/codex-package.json` is present. - Splits `InstallContext` into an install `method` plus an optional package layout so the layout is shared across npm, bun, brew, standalone, and other launch contexts. - Stores package-layout paths as `AbsolutePathBuf` values. - Keeps `codex-resources/` and `codex-path/` optional so Codex can still run with degraded behavior if sidecar directories are missing. - Updates `InstallContext::rg_command()` to prefer bundled `codex-path/rg` or `rg.exe`, then fall back to the legacy standalone resources location, then system `rg`. - Updates `codex doctor` reporting so package installs show package, bin, resources, and path directories, and so bundled search detection recognizes `codex-path/` for any install method. ## Test plan - `cargo test -p codex-install-context` - `cargo test -p codex-cli` - `cargo test -p codex-tui update_action::tests::maps_install_context_to_update_action` - `just bazel-lock-check` |
||
|
Charlie Marsh
|
54ef99a365 |
Disable empty Cargo test targets (#21584)
## Summary `cargo test` has entails both running standard Rust tests and doctests. It turns out that the doctest discovery is fairly slow, and it's a cost you pay even for crates that don't include any doctests. This PR disables doctests with `doctest = false` for crates that lack any doctests. For the collection of crates below, this speeds up test execution by >4x. E.g., before this PR: ``` Benchmark 1: cargo test -p codex-utils-absolute-path -p codex-utils-cache -p codex-utils-cli -p codex-utils-home-dir -p codex-utils-output-truncation -p codex-utils-path -p codex-utils-string -p codex-utils-template -p codex-utils-elapsed -p codex-utils-json-to-toml Time (mean ± σ): 1.849 s ± 4.455 s [User: 0.752 s, System: 1.367 s] Range (min … max): 0.418 s … 14.529 s 10 runs ``` And after: ``` Benchmark 1: cargo test -p codex-utils-absolute-path -p codex-utils-cache -p codex-utils-cli -p codex-utils-home-dir -p codex-utils-output-truncation -p codex-utils-path -p codex-utils-string -p codex-utils-template -p codex-utils-elapsed -p codex-utils-json-to-toml Time (mean ± σ): 428.6 ms ± 6.9 ms [User: 187.7 ms, System: 219.7 ms] Range (min … max): 418.0 ms … 436.8 ms 10 runs ``` For a single crate, with >2x speedup, before: ``` Benchmark 1: cargo test -p codex-utils-string Time (mean ± σ): 491.1 ms ± 9.0 ms [User: 229.8 ms, System: 234.9 ms] Range (min … max): 480.9 ms … 512.0 ms 10 runs ``` And after: ``` Benchmark 1: cargo test -p codex-utils-string Time (mean ± σ): 213.9 ms ± 4.3 ms [User: 112.8 ms, System: 84.0 ms] Range (min … max): 206.8 ms … 221.0 ms 13 runs ``` Co-authored-by: Codex <noreply@openai.com> |
||
|
efrazer-oai
|
9d1bf002c6 |
Significantly improve standalone installer (#17022)
## Summary
This PR significantly improves the standalone installer experience.
The main changes are:
1. We now install the codex binary and other dependencies in a
subdirectory under CODEX_HOME.
(`CODEX_HOME/packages/standalone/releases/...`)
2. We replace the `codex.js` launcher that npm/bun rely on with logic in
the Rust binary that automatically resolves its dependencies (like
ripgrep)
## Motivation
A few design constraints pushed this work.
1. Currently, the entrypoint to codex is through `codex.js`, which
forces a node dependency to kick off our rust app. We want to move away
from this so that the entrypoint to codex does not rely on node or
external package managers.
2. Right now, the native script adds codex and its dependencies directly
to user PATH. Given that codex is likely to add more binary dependencies
than ripgrep, we want a solution which does not add arbitrary binaries
to user PATH -- the only one we want to add is the `codex` command
itself.
3. We want upgrades to be atomic. We do not want scenarios where
interrupting an upgrade command can move codex into undefined state (for
example, having a new codex binary but an old ripgrep binary). This was
~possible with the old script.
4. Currently, the Rust binary uses heuristics to determine which
installer created it. These heuristics are flaky and are tied to the
`codex.js` launcher. We need a more stable/deterministic way to
determine how the binary was installed for standalone.
5. We do not want conflicting codex installations on PATH. For example,
the user installing via npm, then installing via brew, then installing
via standalone would make it unclear which version of codex is being
launched and make it tough for us to determine the right upgrade
command.
## Design
### Standalone package layout
Standalone installs now live under `CODEX_HOME/packages/standalone`:
```text
$CODEX_HOME/
packages/
standalone/
current -> releases/0.111.0-x86_64-unknown-linux-musl
releases/
0.111.0-x86_64-unknown-linux-musl/
codex
codex-resources/
rg
```
where `standalone/current` is a symlink to a release directory.
On Windows, the release directory has the same shape, with `.exe` names
and Windows helpers in `codex-resources`:
```text
%CODEX_HOME%\
packages\
standalone\
current -> releases\0.111.0-x86_64-pc-windows-msvc
releases\
0.111.0-x86_64-pc-windows-msvc\
codex.exe
codex-resources\
rg.exe
codex-command-runner.exe
codex-windows-sandbox-setup.exe
```
This gives us:
- atomic upgrades because we can fully stage a release before switching
`standalone/current`
- a stable way for the binary to recognize a standalone install from its
canonical `current_exe()` path under CODEX_HOME
- a clean place for binary dependencies like `rg`, Windows sandbox
helpers, and, in the future, our custom `zsh` etc
### Command location
On Unix, we add a symlink at `~/.local/bin/codex` which points directly
to the `$CODEX_HOME/packages/standalone/current/codex` binary. This
becomes the main entrypoint for the CLI.
On Windows, we store the link at
`%LOCALAPPDATA%\Programs\OpenAI\Codex\bin`.
### PATH persistence
This is a tricky part of the PR, as there's no ~super reliable way to
ensure that we end up on PATH without significant tradeoffs.
Most Unix variants will have `~/.local/bin` on PATH already, which means
we *should* be fine simply registering the command there in most cases.
However, there are cases where this is not the case. In these cases, we
directly edit the profile depending on the shell we're in.
- macOS zsh: `~/.zprofile`
- macOS bash: `~/.bash_profile`
- Linux zsh: `~/.zshrc`
- Linux bash: `~/.bashrc`
- fallback: `~/.profile`
On Windows, we update the User `Path` environment variable directly and
we don't need to worry about shell profiles.
### Standalone runtime detection
This PR adds a new shared crate, `codex-install-context`, which computes
install ownership once per process and caches it in a `OnceLock`.
That context includes:
- install manager (`Standalone`, `Npm`, `Bun`, `Brew`, `Other`)
- the managed standalone release directory, when applicable
- the managed standalone `codex-resources` directory, when present
- the resolved `rg_command`
The standalone path is detected by canonicalizing `current_exe()`,
canonicalizing CODEX_HOME via `find_codex_home()`, and checking whether
the binary is running from under
`$CODEX_HOME/packages/standalone/releases`.
We intentionally do not use a release metadata file. The binary path is
the source of truth.
### Dependency resolution
For standalone installs, `grep_files` now resolves bundled `rg` from
`codex-resources` next to the Codex binary.
For npm/bun/brew/other installs, `grep_files` falls back to resolving
`rg` from PATH.
For Windows standalone installs, Windows sandbox helpers are still found
as direct siblings when present. If they are not direct siblings, the
lookup also checks the sibling `codex-resources` directory.
### TUI update path
The TUI now has `UpdateAction::StandaloneUnix` and
`UpdateAction::StandaloneWindows`, which rerun the standalone install
commands.
Unix update command:
```sh
sh -c "curl -fsSL https://chatgpt.com/codex/install.sh | sh"
```
Windows update command:
```powershell
powershell -c "irm https://chatgpt.com/codex/install.ps1|iex"
```
The Windows updater runs PowerShell directly. We do this because `cmd
/C` would parse the `|iex` as a cmd pipeline instead of passing it to
PowerShell.
## Additional installer behavior
- standalone installs now warn about conflicting npm/bun/brew-managed
`codex` installs and offer to uninstall them
- same-version reruns do not redownload the release if it is already
staged locally
## Testing
Installer smoke tests run:
- macOS: fresh install into isolated `HOME` and `CODEX_HOME` with
`scripts/install/install.sh --release latest`
- macOS: reran the installer against the same isolated install to verify
the same-version/update path and PATH block idempotence
- macOS: verified the installed `codex --version` and bundled
`codex-resources/rg --version`
- Windows: parsed `scripts/install/install.ps1` with PowerShell via
`[scriptblock]::Create(...)`
- Windows: verified the standalone update action builds a direct
PowerShell command and does not route the `irm ...|iex` command through
`cmd /C`
---------
Co-authored-by: Codex <noreply@openai.com>
|