Commit Graph
24 Commits
Author SHA1 Message Date
Channing Conger e2f074e16c code-mode: move cell state into library actor (#28599)
A code-mode cell is a single JavaScript execution that can produce
output, call tools, wait for asynchronous work, resume, or be
terminated. This PR extracts the existing per-cell run loop into a
dedicated actor that owns the cell’s lifecycle state. It is primarily an
ownership change rather than a new lifecycle contract: existing behavior
now has one clear implementation boundary.

### Architecture
The session service remains responsible for session-wide concerns:
allocating cell IDs, storing shared values, creating cells, and routing
requests to them.

Once a cell is created, its execution state belongs to its actor.
Callers interact with the actor through a handle. The actor receives two
kinds of input: runtime events and control requests.

A single event loop serializes these inputs and applies the lifecycle
rules. It tracks the current observer—the caller waiting for an
update—along with accumulated output, outstanding callbacks, runtime
state, yield deadlines, and termination progress. Observation,
termination, completion, and cleanup therefore have one consistent
owner.

When the runtime has no immediately runnable work and is waiting only on
timers or tool results, the actor can return accumulated output and
information about outstanding tool calls while keeping the cell
available to resume. On completion or termination, it performs the
appropriate callback cleanup before publishing the final result and
removing the cell from the session.

A small host interface connects the actor to session-owned facilities
such as tool dispatch, notifications, stored values, and final cell
removal, keeping those responsibilities outside the actor itself.

### Why
Previously, cell lifecycle state and coordination lived alongside
session management. The actor boundary makes each cell a self-contained
state machine with a single writer, while the service becomes a registry
and adapter around it.

This makes lifecycle behavior easier to reason about and test in
isolation. It also establishes a clean boundary for later changing where
cells run or how they communicate without recreating their lifecycle
rules.
2026-06-16 19:28:55 -07:00
Channing Conger e93516e259 code-mode: extend test coverage to lock in cell lifecycle (#28468)
This PR establishes the intended behavior as an executable contract
before a refactor of the cell runtime begins. It also fixes cases where
a second observer or termination request could replace an existing
response channel and leave the original caller unresolved.

### Behavior codified
- A cell can yield output and subsequently resume to completion.
- A caller can run a cell until it has no immediately runnable work,
receive its accumulated output and outstanding tool-call IDs, and then
resume the same cell when the awaited work is available.
- Each cell admits one active observer:
   - a second observer receives an explicit busy error
   - the existing observer remains registered and is not displaced
- A natural result (conclusion of the js module) that has already
reached the cell controller wins over a later termination request.
- Otherwise, termination preempts execution and resolves both:
  - the active observer, if present
  - the caller requesting termination
- Repeated termination requests are rejected while termination is
already in progress.
- Terminal responses are sent only after outstanding callback work has
been handled:
- natural completion drains notifications and cancels outstanding tool
calls
- termination cancels and drains both notification and tool callbacks.
- Cell removal and cell_closed notification happen after callback
cleanup
2026-06-16 13:34:16 -07:00
Channing Conger 009a2bb93d ci: run code-mode unit tests on all bazel targets (#28562)
## Why

V8 should be stable under Bazel, so the `codex-code-mode` unit tests
should run across the Bazel platform matrix. If these tests prove
unstable, we should fix the tests rather than exclude them from CI.

## What changed

- Remove the explicit `//codex-rs/code-mode:code-mode-unit-tests`
exclusion from the macOS and Linux Bazel test jobs.
- Remove the same exclusion from the native Windows post-merge job.
- Keep the existing Windows gnullvm shard coverage.

## Bazel test coverage

The target contains 26 unit tests. A fresh uncached local Bazel
execution ran all 26 with 0 failures, 0 ignored tests, and 0 filtered
tests.

PR Bazel CI selected the target on every enabled platform and reported a
cached pass:

| Platform | Passing CI job |
| --- | --- |
| macOS aarch64 | [Bazel test
passed](https://github.com/openai/codex/actions/runs/27636617545/job/81725447804)
|
| macOS x86_64 | [Bazel test passed in
2.2s](https://github.com/openai/codex/actions/runs/27636617545/job/81725448008)
|
| Linux GNU | [Bazel test passed in
0.4s](https://github.com/openai/codex/actions/runs/27636617545/job/81725447898)
|
| Linux musl | [Bazel test passed in
0.4s](https://github.com/openai/codex/actions/runs/27636617545/job/81725448117)
|
| Windows gnullvm | [Bazel test passed in shard 4/4 in
1.6s](https://github.com/openai/codex/actions/runs/27636617545/job/81725448166)
|
2026-06-16 11:26:33 -07:00
Channing Conger aa46f2debf code-mode standalone: extract protocol and add host crate (#27724)
This is phase 1 of a 4 phase stack:
1. **Add protocol and host crates for new IPC code mode implementation**
2. Create the new standalone binary
3. Create a new IPC `CodeModeSessionProvider` to use new binary
4. Remove v8 from core and only use IPC provider


## Add protocol and host crates for new IPC code mode implementation
Establish a clean process boundary without changing the existing
in-process behavior.

- Add the codex-code-mode-protocol crate for shared session, runtime,
response, and tool-definition types.
- Move protocol-facing code out of the V8-backed implementation.
- Add a buildable codex-code-mode-host crate as the foundation for the
standalone process.
- Keep the existing in-process runtime as the active implementation.
2026-06-11 22:37:26 -07:00
Channing Conger 16c7c79540 ci(v8): gate Windows source builds on relevant changes (#27715)
Avoid rebuilding sandboxed Windows MSVC V8 artifacts for unrelated
changes to `codex-rs/Cargo.toml`.

The V8 canary now compares the resolved V8 version between the base and
head commits and only runs the Windows source-build matrix when:

- the resolved V8 crate version changes;
- Windows artifact-production scripts or workflows change; or
- the workflow is manually dispatched.

The existing Bazel V8 matrix is unchanged.

## Why

The Windows MSVC source builds take roughly two to three hours and
currently run whenever any entry in the broad `v8-canary` path filter
changes.
2026-06-11 18:44:42 -07:00
Channing Conger b89ce9a2bc build(v8): update rusty_v8 to 149.2.0 (#26464) 2026-06-06 14:27:23 -07:00
Channing Conger 4be1a168fc ci: test windows cross build (#25000)
We cross build when using bazel for windows. This causes a couple
hiccups in that v8 does a mksnapshot step that is expecting to snapshot
on the host arch which wasn't matching when we were doing the
crossbuild. This was causing segfault failiures when starting up
codemode from a cross built artifact.

This changes things such that we cross build the library and then run
and link a snapshot on the host machine/arch which is windows. This
gives us a functional snapshot and library that can start code-mode on
windows.

This fixes the build and then fixes two test regressions we had.
2026-06-04 17:51:13 -07:00
Channing Conger c9dc0f6338 code-mode: introduce durable session interface (#24180)
## Summary

Introduce a `CodeModeSession` interface for executing and managing
code-mode cells.

This moves cell lifecycle, callback delegation, termination, and
shutdown behind a session abstraction, while continuing to use the
existing in-process implementation, and the ability to implement an
external process one behind this interface.

A Codex session owns one `CodeModeSession`, which in turn owns its
running cells and stored code-mode state. Each cell is represented to
the caller as a `StartedCell`, exposing its cell ID and initial
response.

It also introduces a `CodeModeSessionDelegate` callback interface. A
session uses the delegate to invoke nested host tools and emit
notifications while a cell is running, allowing the runtime to
communicate with its owning Codex session without depending directly on
core turn handling.

<img width="2121" height="1001" alt="image"
src="https://github.com/user-attachments/assets/c349a819-2a59-485c-bda4-2caf68ac4c31"
/>
2026-05-29 11:42:52 -07:00
Channing Conger f94157a4b2 code-mode: merge stored values by key (#24159)
## Summary

Change code-mode stored value updates to merge writes by key instead of
replacing the session's complete stored-value map after each cell
completes.

Previously, each cell received a snapshot of stored values and returned
the complete resulting map. When multiple cells ran concurrently, a
later completion could overwrite values written by another cell because
it committed an older snapshot.

This change moves stored-value ownership into `CodeModeService`:

- Each runtime starts from the service's current stored values.
- Runtime completion reports only keys written by that cell.
- The service merges those writes into the current stored-value map on
successful completion.
- Core no longer replaces its stored-value state from a cell result.

As a result, concurrently executing cells can update different stored
keys without clobbering one another.

The move into CodeModeService is motivated by a desire to have this
lifetime tied to a new lifetime object on that side in a subsequent PR.
2026-05-22 19:09:02 -07:00
Channing Conger 014f19af5f ci: Use codex produced v8 artifacts for release builds (#23934)
Updates our build script to pull down the artifacts like we do in CI for
building v8 into our targets.

This changes the flow so that we now pre-install rusty v8 assets for all
of our release targets from pre-built in workflow.
Secondarily if running it locally we now optionally pull the assets down
on python run assuming the user hasn't set the proper values, it then
provides them.

Sorry for the miss here.
2026-05-22 09:42:08 -07:00
Channing CongerandCodex 7cdeab33d1 CI: Customize v8 building (#22086)
## Summary

Move the rusty_v8 artifact production into hermetic Bazel path and bump
the `v8` crate to `147.4.0`

The new flow builds V8 release artifacts from source for Darwin and
Linux targets, publishes both the current release-compatible artifacts
and sandbox-enabled variants, and keeps Cargo consumers on prebuilt
binaries by continuing to feed the `v8` crate the archive and generated
binding files it already expects.

## Why

We need control over V8 build-time features without giving up prebuilt
artifacts for downstream Cargo builds.

Upstream `rusty_v8` already supports source-only features such as
`v8_enable_sandbox`, but its normal prebuilt release assets do not cover
every feature combination we need. Building the artifacts ourselves lets
us enable settings such as the V8 sandbox and pointer compression at
artifact build time, then publish those outputs so ordinary Cargo builds
can still consume prebuilts instead of compiling V8 locally.

This keeps the fast consumer experience of prebuilt `rusty_v8` archives
while giving us a reproducible path to ship featureful variants that
upstream does not currently publish for us.

## Implementation Notes

The Bazel graph in this PR is not copied wholesale from `rusty_v8`;
`rusty_v8`'s normal source build is still GN/Ninja-based.

Instead, this change starts from upstream V8's Bazel rules and adapts
them to Codex's hermetic toolchains and dependency layout. Where we
intentionally follow `rusty_v8`, we mirror its existing artifact
contract:

- the same `v8` crate version and generated binding expectations
- the same sandbox feature relationship, where sandboxing requires
pointer compression
- the same custom libc++ model expected by Cargo's default
`use_custom_libcxx` feature
- the same release-style archive plus `src_binding` outputs consumed by
the `v8` crate

To preserve that contract, the Bazel release path pins the libc++,
libc++abi, and llvm-libc revisions used by `rusty_v8 v147.4.0`, builds
release artifacts with `--config=rusty-v8-upstream-libcxx`, and folds
the matching runtime objects into the final static archive.

## Windows

Windows is annoyingly handled differently.

Codex's current hermetic Bazel Windows C++ platform is `windows-gnullvm`
/ `x86_64-w64-windows-gnu`, while upstream `rusty_v8` publishes Windows
prebuilts for `*-pc-windows-msvc`. Those are different ABIs, so the
Bazel graph cannot truthfully reproduce the upstream MSVC artifacts
until we add a real MSVC-targeting C++ toolchain.

For now:

- Windows MSVC consumers continue to use upstream `rusty_v8` release
archives.
- Windows GNU targets are built in-tree so they link against a matching
GNU ABI.
- The canary workflow separately exercises upstream `rusty_v8` source
builds for MSVC sandbox artifacts, but MSVC is not yet part of the
Bazel-produced release matrix.

## Validation
This PR is technically self validating through CI. I have already
published it as a release tag so the artifacts from this branch are
published to
https://github.com/openai/codex/releases/tag/rusty-v8-v147.4.0 CI for
this PR should therefore consume our own release targets. I have also
locally tested for linux and darwin.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-18 21:33:05 -07:00
Channing Conger 589b820d6e code-mode: Add pending-aware code mode execution (#22280)
Introduce execute_to_pending and wait_to_pending APIs that freeze
pending-mode runtimes until an explicit resume, while preserving the
existing continuously-running execute path. Add runtime and service
coverage for pending, resume, completion, and freeze behavior.
2026-05-12 17:16:57 -07:00
Channing Conger 03d3403a41 ci: trigger rusty-v8 releases from tags (#21259)
Swap to tag based releasing and allow tags of type `rusty-v8-v*.*.*`
2026-05-05 16:56:43 -07:00
Channing Conger 36460387ec Enable V8 sandboxing for source-built builds (#21146)
## Summary

This is the first PR in the V8 in-process sandboxing rollout.

It adds the build-system and Rust feature plumbing needed to support
sandboxed V8 builds, then enables sandboxing by default for the
source-built Bazel V8 path that we control directly. It deliberately
keeps the published `rusty_v8` artifact workflows on their current
non-sandboxed contract so this PR can land and ship independently before
we change any released artifacts.

## Rollout plan

- [x] **PR 1: land sandbox plumbing and default source-built Bazel V8 to
sandboxed mode**

- [ ] **PR 2: publish sandbox-enabled release artifacts and add
compatibility validation**
- Produce sandboxed artifact pairs for every released Cargo target that
does not already use the source-built Bazel path.
- Add CI coverage that consumes those sandboxed artifacts and verifies:
    - `codex-v8-poc` reports sandbox enabled
    - `codex-code-mode` builds/tests against the sandboxed path

- [ ] **PR 3: switch release consumers to sandboxed artifacts by
default**
  - Update released artifact selectors/checksums.
- Enable the Rust `v8_enable_sandbox` feature in the default release
path.
- Make the sandboxed artifact family the normal path for published
builds.

- [ ] **PR 4: remove rollout-only compatibility paths**
- Remove the temporary non-sandbox release compatibility config once the
new default has shipped and baked.
  - Keep the invariant tests permanently.
2026-05-05 14:36:37 -07:00
Channing Conger a5fbcf1ab4 Prune unused code-mode globals (#20542)
Hide Atomics, SharedArrayBuffer, and WebAssembly from the code-mode
runtime since the harness does not expose worker support or need those
APIs.
2026-05-01 15:11:22 -07:00
Channing Conger c23566b3af Add JIT entitlement for macosx (#15409)
Without this entitlement, hardened mac os release binaries are unable to
allocate the executable memory for the JIT compiled JS.

Tested with local signing.  Without entitlement I reproduce the error:
```
#
# Fatal process out of memory: Failed to reserve virtual memory for CodeRange
#
==== C stack trace ===============================

    0   codex                               0x00000001075d1acc codex + 85760716
    1   codex                               0x00000001075d6a64 codex + 85781092
    2   codex                               0x00000001075c7100 codex + 85717248
    3   codex                               0x0000000107637394 codex + 86176660
    4   codex                               0x0000000107823cfc codex + 88194300
    5   codex                               0x000000010777c438 codex + 87508024
    6   codex                               0x000000010777d130 codex + 87511344
    7   codex                               0x0000000107c87a54 codex + 92797524
    8   codex                               0x0000000107641188 codex + 86217096
    9   codex                               0x00000001076412d8 codex + 86217432
    10  codex                               0x0000000107553908 codex + 85244168
    11  codex                               0x000000010465f124 codex + 36008228
    12  codex                               0x000000010466a0d0 codex + 36053200
    13  codex                               0x000000010466ce78 codex + 36064888
    14  codex                               0x000000010734edb0 codex + 83127728
    15  libsystem_pthread.dylib             0x00000001810d3c08 _pthread_start + 136
    16  libsystem_pthread.dylib             0x00000001810ceba8 thread_start + 8
zsh: trace trap  target/release/codex exec --enable code_mode_only --enable code_mode --
```

With the entitlement the exec succeeds.
2026-03-21 13:43:14 -07:00
Channing CongerandCodex e4eedd6170 Code mode on v8 (#15276)
Moves Code Mode to a new crate with no dependencies on codex. This
create encodes the code mode semantics that we want for lifetime,
mounting, tool calling.

The model-facing surface is mostly unchanged. `exec` still runs raw
JavaScript, `wait` still resumes or terminates a `cell_id`, nested tools
are still available through `tools.*`, and helpers like `text`, `image`,
`store`, `load`, `notify`, `yield_control`, and `exit` still exist.

The major change is underneath that surface:

- Old code mode was an external Node runtime.
- New code mode is an in-process V8 runtime embedded directly in Rust.
- Old code mode managed cells inside a long-lived Node runner process.
- New code mode manages cells in Rust, with one V8 runtime thread per
active `exec`.
- Old code mode used JSON protocol messages over child stdin/stdout plus
Node worker-thread messages.
- New code mode uses Rust channels and direct V8 callbacks/events.

This PR also fixes the two migration regressions that fell out of that
substrate change:

- `wait { terminate: true }` now waits for the V8 runtime to actually
stop before reporting termination.
- synchronous top-level `exit()` now succeeds again instead of surfacing
as a script error.

---

- `core/src/tools/code_mode/*` is now mostly an adapter layer for the
public `exec` / `wait` tools.
- `code-mode/src/service.rs` owns cell sessions and async control flow
in Rust.
- `code-mode/src/runtime/*.rs` owns the embedded V8 isolate and
JavaScript execution.
- each `exec` spawns a dedicated runtime thread plus a Rust
session-control task.
- helper globals are installed directly into the V8 context instead of
being injected through a source prelude.
- helper modules like `tools.js` and `@openai/code_mode` are synthesized
through V8 module resolution callbacks in Rust.

---

Also added a benchmark for showing the speed of init and use of a code
mode env:
```
$ cargo bench -p codex-code-mode --bench exec_overhead -- --samples 30 --warm-iterations 25 --tool-counts 0,32,128
Finished [`bench` profile [optimized]](https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles) target(s) in 0.18s
     Running benches/exec_overhead.rs (target/release/deps/exec_overhead-008c440d800545ae)
exec_overhead: samples=30, warm_iterations=25, tool_counts=[0, 32, 128]
scenario       tools samples    warmups      iters      mean/exec       p95/exec       rssΔ p50       rssΔ max
cold_exec          0      30          0          1         1.13ms         1.20ms        8.05MiB        8.06MiB
warm_exec          0      30          1         25       473.43us       512.49us      912.00KiB        1.33MiB
cold_exec         32      30          0          1         1.03ms         1.15ms        8.08MiB        8.11MiB
warm_exec         32      30          1         25       509.73us       545.76us      960.00KiB        1.30MiB
cold_exec        128      30          0          1         1.14ms         1.19ms        8.30MiB        8.34MiB
warm_exec        128      30          1         25       575.08us       591.03us      736.00KiB      864.00KiB
memory uses a fresh-process max RSS delta for each scenario
```

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-20 23:36:58 -07:00
Channing Conger 1350477150 Add v8-poc consumer of our new built v8 (#15203)
This adds a dummy v8-poc project that in Cargo links against our
prebuilt binaries and the ones provided by rusty_v8 for non musl
platforms. This demonstrates that we can successfully link and use v8 on
all platforms that we want to target.

In bazel things are slightly more complicated. Since the libraries as
published have libc++ linked in already we end up with a lot of double
linked symbols if we try to use them in bazel land. Instead we fall back
to building rusty_v8 and v8 from source (cached of course) on the
platforms we ship to.

There is likely some compatibility drift in the windows bazel builder
that we'll need to reconcile before we can re-enable them. I'm happy to
be on the hook to unwind that.
2026-03-20 12:08:25 -07:00
Channing Conger a941d8439d Bump aws-lc-rs (#15337)
Bump our dep.

RUSTSEC-2026-0048
Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0048
2026-03-20 18:59:13 +00:00
Channing Conger ded7854f09 V8 Bazel Build (#15021)
Alternative approach, we use rusty_v8 for all platforms that its
predefined, but lets build from source a musl v8 version with bazel for
x86 and aarch64 only. We would need to release this on github and then
use the release.
2026-03-19 18:05:23 -07:00
Channing Conger 70eddad6b0 dynamic tool calls: add param exposeToContext to optionally hide tool (#14501)
This extends dynamic_tool_calls to allow us to hide a tool from the
model context but still use it as part of the general tool calling
runtime (for ex from js_repl/code_mode)
2026-03-14 01:58:43 -07:00
Channing Conger 0daffe667a code_mode: Move exec params from runtime declarations to @pragma (#14511)
This change moves code_mode exec session settings out of the runtime API
and into an optional first-line pragma, so instead of calling runtime
helpers like set_yield_time() or set_max_output_tokens_per_exec_call(),
the model can write // @exec: {"yield_time_ms": ...,
"max_output_tokens": ...} at the top of the freeform exec source. Rust
now parses that pragma before building the source, validates it, and
passes the values directly in the exec start message to the code-mode
broker, which applies them at session start without any worker-runtime
mutation path. The @openai/code_mode module no longer exposes those
setter functions, the docs and grammar were updated to describe the
pragma form, and the existing code_mode tests were converted to use
pragma-based configuration instead.
2026-03-13 03:27:42 +00:00
Channing Conger fd4a673525 Responses: set x-client-request-id as convesration_id when talking to responses (#14312)
Right now we're sending the header session_id to responses which is
ignored/dropped. This sets a useful x-client-request-id to the
conversation_id.
2026-03-11 12:33:10 -07:00
Channing Conger c6343e0649 Implemented thread-level atomic elicitation counter for stopwatch pausing (#12296)
### Purpose
While trying to build out CLI-Tools for the agent to use under skills we
have found that those tools sometimes need to invoke a user elicitation.
These elicitations are handled out of band of the codex app-server but
need to indicate to the exec manager that the command running is not
going to progress on the usual timeout horizon.

### Example
Model calls universal exec:
`$ download-credit-card-history --start-date 2026-01-19 --end-date
2026-02-19 > credit_history.jsonl`

download-cred-card-history might hit a hosted/preauthenticated service
to fetch data. That service might decide that the request requires an
end user approval the access to the personal data. It should be able to
signal to the running thread that the command in question is blocked on
user elicitation. In that case we want the exec to continue, but the
timeout to not expire on the tool call, essentially freezing time until
the user approves or rejects the command at which point the tool would
signal the app-server to decrement the outstanding elicitation count.
Now timeouts would proceed as normal.

### What's Added

- New v2 RPC methods:
    - thread/increment_elicitation
    - thread/decrement_elicitation
- Protocol updates in:
    - codex-rs/app-server-protocol/src/protocol/common.rs
    - codex-rs/app-server-protocol/src/protocol/v2.rs
- App-server handlers wired in:
    - codex-rs/app-server/src/codex_message_processor.rs

### Behavior

- Counter starts at 0 per thread.
- increment atomically increases the counter.
- decrement atomically decreases the counter; decrement at 0 returns
invalid request.
- Transition rules:
- 0 -> 1: broadcast pause state, pausing all active stopwatches
immediately.
    - \>0 -> >0: remain paused.
    - 1 -> 0: broadcast unpause state, resuming stopwatches.
- Core thread/session logic:
    - codex-rs/core/src/codex_thread.rs
    - codex-rs/core/src/codex.rs
    - codex-rs/core/src/mcp_connection_manager.rs

### Exec-server stopwatch integration

- Added centralized stopwatch tracking/controller:
    - codex-rs/exec-server/src/posix/stopwatch_controller.rs
- Hooked pause/unpause broadcast handling + stopwatch registration:
    - codex-rs/exec-server/src/posix/mcp.rs
    - codex-rs/exec-server/src/posix/stopwatch.rs
    - codex-rs/exec-server/src/posix.rs
2026-03-09 22:29:26 -07:00