openai/codex
1
0
Fork 0
mirror of https://github.com/openai/codex.git synced 2026-10-31 19:28:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge origin/main into codex/workspace-owner-usage-nudge

Browse Source 
Resolve the TUI and config schema conflicts between the workspace owner usage nudge work and the latest main changes without rewriting branch history.

Made-with: Cursor
This commit is contained in:
Richard Lee
2026-04-16 12:15:16 -07:00
parent 316baa8ca5 71174574ad
commit 1aba3c4ae7
402 changed files with 22084 additions and 6301 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.codex/skills/codex-pr-body/SKILL.md
+59
View File
@@ -0,0 +1,59 @@
---
name: codex-pr-body
description: Update the title and body of one or more pull requests.
---
## Determining the PR(s)
When this skill is invoked, the PR(s) to update may be specified explicitly, but in the common case, the PR(s) to update will be inferred from the branch / commit that the user is currently working on. For ordinary Git usage (i.e., not Sapling as discussed below), you may have to use a combination of `git branch` and `gh pr view <branch> --repo openai/codex --json number --jq '.number'` to determine the PR associated with the current branch / commit.
## PR Body Contents
When invoked, use `gh` to edit the pull request body and title to reflect the contents of the specified PR. Make sure to check the existing pull request body to see if there is key information that should be preserved. For example, NEVER remove an image in the existing pull request body, as the author may have no way to recover it if you remove it.
It is critically important to explain _why_ the change is being made. If the current conversation in which this skill is invoked has discussed the motivation, be sure to capture this in the pull request body.
The body should also explain _what_ changed, but this should appear after the _why_.
Limit discussion to the _net change_ of the commit. It is generally frowned upon to discuss changes that were attempted but later undone in the course of the development of the pull request. When rewriting the pull request body, you may need to eliminate details such as these when they are no longer appropriate / of interest to future readers.
Avoid references to absolute paths on my local disk. When talking about a path that is within the repository, simply use the repo-relative path.
It is generally helpful to discuss how the change was verified. That said, it is unnecessary to mention things that CI checks automatically, e.g., do not include "ran `just fmt`" as part of the test plan. Though identifying the new tests that were purposely introduced to verify the new behavior introduced by the pull request is often appropriate.
Make use of Markdown to format the pull request professionally. Ensure "code things" appear in single backticks when referenced inline. Fenced code blocks are useful when referencing code or showing a shell transcript. Also, make use of GitHub permalinks when citing existing pieces of code that are relevant to the change.
Make sure to reference any relevant pull requests or issues, though there should be no need to reference the pull request in its own PR body.
If there is documentation that should be updated on https://developers.openai.com/codex as a result of this change, please note that in a separate section near the end of the pull request. Omit this section if there is no documentation that needs to be updated.
## Working with Stacks
Sometimes a pull request is composed of a stack of commits that build on one another. In these cases, the PR body should reflect the _net_ change introduced by the stack as a whole, rather than the individual commits that make up the stack.
Similarly, sometimes a user may be using a tool like Sapling to leverage _stacked pull requests_, in which case the `base` of the PR may be the a branch that is the `head` of another PR in the stack rather than `main`. In this case, be sure to discuss only the net change between the `base` and `head` of the PR that is being opened against that stacked base, rather than the changes relative to `main`.
## Sapling
If `.git/sl/store` is present, then this Git repository is governed by Sapling SCM (https://sapling-scm.com).
In Sapling, run the following to see if there is a GitHub pull request associated with the current revision:
```shell
sl log --template '{github_pull_request_url}' -r .
```
Alternatively, you can run `sl sl` to see the current development branch and whether there is a GitHub pull request associated with the current commit. For example, if the output were:
```
@ cb032b31cf 72 minutes ago mbolin #11412
╭─╯ tui: show non-file layer content in /debug-config
│
o fdd0cd1de9 Today at 20:09 origin/main
│
~
```
- `@` indicates the current commit is `cb032b31cf`
- it is a development branch containing a single commit branched off of `origin/main`
- it is associated with GitHub pull request #11412
.github/scripts/run-bazel-ci.sh
+5
View File
@@ -94,6 +94,11 @@ print_bazel_test_log_tails() {
local rel_path="${target#//}"
rel_path="${rel_path/://}"
local test_log="${testlogs_dir}/${rel_path}/test.log"
local reported_test_log
reported_test_log="$(grep -F "FAIL: ${target} " "$console_log" | sed -nE 's#.* \(see ([^)]+/test\.log)\).*#\1#p' | head -n 1 || true)"
if [[ -n "$reported_test_log" ]]; then
test_log="$reported_test_log"
fi
echo "::group::Bazel test log tail for ${target}"
if [[ -f "$test_log" ]]; then
.github/workflows/rust-ci-full.yml
+1
View File
@@ -664,6 +664,7 @@ jobs:
export CODEX_TEST_REMOTE_ENV_CONTAINER_NAME=codex-remote-test-env
source "${GITHUB_WORKSPACE}/scripts/test-remote-env.sh"
echo "CODEX_TEST_REMOTE_ENV=${CODEX_TEST_REMOTE_ENV}" >> "$GITHUB_ENV"
echo "CODEX_TEST_REMOTE_EXEC_SERVER_URL=${CODEX_TEST_REMOTE_EXEC_SERVER_URL}" >> "$GITHUB_ENV"
- name: tests
id: test
MODULE.bazel.lock
Generated
+8 -2
View File
File diff suppressed because one or more lines are too long
SECURITY.md
+4
View File
@@ -11,3 +11,7 @@ Our security program is managed through Bugcrowd, and we ask that any validated
## Vulnerability Disclosure Program
Our Vulnerability Program Guidelines are defined on our [Bugcrowd program page](https://bugcrowd.com/engagements/openai).
## How to operate CODEX safely
For details on Codex security boundaries, including sandboxing, approvals, and network controls, see [Agent approvals & security](https://developers.openai.com/codex/agent-approvals-security).
announcement_tip.toml
+6 -12
View File
@@ -4,20 +4,14 @@
# version_regex matches against the CLI version (env!("CARGO_PKG_VERSION")); omit to apply to all versions.
# target_app specify which app should display the announcement (cli, vsce, ...).
[[announcements]]
content = "Welcome to Codex! Check out the new onboarding flow."
from_date = "2024-10-01"
to_date = "2024-10-15"
target_app = "cli"
# Test announcement only for local build version until 2026-01-10 excluded (past)
# Test announcement only for local build version until 2027-05-10 excluded
[[announcements]]
content = "This is a test announcement"
version_regex = "^0\\.0\\.0$"
to_date = "2026-05-10"
to_date = "2027-05-10"
[[announcements]]
content = "**BREAKING NEWS**: `gpt-5.3-codex` is out! Upgrade to `0.98.0` for a faster, smarter, more steerable agent."
from_date = "2026-02-01"
to_date = "2026-02-16"
version_regex = "^0\\.(?:[0-9]|[1-8][0-9]|9[0-7])\\."
content = "Update Required - This version will no longer be supported starting May 8th. Please upgrade to the latest version (https://github.com/openai/codex/releases/latest) using your preferred package manager."
# Matches 0.x.y versions from 0.0.y through 0.119.y; excludes 0.120.0 and newer.
version_regex = "^0\\.(?:[0-9]|[1-9][0-9]|1[01][0-9])\\."
to_date = "2026-05-08"
codex-rs/.cargo/audit.toml
+2 -1
View File
@@ -1,9 +1,10 @@
[advisories]
# Reviewed 2026-04-11. Keep this list in sync with ../deny.toml.
# Reviewed 2026-04-15. Keep this list in sync with ../deny.toml.
ignore = [
"RUSTSEC-2024-0388", # derivative 2.2.0 via starlark; upstream crate is unmaintained
"RUSTSEC-2025-0057", # fxhash 0.2.1 via starlark_map; upstream crate is unmaintained
"RUSTSEC-2024-0436", # paste 1.0.15 via starlark/ratatui; upstream crate is unmaintained
"RUSTSEC-2024-0320", # yaml-rust via syntect; remove when syntect drops or updates it
"RUSTSEC-2025-0141", # bincode via syntect; remove when syntect drops or updates it
"RUSTSEC-2026-0097", # rand 0.8.5 via age/codex-secrets and zbus/keyring; remove when transitive deps move to rand >=0.9.3
]
codex-rs/Cargo.lock
Generated
+177 -28
View File
@@ -1446,6 +1446,7 @@ dependencies = [
"codex-cloud-requirements",
"codex-config",
"codex-core",
"codex-core-plugins",
"codex-exec-server",
"codex-features",
"codex-feedback",
@@ -1461,6 +1462,7 @@ dependencies = [
"codex-sandboxing",
"codex-shell-command",
"codex-state",
"codex-thread-store",
"codex-tools",
"codex-utils-absolute-path",
"codex-utils-cargo-bin",
@@ -1647,6 +1649,7 @@ version = "0.0.0"
dependencies = [
"anyhow",
"clap",
"codex-app-server-protocol",
"codex-config",
"codex-connectors",
"codex-core",
@@ -1694,6 +1697,7 @@ dependencies = [
"codex-stdio-to-uds",
"codex-terminal-detection",
"codex-tui",
"codex-utils-absolute-path",
"codex-utils-cargo-bin",
"codex-utils-cli",
"codex-utils-path",
@@ -1728,7 +1732,7 @@ dependencies = [
"opentelemetry",
"opentelemetry_sdk",
"pretty_assertions",
"rand 0.9.2",
"rand 0.9.3",
"reqwest",
"rustls",
"rustls-native-certs",
@@ -1830,6 +1834,7 @@ name = "codex-code-mode"
version = "0.0.0"
dependencies = [
"async-trait",
"codex-protocol",
"deno_core_icudata",
"pretty_assertions",
"serde",
@@ -1905,10 +1910,12 @@ dependencies = [
"codex-api",
"codex-app-server-protocol",
"codex-apply-patch",
"codex-arg0",
"codex-async-utils",
"codex-code-mode",
"codex-config",
"codex-connectors",
"codex-core-plugins",
"codex-core-skills",
"codex-exec-server",
"codex-execpolicy",
@@ -1935,6 +1942,7 @@ dependencies = [
"codex-state",
"codex-terminal-detection",
"codex-test-binary-support",
"codex-thread-store",
"codex-tools",
"codex-utils-absolute-path",
"codex-utils-cache",
@@ -1956,6 +1964,7 @@ dependencies = [
"ctor 0.6.3",
"dirs",
"dunce",
"ed25519-dalek",
"env-flags",
"eventsource-stream",
"futures",
@@ -1973,7 +1982,7 @@ dependencies = [
"opentelemetry_sdk",
"predicates",
"pretty_assertions",
"rand 0.9.2",
"rand 0.9.3",
"regex-lite",
"reqwest",
"rmcp",
@@ -2007,6 +2016,34 @@ dependencies = [
"zstd 0.13.3",
]
[[package]]
name = "codex-core-plugins"
version = "0.0.0"
dependencies = [
"chrono",
"codex-app-server-protocol",
"codex-config",
"codex-core-skills",
"codex-exec-server",
"codex-git-utils",
"codex-login",
"codex-plugin",
"codex-protocol",
"codex-utils-absolute-path",
"codex-utils-plugins",
"dirs",
"pretty_assertions",
"reqwest",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.18",
"tokio",
"toml 0.9.11+spec-1.1.0",
"tracing",
"url",
]
[[package]]
name = "codex-core-skills"
version = "0.0.0"
@@ -2249,6 +2286,15 @@ dependencies = [
"tokio",
]
[[package]]
name = "codex-install-context"
version = "0.0.0"
dependencies = [
"codex-utils-home-dir",
"pretty_assertions",
"tempfile",
]
[[package]]
name = "codex-instructions"
version = "0.0.0"
@@ -2326,7 +2372,7 @@ dependencies = [
"once_cell",
"os_info",
"pretty_assertions",
"rand 0.9.2",
"rand 0.9.3",
"regex-lite",
"reqwest",
"serde",
@@ -2570,6 +2616,7 @@ dependencies = [
"codex-utils-string",
"codex-utils-template",
"encoding_rs",
"globset",
"http 1.4.0",
"icu_decimal",
"icu_locale_core",
@@ -2694,6 +2741,8 @@ dependencies = [
name = "codex-sandboxing"
version = "0.0.0"
dependencies = [
"anyhow",
"async-trait",
"codex-network-proxy",
"codex-protocol",
"codex-utils-absolute-path",
@@ -2702,6 +2751,7 @@ dependencies = [
"pretty_assertions",
"serde_json",
"tempfile",
"tokio",
"tracing",
"url",
"which 8.0.0",
@@ -2718,7 +2768,7 @@ dependencies = [
"codex-keyring-store",
"keyring",
"pretty_assertions",
"rand 0.9.2",
"rand 0.9.3",
"regex",
"schemars 0.8.22",
"serde",
@@ -2834,9 +2884,22 @@ version = "0.0.0"
dependencies = [
"async-trait",
"chrono",
"codex-git-utils",
"codex-protocol",
"codex-rollout",
"codex-state",
"pretty_assertions",
"prost 0.14.3",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.18",
"tokio",
"tokio-stream",
"tonic",
"tonic-prost",
"tonic-prost-build",
"uuid",
]
[[package]]
@@ -2874,11 +2937,13 @@ dependencies = [
"codex-cli",
"codex-cloud-requirements",
"codex-config",
"codex-connectors",
"codex-exec-server",
"codex-features",
"codex-feedback",
"codex-file-search",
"codex-git-utils",
"codex-install-context",
"codex-login",
"codex-mcp",
"codex-model-provider-info",
@@ -2917,7 +2982,7 @@ dependencies = [
"pathdiff",
"pretty_assertions",
"pulldown-cmark",
"rand 0.9.2",
"rand 0.9.3",
"ratatui",
"ratatui-macros",
"regex-lite",
@@ -3650,6 +3715,7 @@ dependencies = [
"cfg-if",
"cpufeatures",
"curve25519-dalek-derive",
"digest",
"fiat-crypto",
"rustc_version",
"subtle",
@@ -4126,7 +4192,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users 0.5.2",
"windows-sys 0.59.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -4230,6 +4296,30 @@ version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]]
name = "either"
version = "1.15.0"
@@ -5159,7 +5249,7 @@ dependencies = [
"idna",
"ipnet",
"once_cell",
"rand 0.9.2",
"rand 0.9.3",
"ring",
"thiserror 2.0.18",
"tinyvec",
@@ -5181,7 +5271,7 @@ dependencies = [
"moka",
"once_cell",
"parking_lot",
"rand 0.9.2",
"rand 0.9.3",
"resolv-conf",
"smallvec",
"thiserror 2.0.18",
@@ -6728,7 +6818,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.59.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -7327,7 +7417,7 @@ dependencies = [
"futures-util",
"opentelemetry",
"percent-encoding",
"rand 0.9.2",
"rand 0.9.3",
"thiserror 2.0.18",
"tokio",
"tokio-stream",
@@ -7480,7 +7570,7 @@ dependencies = [
"heck 0.4.1",
"itertools 0.11.0",
"prost 0.12.6",
"prost-types",
"prost-types 0.12.6",
]
[[package]]
@@ -7494,7 +7584,7 @@ dependencies = [
"pbjson",
"pbjson-build",
"prost 0.12.6",
"prost-build",
"prost-build 0.12.6",
"serde",
]
@@ -7876,7 +7966,7 @@ checksum = "bee689443a2bd0a16ab0348b52ee43e3b2d1b1f931c8aa5c9f8de4c86fbe8c40"
dependencies = [
"bitflags 2.10.0",
"num-traits",
"rand 0.9.2",
"rand 0.9.3",
"rand_chacha 0.9.0",
"rand_xorshift",
"regex-syntax 0.8.8",
@@ -7918,7 +8008,26 @@ dependencies = [
"petgraph 0.6.5",
"prettyplease",
"prost 0.12.6",
"prost-types",
"prost-types 0.12.6",
"regex",
"syn 2.0.114",
"tempfile",
]
[[package]]
name = "prost-build"
version = "0.14.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "343d3bd7056eda839b03204e68deff7d1b13aba7af2b2fd16890697274262ee7"
dependencies = [
"heck 0.5.0",
"itertools 0.14.0",
"log",
"multimap",
"petgraph 0.8.3",
"prettyplease",
"prost 0.14.3",
"prost-types 0.14.3",
"regex",
"syn 2.0.114",
"tempfile",
@@ -7959,6 +8068,15 @@ dependencies = [
"prost 0.12.6",
]
[[package]]
name = "prost-types"
version = "0.14.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8991c4cbdb8bc5b11f0b074ffe286c30e523de90fee5ba8132f1399f23cb3dd7"
dependencies = [
"prost 0.14.3",
]
[[package]]
name = "psl"
version = "2.1.184"
@@ -8047,7 +8165,7 @@ dependencies = [
"bytes",
"getrandom 0.3.4",
"lru-slab",
"rand 0.9.2",
"rand 0.9.3",
"ring",
"rustc-hash 2.1.1",
"rustls",
@@ -8179,7 +8297,7 @@ dependencies = [
"rama-http-types",
"rama-net",
"rama-utils",
"rand 0.9.2",
"rand 0.9.3",
"serde",
"serde_html_form",
"serde_json",
@@ -8249,7 +8367,7 @@ dependencies = [
"rama-macros",
"rama-net",
"rama-utils",
"rand 0.9.2",
"rand 0.9.3",
"serde",
"sha1",
]
@@ -8277,7 +8395,7 @@ dependencies = [
"rama-error",
"rama-macros",
"rama-utils",
"rand 0.9.2",
"rand 0.9.3",
"serde",
"serde_json",
"sync_wrapper",
@@ -8351,7 +8469,7 @@ dependencies = [
"rama-http-types",
"rama-net",
"rama-utils",
"rand 0.9.2",
"rand 0.9.3",
"tokio",
]
@@ -8431,9 +8549,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.9.2"
version = "0.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
checksum = "7ec095654a25171c2124e9e3393a930bddbffdc939556c914957a4c3e0a87166"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.5",
@@ -8748,7 +8866,7 @@ dependencies = [
"pastey",
"pin-project-lite",
"process-wrap",
"rand 0.9.2",
"rand 0.9.3",
"reqwest",
"rmcp-macros",
"schemars 1.2.1",
@@ -8955,9 +9073,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "8279bb85272c9f10811ae6a6c547ff594d6a7f3c6c6b02ee9726d1d0dcfcdd06"
dependencies = [
"aws-lc-rs",
"ring",
@@ -9332,7 +9450,7 @@ version = "0.46.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26ab054c34b87f96c3e4701bea1888317cde30cc7e4a6136d2c48454ab96661c"
dependencies = [
"rand 0.9.2",
"rand 0.9.3",
"sentry-types",
"serde",
"serde_json",
@@ -9380,7 +9498,7 @@ checksum = "eecbd63e9d15a26a40675ed180d376fcb434635d2e33de1c24003f61e3e2230d"
dependencies = [
"debugid",
"hex",
"rand 0.9.2",
"rand 0.9.3",
"serde",
"serde_json",
"thiserror 2.0.18",
@@ -10867,8 +10985,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a286e33f82f8a1ee2df63f4fa35c0becf4a85a0cb03091a15fd7bf0b402dc94a"
dependencies = [
"async-trait",
"axum",
"base64 0.22.1",
"bytes",
"h2",
"http 1.4.0",
"http-body",
"http-body-util",
@@ -10878,6 +10998,7 @@ dependencies = [
"percent-encoding",
"pin-project",
"rustls-native-certs",
"socket2 0.6.2",
"sync_wrapper",
"tokio",
"tokio-rustls",
@@ -10888,6 +11009,18 @@ dependencies = [
"tracing",
]
[[package]]
name = "tonic-build"
version = "0.14.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27aac809edf60b741e2d7db6367214d078856b8a5bff0087e94ff330fb97b6fc"
dependencies = [
"prettyplease",
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "tonic-prost"
version = "0.14.3"
@@ -10899,6 +11032,22 @@ dependencies = [
"tonic",
]
[[package]]
name = "tonic-prost-build"
version = "0.14.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4556786613791cfef4ed134aa670b61a85cfcacf71543ef33e8d801abae988f"
dependencies = [
"prettyplease",
"proc-macro2",
"prost-build 0.14.3",
"prost-types 0.14.3",
"quote",
"syn 2.0.114",
"tempfile",
"tonic-build",
]
[[package]]
name = "tower"
version = "0.5.3"
@@ -11165,7 +11314,7 @@ dependencies = [
"http 1.4.0",
"httparse",
"log",
"rand 0.9.2",
"rand 0.9.3",
"rustls",
"rustls-pki-types",
"sha1",
@@ -11852,7 +12001,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.61.2",
]
[[package]]
codex-rs/Cargo.toml
+8
View File
@@ -13,6 +13,7 @@ members = [
"arg0",
"feedback",
"features",
"install-context",
"codex-backend-openapi-models",
"code-mode",
"cloud-requirements",
@@ -27,6 +28,7 @@ members = [
"shell-escalation",
"skills",
"core",
"core-plugins",
"core-skills",
"hooks",
"instructions",
@@ -127,6 +129,7 @@ codex-code-mode = { path = "code-mode" }
codex-config = { path = "config" }
codex-connectors = { path = "connectors" }
codex-core = { path = "core" }
codex-core-plugins = { path = "core-plugins" }
codex-core-skills = { path = "core-skills" }
codex-exec = { path = "exec" }
codex-exec-server = { path = "exec-server" }
@@ -134,6 +137,7 @@ codex-execpolicy = { path = "execpolicy" }
codex-experimental-api-macros = { path = "codex-experimental-api-macros" }
codex-features = { path = "features" }
codex-feedback = { path = "feedback" }
codex-install-context = { path = "install-context" }
codex-file-search = { path = "file-search" }
codex-git-utils = { path = "git-utils" }
codex-hooks = { path = "hooks" }
@@ -166,6 +170,7 @@ codex-state = { path = "state" }
codex-stdio-to-uds = { path = "stdio-to-uds" }
codex-terminal-detection = { path = "terminal-detection" }
codex-test-binary-support = { path = "test-binary-support" }
codex-thread-store = { path = "thread-store" }
codex-tools = { path = "tools" }
codex-tui = { path = "tui" }
codex-utils-absolute-path = { path = "utils/absolute-path" }
@@ -227,6 +232,7 @@ diffy = "0.4.2"
dirs = "6"
dotenvy = "0.15.7"
dunce = "1.0.4"
ed25519-dalek = { version = "2.2.0", features = ["pkcs8"] }
encoding_rs = "0.8.35"
env-flags = "0.1.1"
env_logger = "0.11.9"
@@ -341,6 +347,8 @@ tracing-appender = "0.2.3"
tracing-opentelemetry = "0.32.0"
tracing-subscriber = "0.3.22"
tracing-test = "0.2.5"
tonic = { version = "0.14.3", default-features = false, features = ["channel", "codegen"] }
tonic-prost = "0.14.3"
tree-sitter = "0.25.10"
tree-sitter-bash = "0.25"
ts-rs = "11"
codex-rs/README.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Codex CLI (Rust Implementation)
We provide Codex CLI as a standalone, native executable to ensure a zero-dependency install.
We provide Codex CLI as a standalone executable to ensure a zero-dependency install.
## Installing Codex
codex-rs/app-server-client/src/lib.rs
+9 -3
View File
@@ -28,6 +28,7 @@ use std::time::Duration;
pub use codex_app_server::in_process::DEFAULT_IN_PROCESS_CHANNEL_CAPACITY;
pub use codex_app_server::in_process::InProcessServerEvent;
use codex_app_server::in_process::InProcessStartArgs;
use codex_app_server::in_process::LogDbLayer;
use codex_app_server_protocol::ClientInfo;
use codex_app_server_protocol::ClientNotification;
use codex_app_server_protocol::ClientRequest;
@@ -64,9 +65,9 @@ pub use crate::remote::RemoteAppServerConnectArgs;
/// while legacy startup/config paths are migrated to RPCs.
pub mod legacy_core {
pub use codex_core::Cursor;
pub use codex_core::DEFAULT_PROJECT_DOC_FILENAME;
pub use codex_core::DEFAULT_AGENTS_MD_FILENAME;
pub use codex_core::INTERACTIVE_SESSION_SOURCES;
pub use codex_core::LOCAL_PROJECT_DOC_FILENAME;
pub use codex_core::LOCAL_AGENTS_MD_FILENAME;
pub use codex_core::McpManager;
pub use codex_core::PLUGIN_TEXT_MENTION_SIGIL;
pub use codex_core::RolloutRecorder;
@@ -76,7 +77,6 @@ pub mod legacy_core {
pub use codex_core::ThreadsPage;
pub use codex_core::append_message_history_entry;
pub use codex_core::check_execpolicy_for_warnings;
pub use codex_core::discover_project_doc_paths;
pub use codex_core::find_thread_meta_by_name_str;
pub use codex_core::find_thread_name_by_id;
pub use codex_core::find_thread_names_by_ids;
@@ -354,6 +354,8 @@ pub struct InProcessClientStartArgs {
pub cloud_requirements: CloudRequirementsLoader,
/// Feedback sink used by app-server/core telemetry and logs.
pub feedback: CodexFeedback,
/// SQLite tracing layer used to flush recently emitted logs before feedback upload.
pub log_db: Option<LogDbLayer>,
/// Environment manager used by core execution and filesystem operations.
pub environment_manager: Arc<EnvironmentManager>,
/// Startup warnings emitted after initialize succeeds.
@@ -405,6 +407,7 @@ impl InProcessClientStartArgs {
loader_overrides: self.loader_overrides,
cloud_requirements: self.cloud_requirements,
feedback: self.feedback,
log_db: self.log_db,
environment_manager: self.environment_manager,
config_warnings: self.config_warnings,
session_source: self.session_source,
@@ -967,6 +970,7 @@ mod tests {
match ConfigBuilder::default().build().await {
Ok(config) => config,
Err(_) => Config::load_default_with_cli_overrides(Vec::new())
.await
.expect("default config should load"),
}
}
@@ -982,6 +986,7 @@ mod tests {
loader_overrides: LoaderOverrides::default(),
cloud_requirements: CloudRequirementsLoader::default(),
feedback: CodexFeedback::new(),
log_db: None,
environment_manager: Arc::new(EnvironmentManager::new(/*exec_server_url*/ None)),
config_warnings: Vec::new(),
session_source,
@@ -1994,6 +1999,7 @@ mod tests {
loader_overrides: LoaderOverrides::default(),
cloud_requirements: CloudRequirementsLoader::default(),
feedback: CodexFeedback::new(),
log_db: None,
environment_manager: environment_manager.clone(),
config_warnings: Vec::new(),
session_source: SessionSource::Exec,
codex-rs/app-server-protocol/schema/json/ServerNotification.json
+6
View File
@@ -2911,6 +2911,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/codex_app_server_protocol.schemas.json
+6
View File
@@ -13453,6 +13453,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/codex_app_server_protocol.v2.schemas.json
+6
View File
@@ -11297,6 +11297,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ItemCompletedNotification.json
+6
View File
@@ -787,6 +787,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ItemStartedNotification.json
+6
View File
@@ -787,6 +787,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ReviewStartResponse.json
+6
View File
@@ -930,6 +930,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadForkResponse.json
+6
View File
@@ -1444,6 +1444,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadListResponse.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadMetadataUpdateResponse.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadReadResponse.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadResumeResponse.json
+6
View File
@@ -1444,6 +1444,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadRollbackResponse.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadStartResponse.json
+6
View File
@@ -1444,6 +1444,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadStartedNotification.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/ThreadUnarchiveResponse.json
+6
View File
@@ -1206,6 +1206,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/TurnCompletedNotification.json
+6
View File
@@ -930,6 +930,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/TurnStartResponse.json
+6
View File
@@ -930,6 +930,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/json/v2/TurnStartedNotification.json
+6
View File
@@ -930,6 +930,12 @@
"id": {
"type": "string"
},
"mcpAppResourceUri": {
"type": [
"string",
"null"
]
},
"result": {
"anyOf": [
{
codex-rs/app-server-protocol/schema/typescript/v2/ThreadItem.ts
+1 -1
View File
@@ -53,7 +53,7 @@ exitCode: number | null,
/**
* The duration of the command execution in milliseconds.
*/
durationMs: number | null, } | { "type": "fileChange", id: string, changes: Array<FileUpdateChange>, status: PatchApplyStatus, } | { "type": "mcpToolCall", id: string, server: string, tool: string, status: McpToolCallStatus, arguments: JsonValue, result: McpToolCallResult | null, error: McpToolCallError | null,
durationMs: number | null, } | { "type": "fileChange", id: string, changes: Array<FileUpdateChange>, status: PatchApplyStatus, } | { "type": "mcpToolCall", id: string, server: string, tool: string, status: McpToolCallStatus, arguments: JsonValue, mcpAppResourceUri?: string, result: McpToolCallResult | null, error: McpToolCallError | null,
/**
* The duration of the MCP tool call in milliseconds.
*/
codex-rs/app-server-protocol/src/protocol/common.rs
+6
View File
@@ -289,6 +289,11 @@ client_request_definitions! {
params: v2::ThreadMemoryModeSetParams,
response: v2::ThreadMemoryModeSetResponse,
},
#[experimental("memory/reset")]
MemoryReset => "memory/reset" {
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: v2::MemoryResetResponse,
},
ThreadUnarchive => "thread/unarchive" {
params: v2::ThreadUnarchiveParams,
response: v2::ThreadUnarchiveResponse,
@@ -764,6 +769,7 @@ macro_rules! server_notification_definitions {
Display,
ExperimentalApi,
)]
#[allow(clippy::large_enum_variant)]
#[serde(tag = "method", content = "params", rename_all = "camelCase")]
#[strum(serialize_all = "camelCase")]
pub enum ServerNotification {
codex-rs/app-server-protocol/src/protocol/thread_history.rs
+10 -4
View File
@@ -502,6 +502,7 @@ impl ThreadHistoryBuilder {
.arguments
.clone()
.unwrap_or(serde_json::Value::Null),
mcp_app_resource_uri: payload.mcp_app_resource_uri.clone(),
result: None,
error: None,
duration_ms: None,
@@ -518,11 +519,11 @@ impl ThreadHistoryBuilder {
let duration_ms = i64::try_from(payload.duration.as_millis()).ok();
let (result, error) = match &payload.result {
Ok(value) => (
Some(McpToolCallResult {
Some(Box::new(McpToolCallResult {
content: value.content.clone(),
structured_content: value.structured_content.clone(),
meta: value.meta.clone(),
}),
})),
None,
),
Err(message) => (
@@ -542,6 +543,7 @@ impl ThreadHistoryBuilder {
.arguments
.clone()
.unwrap_or(serde_json::Value::Null),
mcp_app_resource_uri: payload.mcp_app_resource_uri.clone(),
result,
error,
duration_ms,
@@ -1809,6 +1811,7 @@ mod tests {
tool: "lookup".into(),
arguments: Some(serde_json::json!({"id":"123"})),
},
mcp_app_resource_uri: None,
duration: Duration::from_millis(8),
result: Err("boom".into()),
}),
@@ -1857,6 +1860,7 @@ mod tests {
tool: "lookup".into(),
status: McpToolCallStatus::Failed,
arguments: serde_json::json!({"id":"123"}),
mcp_app_resource_uri: None,
result: None,
error: Some(McpToolCallError {
message: "boom".into(),
@@ -1882,6 +1886,7 @@ mod tests {
tool: "lookup".into(),
arguments: Some(serde_json::json!({"id":"123"})),
},
mcp_app_resource_uri: Some("ui://widget/lookup.html".into()),
duration: Duration::from_millis(8),
result: Ok(CallToolResult {
content: vec![serde_json::json!({
@@ -1911,7 +1916,8 @@ mod tests {
tool: "lookup".into(),
status: McpToolCallStatus::Completed,
arguments: serde_json::json!({"id":"123"}),
result: Some(McpToolCallResult {
mcp_app_resource_uri: Some("ui://widget/lookup.html".into()),
result: Some(Box::new(McpToolCallResult {
content: vec![serde_json::json!({
"type": "text",
"text": "result"
@@ -1920,7 +1926,7 @@ mod tests {
meta: Some(serde_json::json!({
"ui/resourceUri": "ui://widget/lookup.html"
})),
}),
})),
error: None,
duration_ms: Some(8),
}
codex-rs/app-server-protocol/src/protocol/v2.rs
+9 -1
View File
@@ -3145,6 +3145,11 @@ pub struct ThreadMemoryModeSetParams {
#[ts(export_to = "v2/")]
pub struct ThreadMemoryModeSetResponse {}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]
pub struct MemoryResetResponse {}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]
@@ -4611,7 +4616,10 @@ pub enum ThreadItem {
tool: String,
status: McpToolCallStatus,
arguments: JsonValue,
result: Option<McpToolCallResult>,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[ts(optional)]
mcp_app_resource_uri: Option<String>,
result: Option<Box<McpToolCallResult>>,
error: Option<McpToolCallError>,
/// The duration of the MCP tool call in milliseconds.
#[ts(type = "number | null")]
codex-rs/app-server/Cargo.toml
+2
View File
@@ -34,6 +34,7 @@ codex-arg0 = { workspace = true }
codex-cloud-requirements = { workspace = true }
codex-config = { workspace = true }
codex-core = { workspace = true }
codex-core-plugins = { workspace = true }
codex-exec-server = { workspace = true }
codex-features = { workspace = true }
codex-git-utils = { workspace = true }
@@ -54,6 +55,7 @@ codex-rmcp-client = { workspace = true }
codex-rollout = { workspace = true }
codex-sandboxing = { workspace = true }
codex-state = { workspace = true }
codex-thread-store = { workspace = true }
codex-tools = { workspace = true }
codex-utils-absolute-path = { workspace = true }
codex-utils-json-to-toml = { workspace = true }
codex-rs/app-server/README.md
+11
View File
@@ -41,8 +41,11 @@ Security note:
- Non-loopback websocket listeners currently allow unauthenticated connections by default during rollout. If you expose one remotely, configure websocket auth explicitly now.
- Supported auth modes are app-server flags:
- `--ws-auth capability-token --ws-token-file /absolute/path`
- `--ws-auth capability-token --ws-token-sha256 HEX`
- `--ws-auth signed-bearer-token --ws-shared-secret-file /absolute/path` for HMAC-signed JWT/JWS bearer tokens, with optional `--ws-issuer`, `--ws-audience`, `--ws-max-clock-skew-seconds`
- Clients present the credential as `Authorization: Bearer <token>` during the websocket handshake. Auth is enforced before JSON-RPC `initialize`.
- When starting `codex app-server` manually, prefer `--ws-token-file` over passing raw bearer tokens on the command line. Store a high-entropy token in a file readable only by your user, then have your client present that token in the websocket `Authorization` header.
- `--ws-token-sha256` is intended for clients that keep the raw token in a separate local secret store and only need the server to know the SHA-256 verifier. The hash may appear in process listings, but it is not sufficient to authenticate; clients still need the original raw token. Only use this mode with randomly generated high-entropy tokens, not passwords or other guessable values.
Tracing/log output:
@@ -141,6 +144,7 @@ Example with notification opt-out:
- `thread/read` — read a stored thread by id without resuming it; optionally include turns via `includeTurns`. The returned `thread` includes `status` (`ThreadStatus`), defaulting to `notLoaded` when the thread is not currently loaded.
- `thread/metadata/update` — patch stored thread metadata in sqlite; currently supports updating persisted `gitInfo` fields and returns the refreshed `thread`.
- `thread/memoryMode/set` — experimental; set a thread’s persisted memory eligibility to `"enabled"` or `"disabled"` for either a loaded thread or a stored rollout; returns `{}` on success.
- `memory/reset` — experimental; clear the current `CODEX_HOME/memories` directory and reset persisted memory stage data in sqlite while preserving existing thread memory modes; returns `{}` on success.
- `thread/status/changed` — notification emitted when a loaded thread’s status changes (`threadId` + new `status`).
- `thread/archive` — move a thread’s rollout file into the archived directory; returns `{}` on success and emits `thread/archived`.
- `thread/unsubscribe` — unsubscribe this connection from thread turn/item events. If this was the last subscriber, the server keeps the thread loaded and unloads it only after it has had no subscribers and no thread activity for 30 minutes, then emits `thread/closed`.
@@ -413,6 +417,13 @@ Experimental: use `thread/memoryMode/set` to change whether a thread remains eli
{ "id": 26, "result": {} }
```
Experimental: use `memory/reset` to clear local memory artifacts and sqlite-backed memory stage data for the current Codex home. This preserves existing thread memory modes; use `thread/memoryMode/set` separately when a thread's future memory eligibility should change.
```json
{ "method": "memory/reset", "id": 27 }
{ "id": 27, "result": {} }
```
### Example: Archive a thread
Use `thread/archive` to move the persisted rollout (stored as a JSONL file on disk) into the archived sessions directory.
codex-rs/app-server/src/bespoke_event_handling.rs
+14 -4
View File
@@ -2910,6 +2910,7 @@ async fn construct_mcp_tool_call_notification(
tool: begin_event.invocation.tool,
status: McpToolCallStatus::InProgress,
arguments: begin_event.invocation.arguments.unwrap_or(JsonValue::Null),
mcp_app_resource_uri: begin_event.mcp_app_resource_uri,
result: None,
error: None,
duration_ms: None,
@@ -2936,11 +2937,11 @@ async fn construct_mcp_tool_call_end_notification(
let (result, error) = match &end_event.result {
Ok(value) => (
Some(McpToolCallResult {
Some(Box::new(McpToolCallResult {
content: value.content.clone(),
structured_content: value.structured_content.clone(),
meta: value.meta.clone(),
}),
})),
None,
),
Err(message) => (
@@ -2957,6 +2958,7 @@ async fn construct_mcp_tool_call_end_notification(
tool: end_event.invocation.tool,
status,
arguments: end_event.invocation.arguments.unwrap_or(JsonValue::Null),
mcp_app_resource_uri: end_event.mcp_app_resource_uri,
result,
error,
duration_ms,
@@ -4220,6 +4222,7 @@ mod tests {
tool: "list_mcp_resources".to_string(),
arguments: Some(serde_json::json!({"server": ""})),
},
mcp_app_resource_uri: Some("ui://widget/list-resources.html".to_string()),
};
let thread_id = ThreadId::new().to_string();
@@ -4240,6 +4243,7 @@ mod tests {
tool: begin_event.invocation.tool,
status: McpToolCallStatus::InProgress,
arguments: serde_json::json!({"server": ""}),
mcp_app_resource_uri: Some("ui://widget/list-resources.html".to_string()),
result: None,
error: None,
duration_ms: None,
@@ -4380,6 +4384,7 @@ mod tests {
tool: "list_mcp_resources".to_string(),
arguments: None,
},
mcp_app_resource_uri: None,
};
let thread_id = ThreadId::new().to_string();
@@ -4400,6 +4405,7 @@ mod tests {
tool: begin_event.invocation.tool,
status: McpToolCallStatus::InProgress,
arguments: JsonValue::Null,
mcp_app_resource_uri: None,
result: None,
error: None,
duration_ms: None,
@@ -4431,6 +4437,7 @@ mod tests {
tool: "list_mcp_resources".to_string(),
arguments: Some(serde_json::json!({"server": ""})),
},
mcp_app_resource_uri: Some("ui://widget/list-resources.html".to_string()),
duration: Duration::from_nanos(92708),
result: Ok(result),
};
@@ -4453,13 +4460,14 @@ mod tests {
tool: end_event.invocation.tool,
status: McpToolCallStatus::Completed,
arguments: serde_json::json!({"server": ""}),
result: Some(McpToolCallResult {
mcp_app_resource_uri: Some("ui://widget/list-resources.html".to_string()),
result: Some(Box::new(McpToolCallResult {
content,
structured_content: None,
meta: Some(serde_json::json!({
"ui/resourceUri": "ui://widget/list-resources.html"
})),
}),
})),
error: None,
duration_ms: Some(0),
},
@@ -4477,6 +4485,7 @@ mod tests {
tool: "list_mcp_resources".to_string(),
arguments: None,
},
mcp_app_resource_uri: None,
duration: Duration::from_millis(1),
result: Err("boom".to_string()),
};
@@ -4499,6 +4508,7 @@ mod tests {
tool: end_event.invocation.tool,
status: McpToolCallStatus::Failed,
arguments: JsonValue::Null,
mcp_app_resource_uri: None,
result: None,
error: Some(McpToolCallError {
message: "boom".to_string(),
codex-rs/app-server/src/codex_message_processor.rs
+223 -418
View File
@@ -94,6 +94,7 @@ use codex_app_server_protocol::McpServerStatus;
use codex_app_server_protocol::McpServerStatusDetail;
use codex_app_server_protocol::McpServerToolCallParams;
use codex_app_server_protocol::McpServerToolCallResponse;
use codex_app_server_protocol::MemoryResetResponse;
use codex_app_server_protocol::MockExperimentalMethodParams;
use codex_app_server_protocol::MockExperimentalMethodResponse;
use codex_app_server_protocol::ModelListParams;
@@ -189,6 +190,7 @@ use codex_app_server_protocol::ThreadUnsubscribeStatus;
use codex_app_server_protocol::Turn;
use codex_app_server_protocol::TurnError;
use codex_app_server_protocol::TurnInterruptParams;
use codex_app_server_protocol::TurnInterruptResponse;
use codex_app_server_protocol::TurnStartParams;
use codex_app_server_protocol::TurnStartResponse;
use codex_app_server_protocol::TurnStatus;
@@ -207,7 +209,6 @@ use codex_chatgpt::connectors;
use codex_cloud_requirements::cloud_requirements_loader;
use codex_config::types::McpServerTransportConfig;
use codex_core::CodexThread;
use codex_core::Cursor as RolloutCursor;
use codex_core::ForkSnapshot;
use codex_core::NewThread;
use codex_core::RolloutRecorder;
@@ -215,8 +216,8 @@ use codex_core::SessionMeta;
use codex_core::SteerInputError;
use codex_core::ThreadConfigSnapshot;
use codex_core::ThreadManager;
use codex_core::ThreadSortKey as CoreThreadSortKey;
use codex_core::append_thread_name;
use codex_core::clear_memory_roots_contents;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::config::NetworkProxyAuditMetadata;
@@ -236,26 +237,25 @@ use codex_core::find_archived_thread_path_by_id_str;
use codex_core::find_thread_name_by_id;
use codex_core::find_thread_names_by_ids;
use codex_core::find_thread_path_by_id_str;
use codex_core::parse_cursor;
use codex_core::path_utils;
use codex_core::plugins::MarketplaceAddError;
use codex_core::plugins::MarketplaceError;
use codex_core::plugins::MarketplacePluginSource;
use codex_core::plugins::OPENAI_CURATED_MARKETPLACE_NAME;
use codex_core::plugins::PluginInstallError as CorePluginInstallError;
use codex_core::plugins::PluginInstallRequest;
use codex_core::plugins::PluginReadRequest;
use codex_core::plugins::PluginUninstallError as CorePluginUninstallError;
use codex_core::plugins::add_marketplace as add_marketplace_to_codex_home;
use codex_core::plugins::load_plugin_apps;
use codex_core::plugins::load_plugin_mcp_servers;
use codex_core::read_head_for_summary;
use codex_core::read_session_meta_line;
use codex_core::rollout_date_parts;
use codex_core::sandboxing::SandboxPermissions;
use codex_core::windows_sandbox::WindowsSandboxLevelExt;
use codex_core::windows_sandbox::WindowsSandboxSetupMode as CoreWindowsSandboxSetupMode;
use codex_core::windows_sandbox::WindowsSandboxSetupRequest;
use codex_core_plugins::loader::load_plugin_apps;
use codex_core_plugins::loader::load_plugin_mcp_servers;
use codex_core_plugins::manifest::PluginManifestInterface;
use codex_core_plugins::marketplace::MarketplaceError;
use codex_core_plugins::marketplace::MarketplacePluginSource;
use codex_exec_server::LOCAL_FS;
use codex_features::FEATURES;
use codex_features::Feature;
@@ -326,15 +326,20 @@ use codex_state::StateRuntime;
use codex_state::ThreadMetadata;
use codex_state::ThreadMetadataBuilder;
use codex_state::log_db::LogDbLayer;
use codex_thread_store::ArchiveThreadParams as StoreArchiveThreadParams;
use codex_thread_store::ListThreadsParams as StoreListThreadsParams;
use codex_thread_store::LocalThreadStore;
use codex_thread_store::ReadThreadParams as StoreReadThreadParams;
use codex_thread_store::StoredThread;
use codex_thread_store::ThreadSortKey as StoreThreadSortKey;
use codex_thread_store::ThreadStore;
use codex_thread_store::ThreadStoreError;
use codex_utils_absolute_path::AbsolutePathBuf;
use codex_utils_json_to_toml::json_to_toml;
use codex_utils_pty::DEFAULT_OUTPUT_BYTES_CAP;
use std::collections::BTreeMap;
use std::collections::HashMap;
use std::collections::HashSet;
use std::ffi::OsStr;
use std::fs::FileTimes;
use std::fs::OpenOptions;
use std::io::Error as IoError;
use std::path::Path;
use std::path::PathBuf;
@@ -344,7 +349,6 @@ use std::sync::atomic::AtomicBool;
use std::sync::atomic::Ordering;
use std::time::Duration;
use std::time::Instant;
use std::time::SystemTime;
use tokio::sync::Mutex;
use tokio::sync::broadcast;
use tokio::sync::oneshot;
@@ -448,6 +452,7 @@ pub(crate) struct CodexMessageProcessor {
analytics_events_client: AnalyticsEventsClient,
arg0_paths: Arg0DispatchPaths,
config: Arc<Config>,
thread_store: LocalThreadStore,
cli_overrides: Arc<RwLock<Vec<(String, TomlValue)>>>,
runtime_feature_enablement: Arc<RwLock<BTreeMap<String, bool>>>,
cloud_requirements: Arc<RwLock<CloudRequirementsLoader>>,
@@ -617,17 +622,9 @@ pub(crate) struct CodexMessageProcessorArgs {
impl CodexMessageProcessor {
async fn instruction_sources_from_config(config: &Config) -> Vec<AbsolutePathBuf> {
let mut paths: Vec<AbsolutePathBuf> =
config.user_instructions_path.iter().cloned().collect();
match codex_core::discover_project_doc_paths(config, LOCAL_FS.as_ref()).await {
Ok(project_doc_paths) => {
paths.extend(project_doc_paths);
}
Err(err) => {
tracing::warn!(error = %err, "failed to discover project docs for thread response");
}
}
paths
codex_core::AgentsMdManager::new(config)
.instruction_sources(LOCAL_FS.as_ref())
.await
}
pub(crate) fn handle_config_mutation(&self) {
@@ -706,6 +703,7 @@ impl CodexMessageProcessor {
outgoing: outgoing.clone(),
analytics_events_client,
arg0_paths,
thread_store: LocalThreadStore::new(codex_rollout::RolloutConfig::from_view(&config)),
config,
cli_overrides,
runtime_feature_enablement,
@@ -913,6 +911,10 @@ impl CodexMessageProcessor {
self.thread_memory_mode_set(to_connection_request_id(request_id), params)
.await;
}
ClientRequest::MemoryReset { request_id, params } => {
self.memory_reset(to_connection_request_id(request_id), params)
.await;
}
ClientRequest::ThreadUnarchive { request_id, params } => {
self.thread_unarchive(to_connection_request_id(request_id), params)
.await;
@@ -2512,6 +2514,7 @@ impl CodexMessageProcessor {
))
{
let trust_target = resolve_root_git_project_for_trust(config.cwd.as_path())
.await
.unwrap_or_else(|| config.cwd.to_path_buf());
let cli_overrides_with_trust;
let cli_overrides_for_reload = if let Err(err) =
@@ -2786,7 +2789,6 @@ impl CodexMessageProcessor {
}
async fn thread_archive(&self, request_id: ConnectionRequestId, params: ThreadArchiveParams) {
// TODO(jif) mostly rewrite this using sqlite after phase 1
let thread_id = match ThreadId::from_string(&params.thread_id) {
Ok(id) => id,
Err(err) => {
@@ -2800,32 +2802,28 @@ impl CodexMessageProcessor {
}
};
let rollout_path =
match find_thread_path_by_id_str(&self.config.codex_home, &thread_id.to_string()).await
{
Ok(Some(p)) => p,
Ok(None) => {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("no rollout found for thread id {thread_id}"),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
}
Err(err) => {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("failed to locate thread id {thread_id}: {err}"),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
}
};
let thread_id_str = thread_id.to_string();
match self.archive_thread_common(thread_id, &rollout_path).await {
if let Err(err) = self
.thread_store
.read_thread(StoreReadThreadParams {
thread_id,
include_archived: false,
include_history: false,
})
.await
{
self.outgoing
.send_error(request_id, thread_store_archive_error("archive", err))
.await;
return;
}
self.prepare_thread_for_archive(thread_id).await;
match self
.thread_store
.archive_thread(StoreArchiveThreadParams { thread_id })
.await
{
Ok(()) => {
let response = ThreadArchiveResponse {};
self.outgoing.send_response(request_id, response).await;
@@ -2837,7 +2835,9 @@ impl CodexMessageProcessor {
.await;
}
Err(err) => {
self.outgoing.send_error(request_id, err).await;
self.outgoing
.send_error(request_id, thread_store_archive_error("archive", err))
.await;
}
}
}
@@ -3123,6 +3123,50 @@ impl CodexMessageProcessor {
.await;
}
async fn memory_reset(&self, request_id: ConnectionRequestId, _params: Option<()>) {
let state_db = match StateRuntime::init(
self.config.sqlite_home.clone(),
self.config.model_provider_id.clone(),
)
.await
{
Ok(state_db) => state_db,
Err(err) => {
self.send_internal_error(
request_id,
format!("failed to open state db for memory reset: {err}"),
)
.await;
return;
}
};
if let Err(err) = state_db.clear_memory_data().await {
self.send_internal_error(
request_id,
format!("failed to clear memory rows in state db: {err}"),
)
.await;
return;
}
if let Err(err) = clear_memory_roots_contents(&self.config.codex_home).await {
self.send_internal_error(
request_id,
format!(
"failed to clear memory directories under {}: {err}",
self.config.codex_home.display()
),
)
.await;
return;
}
self.outgoing
.send_response(request_id, MemoryResetResponse {})
.await;
}
async fn thread_metadata_update(
&self,
request_id: ConnectionRequestId,
@@ -3426,7 +3470,6 @@ impl CodexMessageProcessor {
request_id: ConnectionRequestId,
params: ThreadUnarchiveParams,
) {
// TODO(jif) mostly rewrite this using sqlite after phase 1
let thread_id = match ThreadId::from_string(&params.thread_id) {
Ok(id) => id,
Err(err) => {
@@ -3440,152 +3483,21 @@ impl CodexMessageProcessor {
}
};
let archived_path = match find_archived_thread_path_by_id_str(
&self.config.codex_home,
&thread_id.to_string(),
)
.await
{
Ok(Some(path)) => path,
Ok(None) => {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("no archived rollout found for thread id {thread_id}"),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
}
Err(err) => {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("failed to locate archived thread id {thread_id}: {err}"),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
}
};
let rollout_path_display = archived_path.display().to_string();
let fallback_provider = self.config.model_provider_id.clone();
let state_db_ctx = get_state_db(&self.config).await;
let archived_folder = self
.config
.codex_home
.join(codex_core::ARCHIVED_SESSIONS_SUBDIR);
let result: Result<Thread, JSONRPCErrorError> = async {
let canonical_archived_dir = tokio::fs::canonicalize(&archived_folder).await.map_err(
|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!(
"failed to unarchive thread: unable to resolve archived directory: {err}"
),
data: None,
},
)?;
let canonical_rollout_path = tokio::fs::canonicalize(&archived_path).await;
let canonical_rollout_path = if let Ok(path) = canonical_rollout_path
&& path.starts_with(&canonical_archived_dir)
{
path
} else {
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{rollout_path_display}` must be in archived directory"
),
data: None,
});
};
let required_suffix = format!("{thread_id}.jsonl");
let Some(file_name) = canonical_rollout_path.file_name().map(OsStr::to_owned) else {
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("rollout path `{rollout_path_display}` missing file name"),
data: None,
});
};
if !file_name
.to_string_lossy()
.ends_with(required_suffix.as_str())
{
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{rollout_path_display}` does not match thread id {thread_id}"
),
data: None,
});
}
let Some((year, month, day)) = rollout_date_parts(&file_name) else {
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{rollout_path_display}` missing filename timestamp"
),
data: None,
});
};
let sessions_folder = self.config.codex_home.join(codex_core::SESSIONS_SUBDIR);
let dest_dir = sessions_folder.join(year).join(month).join(day);
let restored_path = dest_dir.join(&file_name);
tokio::fs::create_dir_all(&dest_dir)
.await
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to unarchive thread: {err}"),
data: None,
})?;
tokio::fs::rename(&canonical_rollout_path, &restored_path)
.await
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to unarchive thread: {err}"),
data: None,
})?;
tokio::task::spawn_blocking({
let restored_path = restored_path.clone();
move || -> std::io::Result<()> {
let times = FileTimes::new().set_modified(SystemTime::now());
OpenOptions::new()
.append(true)
.open(&restored_path)?
.set_times(times)?;
Ok(())
}
})
let result = self
.thread_store
.unarchive_thread(StoreArchiveThreadParams { thread_id })
.await
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to update unarchived thread timestamp: {err}"),
data: None,
})?
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to update unarchived thread timestamp: {err}"),
data: None,
})?;
if let Some(ctx) = state_db_ctx {
let _ = ctx
.mark_unarchived(thread_id, restored_path.as_path())
.await;
}
let summary =
read_summary_from_rollout(restored_path.as_path(), fallback_provider.as_str())
.await
.map_err(|err| JSONRPCErrorError {
.map_err(|err| thread_store_archive_error("unarchive", err))
.and_then(|stored_thread| {
summary_from_stored_thread(stored_thread, fallback_provider.as_str())
.map(|summary| summary_to_thread(summary, &self.config.cwd))
.ok_or_else(|| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to read unarchived thread: {err}"),
message: format!("failed to read unarchived thread {thread_id}"),
data: None,
})?;
Ok(summary_to_thread(summary, &self.config.cwd))
}
.await;
})
});
match result {
Ok(mut thread) => {
@@ -3811,15 +3723,15 @@ impl CodexMessageProcessor {
.map(|value| value as usize)
.unwrap_or(THREAD_LIST_DEFAULT_LIMIT)
.clamp(1, THREAD_LIST_MAX_LIMIT);
let core_sort_key = match sort_key.unwrap_or(ThreadSortKey::CreatedAt) {
ThreadSortKey::CreatedAt => CoreThreadSortKey::CreatedAt,
ThreadSortKey::UpdatedAt => CoreThreadSortKey::UpdatedAt,
let store_sort_key = match sort_key.unwrap_or(ThreadSortKey::CreatedAt) {
ThreadSortKey::CreatedAt => StoreThreadSortKey::CreatedAt,
ThreadSortKey::UpdatedAt => StoreThreadSortKey::UpdatedAt,
};
let (summaries, next_cursor) = match self
.list_threads_common(
requested_page_size,
cursor,
core_sort_key,
store_sort_key,
ThreadListFilters {
model_providers,
source_kinds,
@@ -5063,7 +4975,7 @@ impl CodexMessageProcessor {
&self,
requested_page_size: usize,
cursor: Option<String>,
sort_key: CoreThreadSortKey,
sort_key: StoreThreadSortKey,
filters: ThreadListFilters,
) -> Result<(Vec<ConversationSummary>, Option<String>), JSONRPCErrorError> {
let ThreadListFilters {
@@ -5073,16 +4985,7 @@ impl CodexMessageProcessor {
cwd,
search_term,
} = filters;
let mut cursor_obj: Option<RolloutCursor> = match cursor.as_ref() {
Some(cursor_str) => {
Some(parse_cursor(cursor_str).ok_or_else(|| JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!("invalid cursor: {cursor_str}"),
data: None,
})?)
}
None => None,
};
let mut cursor_obj = cursor;
let mut last_cursor = cursor_obj.clone();
let mut remaining = requested_page_size;
let mut items = Vec::with_capacity(requested_page_size);
@@ -5101,54 +5004,26 @@ impl CodexMessageProcessor {
let fallback_provider = self.config.model_provider_id.clone();
let (allowed_sources_vec, source_kind_filter) = compute_source_filters(source_kinds);
let allowed_sources = allowed_sources_vec.as_slice();
let state_db_ctx = get_state_db(&self.config).await;
while remaining > 0 {
let page_size = remaining.min(THREAD_LIST_MAX_LIMIT);
let page = if archived {
RolloutRecorder::list_archived_threads(
&self.config,
let page = self
.thread_store
.list_threads(StoreListThreadsParams {
page_size,
cursor_obj.as_ref(),
cursor: cursor_obj.clone(),
sort_key,
allowed_sources,
model_provider_filter.as_deref(),
fallback_provider.as_str(),
search_term.as_deref(),
)
allowed_sources: allowed_sources.to_vec(),
model_providers: model_provider_filter.clone(),
archived,
search_term: search_term.clone(),
})
.await
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to list threads: {err}"),
data: None,
})?
} else {
RolloutRecorder::list_threads(
&self.config,
page_size,
cursor_obj.as_ref(),
sort_key,
allowed_sources,
model_provider_filter.as_deref(),
fallback_provider.as_str(),
search_term.as_deref(),
)
.await
.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to list threads: {err}"),
data: None,
})?
};
.map_err(thread_store_list_error)?;
let mut filtered = Vec::with_capacity(page.items.len());
for it in page.items {
let Some(summary) = summary_from_thread_list_item(
it,
fallback_provider.as_str(),
state_db_ctx.as_ref(),
)
.await
let Some(summary) = summary_from_stored_thread(it, fallback_provider.as_str())
else {
continue;
};
@@ -5168,12 +5043,8 @@ impl CodexMessageProcessor {
items.extend(filtered);
remaining = requested_page_size.saturating_sub(items.len());
// Encode RolloutCursor into the JSON-RPC string form returned to clients.
let next_cursor_value = page.next_cursor.clone();
next_cursor = next_cursor_value
.as_ref()
.and_then(|cursor| serde_json::to_value(cursor).ok())
.and_then(|value| value.as_str().map(str::to_owned));
next_cursor = next_cursor_value.clone();
if remaining == 0 {
break;
}
@@ -5974,76 +5845,10 @@ impl CodexMessageProcessor {
.await;
}
async fn archive_thread_common(
&self,
thread_id: ThreadId,
rollout_path: &Path,
) -> Result<(), JSONRPCErrorError> {
// Verify rollout_path is under sessions dir.
let rollout_folder = self.config.codex_home.join(codex_core::SESSIONS_SUBDIR);
let canonical_sessions_dir = match tokio::fs::canonicalize(&rollout_folder).await {
Ok(path) => path,
Err(err) => {
return Err(JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!(
"failed to archive thread: unable to resolve sessions directory: {err}"
),
data: None,
});
}
};
let canonical_rollout_path = tokio::fs::canonicalize(rollout_path).await;
let canonical_rollout_path = if let Ok(path) = canonical_rollout_path
&& path.starts_with(&canonical_sessions_dir)
{
path
} else {
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{}` must be in sessions directory",
rollout_path.display()
),
data: None,
});
};
// Verify file name matches thread id.
let required_suffix = format!("{thread_id}.jsonl");
let Some(file_name) = canonical_rollout_path.file_name().map(OsStr::to_owned) else {
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{}` missing file name",
rollout_path.display()
),
data: None,
});
};
if !file_name
.to_string_lossy()
.ends_with(required_suffix.as_str())
{
return Err(JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: format!(
"rollout path `{}` does not match thread id {thread_id}",
rollout_path.display()
),
data: None,
});
}
let mut state_db_ctx = None;
async fn prepare_thread_for_archive(&self, thread_id: ThreadId) {
// If the thread is active, request shutdown and wait briefly.
let removed_conversation = self.thread_manager.remove_thread(&thread_id).await;
if let Some(conversation) = removed_conversation {
if let Some(ctx) = conversation.state_db() {
state_db_ctx = Some(ctx);
}
info!("thread {thread_id} was active; shutting down");
match Self::wait_for_thread_shutdown(&conversation).await {
ThreadShutdownResult::Complete => {}
@@ -6058,34 +5863,6 @@ impl CodexMessageProcessor {
}
}
self.finalize_thread_teardown(thread_id).await;
if state_db_ctx.is_none() {
state_db_ctx = get_state_db(&self.config).await;
}
// Move the rollout file to archived.
let result: std::io::Result<()> = async move {
let archive_folder = self
.config
.codex_home
.join(codex_core::ARCHIVED_SESSIONS_SUBDIR);
tokio::fs::create_dir_all(&archive_folder).await?;
let archived_path = archive_folder.join(&file_name);
tokio::fs::rename(&canonical_rollout_path, &archived_path).await?;
if let Some(ctx) = state_db_ctx {
let _ = ctx
.mark_archived(thread_id, archived_path.as_path(), Utc::now())
.await;
}
Ok(())
}
.await;
result.map_err(|err| JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to archive thread: {err}"),
data: None,
})
}
async fn apps_list(&self, request_id: ConnectionRequestId, params: AppsListParams) {
@@ -6470,7 +6247,7 @@ impl CodexMessageProcessor {
force_remote_sync,
} = params;
let roots = cwds.unwrap_or_default();
plugins_manager.maybe_start_non_curated_plugin_cache_refresh_for_roots(&roots);
plugins_manager.maybe_start_non_curated_plugin_cache_refresh(&roots);
let mut config = match self.load_latest_config(/*fallback_cwd*/ None).await {
Ok(config) => config,
@@ -7817,9 +7594,12 @@ impl CodexMessageProcessor {
async fn turn_interrupt(&self, request_id: ConnectionRequestId, params: TurnInterruptParams) {
let TurnInterruptParams { thread_id, turn_id } = params;
self.outgoing
.record_request_turn_id(&request_id, &turn_id)
.await;
let is_startup_interrupt = turn_id.is_empty();
if !is_startup_interrupt {
self.outgoing
.record_request_turn_id(&request_id, &turn_id)
.await;
}
let (thread_uuid, thread) = match self.load_thread(&thread_id).await {
Ok(v) => v,
@@ -7829,21 +7609,48 @@ impl CodexMessageProcessor {
}
};
let request = request_id.clone();
// Record the pending interrupt so we can reply when TurnAborted arrives.
{
// Record turn interrupts so we can reply when TurnAborted arrives. Startup
// interrupts do not have a turn and are acknowledged after submission.
if !is_startup_interrupt {
let thread_state = self.thread_state_manager.thread_state(thread_uuid).await;
let mut thread_state = thread_state.lock().await;
thread_state
.pending_interrupts
.push((request, ApiVersion::V2));
.push((request_id.clone(), ApiVersion::V2));
}
// Submit the interrupt; we'll respond upon TurnAborted.
let _ = self
// Submit the interrupt. Turn interrupts respond upon TurnAborted; startup
// interrupts respond here because startup cancellation has no turn event.
let submit_result = self
.submit_core_op(&request_id, thread.as_ref(), Op::Interrupt)
.await;
match submit_result {
Ok(_) if is_startup_interrupt => {
self.outgoing
.send_response(request_id, TurnInterruptResponse {})
.await;
}
Ok(_) => {}
Err(err) => {
if !is_startup_interrupt {
let thread_state = self.thread_state_manager.thread_state(thread_uuid).await;
let mut thread_state = thread_state.lock().await;
thread_state
.pending_interrupts
.retain(|(pending_request_id, _)| pending_request_id != &request_id);
}
let interrupt_target = if is_startup_interrupt {
"startup"
} else {
"turn"
};
self.send_internal_error(
request_id,
format!("failed to interrupt {interrupt_target}: {err}"),
)
.await;
}
}
}
async fn ensure_conversation_listener(
@@ -9093,9 +8900,7 @@ fn plugin_skills_to_info(
.collect()
}
fn plugin_interface_to_info(
interface: codex_core::plugins::PluginManifestInterface,
) -> PluginInterface {
fn plugin_interface_to_info(interface: PluginManifestInterface) -> PluginInterface {
PluginInterface {
display_name: interface.display_name,
short_description: interface.short_description,
@@ -9413,67 +9218,67 @@ fn set_thread_name_from_title(thread: &mut Thread, title: String) {
thread.name = Some(title);
}
async fn summary_from_thread_list_item(
it: codex_core::ThreadItem,
fallback_provider: &str,
state_db_ctx: Option<&StateDbHandle>,
) -> Option<ConversationSummary> {
if let Some(thread_id) = it.thread_id {
let timestamp = it.created_at.clone();
let updated_at = it.updated_at.clone().or_else(|| timestamp.clone());
let model_provider = it
.model_provider
.clone()
.unwrap_or_else(|| fallback_provider.to_string());
let cwd = it.cwd?;
let cli_version = it.cli_version.unwrap_or_default();
let source = with_thread_spawn_agent_metadata(
it.source
.unwrap_or(codex_protocol::protocol::SessionSource::Unknown),
it.agent_nickname.clone(),
it.agent_role.clone(),
);
return Some(ConversationSummary {
conversation_id: thread_id,
path: it.path,
preview: it.first_user_message.unwrap_or_default(),
timestamp,
updated_at,
model_provider,
cwd,
cli_version,
source,
git_info: if it.git_sha.is_none()
&& it.git_branch.is_none()
&& it.git_origin_url.is_none()
{
None
} else {
Some(ConversationGitInfo {
sha: it.git_sha,
branch: it.git_branch,
origin_url: it.git_origin_url,
})
},
});
fn thread_store_list_error(err: ThreadStoreError) -> JSONRPCErrorError {
match err {
ThreadStoreError::InvalidRequest { message } => JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message,
data: None,
},
err => JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to list threads: {err}"),
data: None,
},
}
if let Some(thread_id) = thread_id_from_rollout_path(it.path.as_path()) {
return read_summary_from_state_db_context_by_thread_id(state_db_ctx, thread_id).await;
}
None
}
fn thread_id_from_rollout_path(path: &Path) -> Option<ThreadId> {
let file_name = path.file_name()?.to_str()?;
let stem = file_name.strip_suffix(".jsonl")?;
if stem.len() < 37 {
return None;
fn thread_store_archive_error(operation: &str, err: ThreadStoreError) -> JSONRPCErrorError {
match err {
ThreadStoreError::InvalidRequest { message } => JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message,
data: None,
},
err => JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to {operation} thread: {err}"),
data: None,
},
}
let uuid_start = stem.len().saturating_sub(36);
if !stem[..uuid_start].ends_with('-') {
return None;
}
ThreadId::from_string(&stem[uuid_start..]).ok()
}
fn summary_from_stored_thread(
thread: StoredThread,
fallback_provider: &str,
) -> Option<ConversationSummary> {
let path = thread.rollout_path?;
let source = with_thread_spawn_agent_metadata(
thread.source,
thread.agent_nickname.clone(),
thread.agent_role.clone(),
);
let git_info = thread.git_info.map(|git| ConversationGitInfo {
sha: git.commit_hash.map(|sha| sha.0),
branch: git.branch,
origin_url: git.repository_url,
});
Some(ConversationSummary {
conversation_id: thread.thread_id,
path,
preview: thread.first_user_message.unwrap_or(thread.preview),
timestamp: Some(thread.created_at.to_rfc3339_opts(SecondsFormat::Secs, true)),
updated_at: Some(thread.updated_at.to_rfc3339_opts(SecondsFormat::Secs, true)),
model_provider: if thread.model_provider.is_empty() {
fallback_provider.to_string()
} else {
thread.model_provider
},
cwd: thread.cwd,
cli_version: thread.cli_version,
source,
git_info,
})
}
#[allow(clippy::too_many_arguments)]
codex-rs/app-server/src/config_api.rs
+2 -2
View File
@@ -27,8 +27,8 @@ use codex_core::config_loader::LoaderOverrides;
use codex_core::config_loader::ResidencyRequirement as CoreResidencyRequirement;
use codex_core::config_loader::SandboxModeRequirement as CoreSandboxModeRequirement;
use codex_core::plugins::PluginId;
use codex_core::plugins::collect_plugin_enabled_candidates;
use codex_core::plugins::installed_plugin_telemetry_metadata;
use codex_core_plugins::loader::installed_plugin_telemetry_metadata;
use codex_core_plugins::toggles::collect_plugin_enabled_candidates;
use codex_features::canonical_feature_for_key;
use codex_features::feature_for_key;
use codex_protocol::config_types::WebSearchMode;
codex-rs/app-server/src/external_agent_config_api.rs
+1
View File
@@ -37,6 +37,7 @@ impl ExternalAgentConfigApi {
include_home: params.include_home,
cwds: params.cwds,
})
.await
.map_err(map_io_error)?;
Ok(ExternalAgentConfigDetectResponse {
codex-rs/app-server/src/in_process.rs
+6 -1
View File
@@ -82,6 +82,7 @@ use codex_exec_server::EnvironmentManager;
use codex_feedback::CodexFeedback;
use codex_login::AuthManager;
use codex_protocol::protocol::SessionSource;
pub use codex_state::log_db::LogDbLayer;
use tokio::sync::mpsc;
use tokio::sync::oneshot;
use tokio::time::timeout;
@@ -117,6 +118,8 @@ pub struct InProcessStartArgs {
pub cloud_requirements: CloudRequirementsLoader,
/// Feedback sink used by app-server/core telemetry and logs.
pub feedback: CodexFeedback,
/// SQLite tracing layer used to flush recently emitted logs before feedback upload.
pub log_db: Option<LogDbLayer>,
/// Environment manager used by core execution and filesystem operations.
pub environment_manager: Arc<EnvironmentManager>,
/// Startup warnings emitted after initialize succeeds.
@@ -395,7 +398,7 @@ fn start_uninitialized(args: InProcessStartArgs) -> InProcessClientHandle {
loader_overrides: args.loader_overrides,
cloud_requirements: args.cloud_requirements,
feedback: args.feedback,
log_db: None,
log_db: args.log_db,
config_warnings: args.config_warnings,
session_source: args.session_source,
auth_manager,
@@ -713,6 +716,7 @@ mod tests {
match ConfigBuilder::default().build().await {
Ok(config) => config,
Err(_) => Config::load_default_with_cli_overrides(Vec::new())
.await
.expect("default config should load"),
}
}
@@ -728,6 +732,7 @@ mod tests {
loader_overrides: LoaderOverrides::default(),
cloud_requirements: CloudRequirementsLoader::default(),
feedback: CodexFeedback::new(),
log_db: None,
environment_manager: Arc::new(EnvironmentManager::new(/*exec_server_url*/ None)),
config_warnings: Vec::new(),
session_source,
codex-rs/app-server/src/lib.rs
+8 -6
View File
@@ -432,12 +432,14 @@ pub async fn run_main_with_transport(
Err(err) => {
let message = config_warning_from_error("Invalid configuration; using defaults.", &err);
config_warnings.push(message);
Config::load_default_with_cli_overrides(cli_kv_overrides.clone()).map_err(|e| {
std::io::Error::new(
ErrorKind::InvalidData,
format!("error loading default config after config error: {e}"),
)
})?
Config::load_default_with_cli_overrides(cli_kv_overrides.clone())
.await
.map_err(|e| {
std::io::Error::new(
ErrorKind::InvalidData,
format!("error loading default config after config error: {e}"),
)
})?
}
};
codex-rs/app-server/src/message_processor.rs
+1 -2
View File
@@ -725,8 +725,6 @@ impl MessageProcessor {
session: Arc<ConnectionSessionState>,
request_context: RequestContext,
) {
let connection_id = connection_request_id.connection_id;
if !session.initialized() {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
@@ -748,6 +746,7 @@ impl MessageProcessor {
self.outgoing.send_error(connection_request_id, error).await;
return;
}
let connection_id = connection_request_id.connection_id;
if self.config.features.enabled(Feature::GeneralAnalytics)
&& let ClientRequest::TurnStart { request_id, .. }
| ClientRequest::TurnSteer { request_id, .. } = &codex_request
codex-rs/app-server/src/transport/auth.rs
+157 -18
View File
@@ -34,6 +34,10 @@ pub struct AppServerWebsocketAuthArgs {
#[arg(long = "ws-token-file", value_name = "PATH")]
pub ws_token_file: Option<PathBuf>,
/// Hex-encoded SHA-256 digest of the capability token.
#[arg(long = "ws-token-sha256", value_name = "HEX")]
pub ws_token_sha256: Option<String>,
/// Absolute path to the shared secret file for signed JWT bearer tokens.
#[arg(long = "ws-shared-secret-file", value_name = "PATH")]
pub ws_shared_secret_file: Option<PathBuf>,
@@ -65,7 +69,7 @@ pub struct AppServerWebsocketAuthSettings {
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AppServerWebsocketAuthConfig {
CapabilityToken {
token_file: AbsolutePathBuf,
source: AppServerWebsocketCapabilityTokenSource,
},
SignedBearerToken {
shared_secret_file: AbsolutePathBuf,
@@ -75,6 +79,12 @@ pub enum AppServerWebsocketAuthConfig {
},
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AppServerWebsocketCapabilityTokenSource {
TokenFile { token_file: AbsolutePathBuf },
TokenSha256 { token_sha256: [u8; 32] },
}
#[derive(Clone, Debug, Default)]
pub(crate) struct WebsocketAuthPolicy {
pub(crate) mode: Option<WebsocketAuthMode>,
@@ -144,17 +154,34 @@ impl AppServerWebsocketAuthArgs {
"`--ws-shared-secret-file`, `--ws-issuer`, `--ws-audience`, and `--ws-max-clock-skew-seconds` require `--ws-auth signed-bearer-token`"
);
}
let token_file = self.ws_token_file.context(
"`--ws-token-file` is required when `--ws-auth capability-token` is set",
)?;
Some(AppServerWebsocketAuthConfig::CapabilityToken {
token_file: absolute_path_arg("--ws-token-file", token_file)?,
})
let source = match (self.ws_token_file, self.ws_token_sha256) {
(Some(_), Some(_)) => {
anyhow::bail!(
"`--ws-token-file` and `--ws-token-sha256` are mutually exclusive"
);
}
(Some(token_file), None) => {
AppServerWebsocketCapabilityTokenSource::TokenFile {
token_file: absolute_path_arg("--ws-token-file", token_file)?,
}
}
(None, Some(token_sha256)) => {
AppServerWebsocketCapabilityTokenSource::TokenSha256 {
token_sha256: sha256_digest_arg("--ws-token-sha256", &token_sha256)?,
}
}
(None, None) => {
anyhow::bail!(
"`--ws-token-file` or `--ws-token-sha256` is required when `--ws-auth capability-token` is set"
);
}
};
Some(AppServerWebsocketAuthConfig::CapabilityToken { source })
}
Some(WebsocketAuthCliMode::SignedBearerToken) => {
if self.ws_token_file.is_some() {
if self.ws_token_file.is_some() || self.ws_token_sha256.is_some() {
anyhow::bail!(
"`--ws-token-file` requires `--ws-auth capability-token`, not `signed-bearer-token`"
"`--ws-token-file` and `--ws-token-sha256` require `--ws-auth capability-token`, not `signed-bearer-token`"
);
}
let shared_secret_file = self.ws_shared_secret_file.context(
@@ -174,6 +201,7 @@ impl AppServerWebsocketAuthArgs {
}
None => {
if self.ws_token_file.is_some()
|| self.ws_token_sha256.is_some()
|| self.ws_shared_secret_file.is_some()
|| self.ws_issuer.is_some()
|| self.ws_audience.is_some()
@@ -195,12 +223,19 @@ pub(crate) fn policy_from_settings(
settings: &AppServerWebsocketAuthSettings,
) -> io::Result<WebsocketAuthPolicy> {
let mode = match settings.config.as_ref() {
Some(AppServerWebsocketAuthConfig::CapabilityToken { token_file }) => {
let token = read_trimmed_secret(token_file.as_ref())?;
Some(WebsocketAuthMode::CapabilityToken {
token_sha256: sha256_digest(token.as_bytes()),
})
}
Some(AppServerWebsocketAuthConfig::CapabilityToken { source }) => match source {
AppServerWebsocketCapabilityTokenSource::TokenFile { token_file } => {
let token = read_trimmed_secret(token_file.as_ref())?;
Some(WebsocketAuthMode::CapabilityToken {
token_sha256: sha256_digest(token.as_bytes()),
})
}
AppServerWebsocketCapabilityTokenSource::TokenSha256 { token_sha256 } => {
Some(WebsocketAuthMode::CapabilityToken {
token_sha256: *token_sha256,
})
}
},
Some(AppServerWebsocketAuthConfig::SignedBearerToken {
shared_secret_file,
issuer,
@@ -387,6 +422,30 @@ fn absolute_path_arg(flag_name: &str, path: PathBuf) -> anyhow::Result<AbsoluteP
AbsolutePathBuf::try_from(path).with_context(|| format!("{flag_name} must be an absolute path"))
}
fn sha256_digest_arg(flag_name: &str, value: &str) -> anyhow::Result<[u8; 32]> {
let trimmed = value.trim();
if trimmed.len() != 64 {
anyhow::bail!("{flag_name} must be a 64-character hex SHA-256 digest");
}
let mut digest = [0u8; 32];
for (index, pair) in trimmed.as_bytes().chunks_exact(2).enumerate() {
let high = hex_nibble(flag_name, pair[0])?;
let low = hex_nibble(flag_name, pair[1])?;
digest[index] = (high << 4) | low;
}
Ok(digest)
}
fn hex_nibble(flag_name: &str, byte: u8) -> anyhow::Result<u8> {
match byte {
b'0'..=b'9' => Ok(byte - b'0'),
b'a'..=b'f' => Ok(byte - b'a' + 10),
b'A'..=b'F' => Ok(byte - b'A' + 10),
_ => anyhow::bail!("{flag_name} must be a 64-character hex SHA-256 digest"),
}
}
fn sha256_digest(input: &[u8]) -> [u8; 32] {
let mut digest = [0u8; 32];
digest.copy_from_slice(&Sha256::digest(input));
@@ -403,6 +462,7 @@ fn unauthorized(message: &'static str) -> WebsocketAuthError {
#[cfg(test)]
mod tests {
use super::*;
use axum::http::HeaderValue;
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use hmac::Hmac;
@@ -443,19 +503,98 @@ mod tests {
}
#[test]
fn capability_token_args_require_token_file() {
fn capability_token_args_require_token_file_or_hash() {
let err = AppServerWebsocketAuthArgs {
ws_auth: Some(WebsocketAuthCliMode::CapabilityToken),
..Default::default()
}
.try_into_settings()
.expect_err("capability-token mode should require a token file");
.expect_err("capability-token mode should require a token source");
assert!(
err.to_string().contains("--ws-token-file"),
err.to_string().contains("--ws-token-file")
&& err.to_string().contains("--ws-token-sha256"),
"unexpected error: {err}"
);
}
#[test]
fn capability_token_args_accept_token_hash() {
let settings = AppServerWebsocketAuthArgs {
ws_auth: Some(WebsocketAuthCliMode::CapabilityToken),
ws_token_sha256: Some("ab".repeat(32)),
..Default::default()
}
.try_into_settings()
.expect("capability-token hash args should parse");
assert_eq!(
settings,
AppServerWebsocketAuthSettings {
config: Some(AppServerWebsocketAuthConfig::CapabilityToken {
source: AppServerWebsocketCapabilityTokenSource::TokenSha256 {
token_sha256: [0xab; 32],
},
}),
}
);
}
#[test]
fn capability_token_args_reject_multiple_token_sources() {
let err = AppServerWebsocketAuthArgs {
ws_auth: Some(WebsocketAuthCliMode::CapabilityToken),
ws_token_file: Some(PathBuf::from("/tmp/token")),
ws_token_sha256: Some("ab".repeat(32)),
..Default::default()
}
.try_into_settings()
.expect_err("capability-token mode should reject multiple token sources");
assert!(
err.to_string().contains("mutually exclusive"),
"unexpected error: {err}"
);
}
#[test]
fn capability_token_args_reject_malformed_token_hash() {
let err = AppServerWebsocketAuthArgs {
ws_auth: Some(WebsocketAuthCliMode::CapabilityToken),
ws_token_sha256: Some("not-a-sha256".to_string()),
..Default::default()
}
.try_into_settings()
.expect_err("capability-token mode should reject malformed token hashes");
assert!(
err.to_string().contains("64-character hex"),
"unexpected error: {err}"
);
}
#[test]
fn capability_token_hash_policy_authorizes_matching_bearer_token() {
let settings = AppServerWebsocketAuthSettings {
config: Some(AppServerWebsocketAuthConfig::CapabilityToken {
source: AppServerWebsocketCapabilityTokenSource::TokenSha256 {
token_sha256: sha256_digest(b"super-secret-token"),
},
}),
};
let policy = policy_from_settings(&settings).expect("hash policy should build");
let mut headers = HeaderMap::new();
headers.insert(
AUTHORIZATION,
HeaderValue::from_static("Bearer super-secret-token"),
);
authorize_upgrade(&headers, &policy).expect("matching token should authorize");
headers.insert(
AUTHORIZATION,
HeaderValue::from_static("Bearer wrong-token"),
);
let err = authorize_upgrade(&headers, &policy).expect_err("wrong token should fail");
assert_eq!(err.status_code(), StatusCode::UNAUTHORIZED);
}
#[test]
fn signed_bearer_args_require_mode_when_mode_specific_flags_are_set() {
let err = AppServerWebsocketAuthArgs {
codex-rs/app-server/src/transport/remote_control/mod.rs
-14
View File
@@ -4,7 +4,6 @@ mod protocol;
mod websocket;
use crate::transport::remote_control::websocket::RemoteControlWebsocket;
use crate::transport::remote_control::websocket::load_remote_control_auth;
pub use self::protocol::ClientId;
use self::protocol::ServerEvent;
@@ -59,9 +58,6 @@ pub(crate) async fn start_remote_control(
} else {
None
};
if initial_enabled {
validate_remote_control_auth(&auth_manager).await?;
}
let (enabled_tx, enabled_rx) = watch::channel(initial_enabled);
let join_handle = tokio::spawn(async move {
@@ -86,15 +82,5 @@ pub(crate) async fn start_remote_control(
))
}
pub(crate) async fn validate_remote_control_auth(
auth_manager: &Arc<AuthManager>,
) -> io::Result<()> {
match load_remote_control_auth(auth_manager).await {
Ok(_) => Ok(()),
Err(err) if err.kind() == io::ErrorKind::WouldBlock => Ok(()),
Err(err) => Err(err),
}
}
#[cfg(test)]
mod tests;
codex-rs/app-server/src/transport/remote_control/protocol.rs
+3 -4
View File
@@ -48,9 +48,8 @@ pub enum ClientEvent {
message: JSONRPCMessage,
},
/// Backend-generated acknowledgement for all server envelopes addressed to
/// `client_id` whose envelope `seq_id` is less than or equal to this ack's
/// `seq_id`. This cursor is client-scoped, not stream-scoped, so receivers
/// must not use `stream_id` to partition acks.
/// `client_id` and `stream_id` whose envelope `seq_id` is less than or equal
/// to this ack's `seq_id`. This cursor is stream-scoped.
Ack,
Ping,
ClientClosed,
@@ -65,7 +64,7 @@ pub(crate) struct ClientEnvelope {
pub(crate) client_id: ClientId,
#[serde(rename = "stream_id", skip_serializing_if = "Option::is_none")]
pub(crate) stream_id: Option<StreamId>,
/// For `Ack`, this is the backend-generated per-client cursor over
/// For `Ack`, this is the backend-generated per-stream cursor over
/// `ServerEnvelope.seq_id`.
#[serde(rename = "seq_id", skip_serializing_if = "Option::is_none")]
pub(crate) seq_id: Option<u64>,
codex-rs/app-server/src/transport/remote_control/tests.rs
+62 -13
View File
@@ -5,6 +5,7 @@ use super::enroll::update_persisted_remote_control_enrollment;
use super::protocol::ClientEnvelope;
use super::protocol::ClientEvent;
use super::protocol::ClientId;
use super::protocol::StreamId;
use super::protocol::normalize_remote_control_url;
use super::websocket::REMOTE_CONTROL_PROTOCOL_VERSION;
use super::*;
@@ -96,6 +97,7 @@ fn remote_control_auth_dot_json(account_id: Option<&str>) -> AuthDotJson {
account_id: account_id.map(str::to_string),
}),
last_refresh: Some(chrono::Utc::now()),
agent_identity: None,
}
}
@@ -162,7 +164,7 @@ async fn remote_control_transport_manages_virtual_clients_and_routes_messages()
json!({
"type": "pong",
"client_id": "client-1",
"seq_id": 0,
"seq_id": 1,
"status": "unknown",
})
);
@@ -368,7 +370,7 @@ async fn remote_control_transport_manages_virtual_clients_and_routes_messages()
json!({
"type": "pong",
"client_id": "client-1",
"seq_id": 3,
"seq_id": 1,
"status": "unknown",
})
);
@@ -478,6 +480,44 @@ async fn remote_control_start_allows_remote_control_invalid_url_when_disabled()
.expect("remote control task should join");
}
#[tokio::test]
async fn remote_control_start_allows_missing_auth_when_enabled() {
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("listener should bind");
let remote_control_url = remote_control_url_for_listener(&listener);
let codex_home = TempDir::new().expect("temp dir should create");
let auth_manager = AuthManager::shared(
codex_home.path().to_path_buf(),
/*enable_codex_api_key_env*/ false,
AuthCredentialsStoreMode::File,
);
let (transport_event_tx, _transport_event_rx) =
mpsc::channel::<TransportEvent>(CHANNEL_CAPACITY);
let shutdown_token = CancellationToken::new();
let (remote_task, _remote_handle) = start_remote_control(
remote_control_url,
/*state_db*/ None,
auth_manager,
transport_event_tx,
shutdown_token.clone(),
/*app_server_client_name_rx*/ None,
/*initial_enabled*/ true,
)
.await
.expect("remote control should start before ChatGPT auth is available");
timeout(Duration::from_millis(100), listener.accept())
.await
.expect_err("remote control should wait for auth before connecting");
shutdown_token.cancel();
timeout(Duration::from_secs(1), remote_task)
.await
.expect("remote control task should stop")
.expect("remote control task should join");
}
#[tokio::test]
async fn remote_control_handle_set_enabled_stops_and_restarts_connections() {
let listener = TcpListener::bind("127.0.0.1:0")
@@ -617,12 +657,13 @@ async fn remote_control_transport_clears_outgoing_buffer_when_backend_acks() {
))
.await
.expect("remote writer should accept outgoing message");
let (server_event, stream_id) = read_server_event_with_stream_id(&mut first_websocket).await;
assert_eq!(
read_server_event(&mut first_websocket).await,
server_event,
json!({
"type": "server_message",
"client_id": "client-1",
"seq_id": 0,
"seq_id": 1,
"message": {
"method": "configWarning",
"params": {
@@ -638,8 +679,8 @@ async fn remote_control_transport_clears_outgoing_buffer_when_backend_acks() {
ClientEnvelope {
event: ClientEvent::Ack,
client_id: client_id.clone(),
stream_id: None,
seq_id: Some(0),
stream_id: Some(stream_id),
seq_id: Some(1),
cursor: None,
},
)
@@ -853,7 +894,7 @@ async fn remote_control_http_mode_enrolls_before_connecting() {
json!({
"type": "server_message",
"client_id": backend_client_id.0.clone(),
"seq_id": 0,
"seq_id": 1,
"message": {
"id": 11,
"result": {
@@ -881,7 +922,7 @@ async fn remote_control_http_mode_enrolls_before_connecting() {
json!({
"type": "server_message",
"client_id": backend_client_id.0.clone(),
"seq_id": 1,
"seq_id": 2,
"message": {
"method": "configWarning",
"params": {
@@ -1355,6 +1396,12 @@ async fn send_client_event(
}
async fn read_server_event(websocket: &mut WebSocketStream<TcpStream>) -> serde_json::Value {
read_server_event_with_stream_id(websocket).await.0
}
async fn read_server_event_with_stream_id(
websocket: &mut WebSocketStream<TcpStream>,
) -> (serde_json::Value, StreamId) {
loop {
let frame = timeout(Duration::from_secs(5), websocket.next())
.await
@@ -1365,13 +1412,15 @@ async fn read_server_event(websocket: &mut WebSocketStream<TcpStream>) -> serde_
tungstenite::Message::Text(text) => {
let mut event: serde_json::Value =
serde_json::from_str(text.as_ref()).expect("server event should deserialize");
if let Some(stream_id) = event
let stream_id = event
.as_object_mut()
.and_then(|event| event.remove("stream_id"))
{
assert!(stream_id.is_string(), "stream_id should be a string");
}
return event;
.expect("stream_id should be present");
let stream_id = stream_id
.as_str()
.expect("stream_id should be a string")
.to_string();
return (event, StreamId(stream_id));
}
tungstenite::Message::Ping(payload) => {
websocket
codex-rs/app-server/src/transport/remote_control/websocket.rs
+199 -82
View File
@@ -14,6 +14,7 @@ use super::protocol::ClientEvent;
use super::protocol::ClientId;
use super::protocol::RemoteControlTarget;
use super::protocol::ServerEnvelope;
use super::protocol::StreamId;
use axum::http::HeaderValue;
use base64::Engine;
use codex_core::util::backoff;
@@ -25,8 +26,8 @@ use futures::SinkExt;
use futures::StreamExt;
use futures::stream::SplitSink;
use futures::stream::SplitStream;
use std::collections::BTreeMap;
use std::collections::HashMap;
use std::collections::VecDeque;
use std::io;
use std::io::ErrorKind;
use std::sync::Arc;
@@ -57,10 +58,7 @@ const REMOTE_CONTROL_ACCOUNT_ID_RETRY_INTERVAL: std::time::Duration =
std::time::Duration::from_secs(1);
struct BoundedOutboundBuffer {
// Remote-control acks are generated by the backend at client scope, so
// retransmit retention is keyed by client_id only. stream_id stays on each
// envelope for routing, but it is not part of the ack cursor.
buffer_by_client: HashMap<ClientId, BTreeMap<u64, ServerEnvelope>>,
buffer_by_stream: HashMap<(ClientId, StreamId), VecDeque<ServerEnvelope>>,
used_tx: watch::Sender<usize>,
}
@@ -68,46 +66,50 @@ impl BoundedOutboundBuffer {
fn new() -> (Self, watch::Receiver<usize>) {
let (used_tx, used_rx) = watch::channel(0);
let buffer = Self {
buffer_by_client: HashMap::new(),
buffer_by_stream: HashMap::new(),
used_tx,
};
(buffer, used_rx)
}
fn insert(&mut self, server_envelope: &ServerEnvelope) {
self.buffer_by_client
.entry(server_envelope.client_id.clone())
self.buffer_by_stream
.entry((
server_envelope.client_id.clone(),
server_envelope.stream_id.clone(),
))
.or_default()
.insert(server_envelope.seq_id, server_envelope.clone());
.push_back(server_envelope.clone());
self.used_tx.send_modify(|used| *used += 1);
}
fn ack(&mut self, client_id: &ClientId, acked_seq_id: u64) {
let Some(buffer) = self.buffer_by_client.get_mut(client_id) else {
fn ack(&mut self, client_id: &ClientId, stream_id: &StreamId, acked_seq_id: u64) {
let key = (client_id.clone(), stream_id.clone());
let Some(buffer) = self.buffer_by_stream.get_mut(&key) else {
return;
};
while let Some(seq_id) = buffer.first_key_value().map(|(seq_id, _)| seq_id)
&& *seq_id <= acked_seq_id
while let Some(server_envelope) = buffer.front()
&& server_envelope.seq_id <= acked_seq_id
{
buffer.pop_first();
buffer.pop_front();
self.used_tx.send_modify(|used| *used -= 1);
}
if buffer.is_empty() {
self.buffer_by_client.remove(client_id);
self.buffer_by_stream.remove(&key);
}
}
fn server_envelopes(&self) -> impl Iterator<Item = &ServerEnvelope> {
self.buffer_by_client
self.buffer_by_stream
.values()
.flat_map(|buffer| buffer.values())
.flat_map(|buffer| buffer.iter())
}
}
struct WebsocketState {
outbound_buffer: BoundedOutboundBuffer,
subscribe_cursor: Option<String>,
next_seq_id: u64,
next_seq_id_by_stream: HashMap<(ClientId, StreamId), u64>,
}
pub(crate) struct RemoteControlWebsocket {
@@ -162,7 +164,7 @@ impl RemoteControlWebsocket {
state: Arc::new(Mutex::new(WebsocketState {
outbound_buffer,
subscribe_cursor: None,
next_seq_id: 0,
next_seq_id_by_stream: HashMap::new(),
})),
server_event_rx: Arc::new(Mutex::new(server_event_rx)),
used_rx,
@@ -445,10 +447,16 @@ impl RemoteControlWebsocket {
}
}
};
let (server_envelope, write_complete_tx) = {
let (payload, write_complete_tx) = {
let mut state = state.lock().await;
let seq_id = state.next_seq_id;
state.next_seq_id = state.next_seq_id.saturating_add(1);
let seq_key = (
queued_server_envelope.client_id.clone(),
queued_server_envelope.stream_id.clone(),
);
let seq_id = *state
.next_seq_id_by_stream
.entry(seq_key.clone())
.or_insert(1);
let server_envelope = ServerEnvelope {
event: queued_server_envelope.event,
@@ -456,17 +464,19 @@ impl RemoteControlWebsocket {
seq_id,
stream_id: queued_server_envelope.stream_id,
};
let payload = match serde_json::to_string(&server_envelope) {
Ok(payload) => payload,
Err(err) => {
error!("failed to serialize remote-control server event: {err}");
continue;
}
};
state
.next_seq_id_by_stream
.insert(seq_key, seq_id.saturating_add(1));
state.outbound_buffer.insert(&server_envelope);
(server_envelope, queued_server_envelope.write_complete_tx)
};
let payload = match serde_json::to_string(&server_envelope) {
Ok(payload) => payload,
Err(err) => {
error!("failed to serialize remote-control server event: {err}");
continue;
}
(payload, queued_server_envelope.write_complete_tx)
};
tokio::select! {
@@ -590,10 +600,13 @@ impl RemoteControlWebsocket {
}
if let ClientEvent::Ack = &client_envelope.event
&& let Some(acked_seq_id) = client_envelope.seq_id
&& let Some(stream_id) = client_envelope.stream_id.as_ref()
{
websocket_state
.outbound_buffer
.ack(&client_envelope.client_id, acked_seq_id);
websocket_state.outbound_buffer.ack(
&client_envelope.client_id,
stream_id,
acked_seq_id,
);
}
drop(websocket_state);
@@ -920,6 +933,13 @@ mod tests {
use tokio::time::timeout;
use tokio_tungstenite::accept_async;
// Windows Bazel CI can take longer than a few seconds for the websocket
// client connection attempt to reach the local test listener.
#[cfg(windows)]
const TEST_HTTP_ACCEPT_TIMEOUT: Duration = Duration::from_secs(30);
#[cfg(not(windows))]
const TEST_HTTP_ACCEPT_TIMEOUT: Duration = Duration::from_secs(5);
async fn remote_control_state_runtime(codex_home: &TempDir) -> Arc<StateRuntime> {
StateRuntime::init(codex_home.path().to_path_buf(), "test-provider".to_string())
.await
@@ -971,6 +991,7 @@ mod tests {
account_id: Some("account_id".to_string()),
}),
last_refresh: Some(Utc::now()),
agent_identity: None,
}
}
@@ -1219,7 +1240,7 @@ mod tests {
let state = Arc::new(Mutex::new(WebsocketState {
outbound_buffer,
subscribe_cursor: None,
next_seq_id: 0,
next_seq_id_by_stream: HashMap::new(),
}));
let (_server_event_tx, server_event_rx) = mpsc::channel(super::super::CHANNEL_CAPACITY);
let server_event_rx = Arc::new(Mutex::new(server_event_rx));
@@ -1247,6 +1268,83 @@ mod tests {
.expect("writer should stop cleanly");
}
#[tokio::test]
async fn run_server_writer_inner_assigns_contiguous_seq_ids_per_stream() {
let (client_stream, mut server_stream) = connected_websocket_pair().await;
let (websocket_writer, _websocket_reader) = client_stream.split();
let (outbound_buffer, used_rx) = BoundedOutboundBuffer::new();
let state = Arc::new(Mutex::new(WebsocketState {
outbound_buffer,
subscribe_cursor: None,
next_seq_id_by_stream: HashMap::new(),
}));
let (server_event_tx, server_event_rx) = mpsc::channel(super::super::CHANNEL_CAPACITY);
let server_event_rx = Arc::new(Mutex::new(server_event_rx));
let shutdown_token = CancellationToken::new();
let writer_task = tokio::spawn(RemoteControlWebsocket::run_server_writer_inner(
state,
server_event_rx,
used_rx,
websocket_writer,
Duration::from_secs(60),
shutdown_token.clone(),
));
let client_id = ClientId("client-1".to_string());
let first_stream = StreamId("stream-1".to_string());
let second_stream = StreamId("stream-2".to_string());
for stream_id in [&first_stream, &second_stream, &first_stream] {
server_event_tx
.send(super::super::QueuedServerEnvelope {
event: ServerEvent::Pong {
status: crate::transport::remote_control::protocol::PongStatus::Active,
},
client_id: client_id.clone(),
stream_id: stream_id.clone(),
write_complete_tx: None,
})
.await
.expect("server event should queue");
}
assert_eq!(
read_server_text_event(&mut server_stream).await,
serde_json::json!({
"type": "pong",
"client_id": "client-1",
"stream_id": "stream-1",
"seq_id": 1,
"status": "active",
})
);
assert_eq!(
read_server_text_event(&mut server_stream).await,
serde_json::json!({
"type": "pong",
"client_id": "client-1",
"stream_id": "stream-2",
"seq_id": 1,
"status": "active",
})
);
assert_eq!(
read_server_text_event(&mut server_stream).await,
serde_json::json!({
"type": "pong",
"client_id": "client-1",
"stream_id": "stream-1",
"seq_id": 2,
"status": "active",
})
);
shutdown_token.cancel();
writer_task
.await
.expect("writer task should join")
.expect("writer should stop cleanly");
}
#[tokio::test]
async fn run_websocket_reader_inner_times_out_without_pong_frames() {
let (client_stream, _server_stream) = connected_websocket_pair().await;
@@ -1255,7 +1353,7 @@ mod tests {
let state = Arc::new(Mutex::new(WebsocketState {
outbound_buffer,
subscribe_cursor: None,
next_seq_id: 0,
next_seq_id_by_stream: HashMap::new(),
}));
let (server_event_tx, _server_event_rx) = mpsc::channel(super::super::CHANNEL_CAPACITY);
let (transport_event_tx, _transport_event_rx) =
@@ -1286,70 +1384,32 @@ mod tests {
}
#[test]
fn outbound_buffer_acks_by_client_id_across_stream_ids() {
fn outbound_buffer_acks_by_stream_id() {
let (mut outbound_buffer, used_rx) = BoundedOutboundBuffer::new();
let client_1 = ClientId("client-1".to_string());
let client_2 = ClientId("client-2".to_string());
let stream_1 = StreamId("stream-1".to_string());
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-1",
/*seq_id*/ 0,
/*seq_id*/ 1,
"first-client-old-stream",
));
outbound_buffer.insert(&server_envelope(
&client_2,
"stream-1",
/*seq_id*/ 1,
/*seq_id*/ 2,
"second-client",
));
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-2",
/*seq_id*/ 2,
/*seq_id*/ 3,
"first-client-new-stream",
));
outbound_buffer.ack(&client_1, /*acked_seq_id*/ 2);
let mut retained = outbound_buffer
.server_envelopes()
.map(|server_envelope| {
(
server_envelope.client_id.0.as_str(),
server_envelope.stream_id.0.as_str(),
server_envelope.seq_id,
)
})
.collect::<Vec<_>>();
retained.sort_unstable();
assert_eq!(retained, vec![("client-2", "stream-1", 1)]);
assert_eq!(*used_rx.borrow(), 1);
}
#[test]
fn outbound_buffer_retains_unacked_messages_until_ack_advances() {
let (mut outbound_buffer, used_rx) = BoundedOutboundBuffer::new();
let client_1 = ClientId("client-1".to_string());
let client_2 = ClientId("client-2".to_string());
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-1",
/*seq_id*/ 0,
"first-old",
));
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-2",
/*seq_id*/ 1,
"first-new",
));
outbound_buffer.insert(&server_envelope(
&client_2, "stream-1", /*seq_id*/ 2, "second",
));
outbound_buffer.ack(&client_1, /*acked_seq_id*/ 0);
outbound_buffer.ack(&client_1, &stream_1, /*acked_seq_id*/ 3);
let mut retained = outbound_buffer
.server_envelopes()
@@ -1364,7 +1424,50 @@ mod tests {
retained.sort_unstable();
assert_eq!(
retained,
vec![("client-1", "stream-2", 1), ("client-2", "stream-1", 2)]
vec![("client-1", "stream-2", 3), ("client-2", "stream-1", 2)]
);
assert_eq!(*used_rx.borrow(), 2);
}
#[test]
fn outbound_buffer_retains_unacked_messages_until_ack_advances() {
let (mut outbound_buffer, used_rx) = BoundedOutboundBuffer::new();
let client_1 = ClientId("client-1".to_string());
let client_2 = ClientId("client-2".to_string());
let stream_1 = StreamId("stream-1".to_string());
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-1",
/*seq_id*/ 1,
"first-old",
));
outbound_buffer.insert(&server_envelope(
&client_1,
"stream-2",
/*seq_id*/ 2,
"first-new",
));
outbound_buffer.insert(&server_envelope(
&client_2, "stream-1", /*seq_id*/ 3, "second",
));
outbound_buffer.ack(&client_1, &stream_1, /*acked_seq_id*/ 1);
let mut retained = outbound_buffer
.server_envelopes()
.map(|server_envelope| {
(
server_envelope.client_id.0.as_str(),
server_envelope.stream_id.0.as_str(),
server_envelope.seq_id,
)
})
.collect::<Vec<_>>();
retained.sort_unstable();
assert_eq!(
retained,
vec![("client-1", "stream-2", 2), ("client-2", "stream-1", 3)]
);
assert_eq!(*used_rx.borrow(), 2);
}
@@ -1393,7 +1496,7 @@ mod tests {
}
async fn accept_http_request(listener: &TcpListener) -> (TcpStream, String) {
let (stream, _) = timeout(Duration::from_secs(5), listener.accept())
let (stream, _) = timeout(TEST_HTTP_ACCEPT_TIMEOUT, listener.accept())
.await
.expect("HTTP request should arrive in time")
.expect("listener accept should succeed");
@@ -1449,6 +1552,20 @@ mod tests {
(client_stream, server_stream)
}
async fn read_server_text_event(
server_stream: &mut WebSocketStream<TcpStream>,
) -> serde_json::Value {
let message = timeout(Duration::from_secs(5), server_stream.next())
.await
.expect("server event should arrive in time")
.expect("server websocket should stay open")
.expect("server event should read");
let tungstenite::Message::Text(text) = message else {
panic!("expected text event, got {message:?}");
};
serde_json::from_str(text.as_ref()).expect("server event should deserialize")
}
async fn respond_with_status_and_headers(
mut stream: TcpStream,
status: &str,
codex-rs/app-server/tests/common/auth_fixtures.rs
+1
View File
@@ -163,6 +163,7 @@ pub fn write_chatgpt_auth(
openai_api_key: None,
tokens: Some(tokens),
last_refresh,
agent_identity: None,
};
save_auth(codex_home, &auth, cli_auth_credentials_store_mode).context("write auth.json")
codex-rs/app-server/tests/suite/auth.rs
+3 -1
View File
@@ -24,7 +24,9 @@ use wiremock::ResponseTemplate;
use wiremock::matchers::method;
use wiremock::matchers::path;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
// Bazel CI can spend tens of seconds starting app-server subprocesses or
// processing auth RPCs under load.
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
fn create_config_toml_custom_provider(
codex_home: &Path,
codex-rs/app-server/tests/suite/fuzzy_file_search.rs
+5
View File
@@ -11,6 +11,11 @@ use std::path::Path;
use tempfile::TempDir;
use tokio::time::timeout;
// macOS arm64 and Windows Bazel CI can spend tens of seconds in app-server
// startup before the initialize response or fuzzy-search notifications arrive.
#[cfg(any(target_os = "macos", windows))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
#[cfg(not(any(target_os = "macos", windows)))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
const SHORT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(500);
const STOP_GRACE_PERIOD: std::time::Duration = std::time::Duration::from_millis(250);
codex-rs/app-server/tests/suite/v2/app_list.rs
+4 -1
View File
@@ -56,7 +56,9 @@ use tokio::net::TcpListener;
use tokio::task::JoinHandle;
use tokio::time::timeout;
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(10);
// Bazel CI can spend tens of seconds starting app-server subprocesses or
// processing app-list RPCs under load.
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(60);
#[tokio::test]
async fn list_apps_returns_empty_when_connectors_disabled() -> Result<()> {
@@ -117,6 +119,7 @@ async fn list_apps_returns_empty_with_api_key_auth() -> Result<()> {
openai_api_key: Some("test-api-key".to_string()),
tokens: None,
last_refresh: None,
agent_identity: None,
},
AuthCredentialsStoreMode::File,
)?;
codex-rs/app-server/tests/suite/v2/client_metadata.rs
+3 -1
View File
@@ -18,7 +18,9 @@ use std::path::Path;
use tempfile::TempDir;
use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
// Bazel CI can spend tens of seconds starting app-server subprocesses or
// processing turn RPCs under load.
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
#[tokio::test]
async fn turn_start_forwards_client_metadata_to_responses_request_v2() -> Result<()> {
codex-rs/app-server/tests/suite/v2/collaboration_mode_list.rs
+3 -1
View File
@@ -21,7 +21,9 @@ use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(10);
// Bazel CI can spend tens of seconds starting app-server subprocesses or
// processing list RPCs under load.
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(60);
/// Confirms the server returns the default collaboration mode presets in a stable order.
#[tokio::test]
codex-rs/app-server/tests/suite/v2/command_exec.rs
+114 -74
View File
@@ -411,11 +411,14 @@ async fn command_exec_streaming_does_not_buffer_output() -> Result<()> {
})
.await?;
let delta = read_command_exec_delta(&mut mcp).await?;
assert_eq!(delta.process_id, process_id.as_str());
assert_eq!(delta.stream, CommandExecOutputStream::Stdout);
assert_eq!(STANDARD.decode(&delta.delta_base64)?, b"abcde");
assert!(delta.cap_reached);
let output = collect_command_exec_output_until(
CommandExecDeltaReader::Mcp(&mut mcp),
process_id.as_str(),
"capped stdout",
|_output, delta| delta.stream == CommandExecOutputStream::Stdout && delta.cap_reached,
)
.await?;
assert_eq!(output.stdout, "abcde");
let terminate_request_id = mcp
.send_command_exec_terminate_request(CommandExecTerminateParams {
process_id: process_id.clone(),
@@ -471,21 +474,13 @@ async fn command_exec_pipe_streams_output_and_accepts_write() -> Result<()> {
})
.await?;
let first_stdout = read_command_exec_delta(&mut mcp).await?;
let first_stderr = read_command_exec_delta(&mut mcp).await?;
let seen = [first_stdout, first_stderr];
assert!(
seen.iter()
.all(|delta| delta.process_id == process_id.as_str())
);
assert!(seen.iter().any(|delta| {
delta.stream == CommandExecOutputStream::Stdout
&& delta.delta_base64 == STANDARD.encode("out-start\n")
}));
assert!(seen.iter().any(|delta| {
delta.stream == CommandExecOutputStream::Stderr
&& delta.delta_base64 == STANDARD.encode("err-start\n")
}));
wait_for_command_exec_outputs_contains(
&mut mcp,
process_id.as_str(),
"out-start\n",
"err-start\n",
)
.await?;
let write_request_id = mcp
.send_command_exec_write_request(CommandExecWriteParams {
@@ -499,21 +494,13 @@ async fn command_exec_pipe_streams_output_and_accepts_write() -> Result<()> {
.await?;
assert_eq!(write_response.result, serde_json::json!({}));
let next_delta = read_command_exec_delta(&mut mcp).await?;
let final_delta = read_command_exec_delta(&mut mcp).await?;
let seen = [next_delta, final_delta];
assert!(
seen.iter()
.all(|delta| delta.process_id == process_id.as_str())
);
assert!(seen.iter().any(|delta| {
delta.stream == CommandExecOutputStream::Stdout
&& delta.delta_base64 == STANDARD.encode("out:hello\n")
}));
assert!(seen.iter().any(|delta| {
delta.stream == CommandExecOutputStream::Stderr
&& delta.delta_base64 == STANDARD.encode("err:hello\n")
}));
wait_for_command_exec_outputs_contains(
&mut mcp,
process_id.as_str(),
"out:hello\n",
"err:hello\n",
)
.await?;
let response = mcp
.read_stream_until_response_message(RequestId::Integer(command_request_id))
@@ -562,17 +549,13 @@ async fn command_exec_tty_implies_streaming_and_reports_pty_output() -> Result<(
})
.await?;
let started_text = read_command_exec_output_until_contains(
wait_for_command_exec_output_contains(
&mut mcp,
process_id.as_str(),
CommandExecOutputStream::Stdout,
"tty\n",
)
.await?;
assert!(
started_text.contains("tty\n"),
"expected TTY startup output, got {started_text:?}"
);
let write_request_id = mcp
.send_command_exec_write_request(CommandExecWriteParams {
@@ -586,17 +569,13 @@ async fn command_exec_tty_implies_streaming_and_reports_pty_output() -> Result<(
.await?;
assert_eq!(write_response.result, serde_json::json!({}));
let echoed_text = read_command_exec_output_until_contains(
wait_for_command_exec_output_contains(
&mut mcp,
process_id.as_str(),
CommandExecOutputStream::Stdout,
"echo:world\n",
)
.await?;
assert!(
echoed_text.contains("echo:world\n"),
"expected TTY echo output, got {echoed_text:?}"
);
let response = mcp
.read_stream_until_response_message(RequestId::Integer(command_request_id))
@@ -643,17 +622,13 @@ async fn command_exec_tty_supports_initial_size_and_resize() -> Result<()> {
})
.await?;
let started_text = read_command_exec_output_until_contains(
wait_for_command_exec_output_contains(
&mut mcp,
process_id.as_str(),
CommandExecOutputStream::Stdout,
"start:31 101\n",
)
.await?;
assert!(
started_text.contains("start:31 101\n"),
"unexpected initial size output: {started_text:?}"
);
let resize_request_id = mcp
.send_command_exec_resize_request(CommandExecResizeParams {
@@ -681,17 +656,13 @@ async fn command_exec_tty_supports_initial_size_and_resize() -> Result<()> {
.await?;
assert_eq!(write_response.result, serde_json::json!({}));
let resized_text = read_command_exec_output_until_contains(
wait_for_command_exec_output_contains(
&mut mcp,
process_id.as_str(),
CommandExecOutputStream::Stdout,
"after:45 132\n",
)
.await?;
assert!(
resized_text.contains("after:45 132\n"),
"unexpected resized output: {resized_text:?}"
);
let response = mcp
.read_stream_until_response_message(RequestId::Integer(command_request_id))
@@ -744,11 +715,13 @@ async fn command_exec_process_ids_are_connection_scoped_and_disconnect_terminate
)
.await?;
let delta = read_command_exec_delta_ws(&mut ws1).await?;
assert_eq!(delta.process_id, "shared-process");
assert_eq!(delta.stream, CommandExecOutputStream::Stdout);
let delta_text = String::from_utf8(STANDARD.decode(&delta.delta_base64)?)?;
assert!(delta_text.contains("ready"));
collect_command_exec_output_until(
CommandExecDeltaReader::Websocket(&mut ws1),
"shared-process",
"websocket ready output",
|output, _delta| output.stdout.contains("ready\n"),
)
.await?;
wait_for_process_marker(&marker, /*should_exist*/ true).await?;
send_request(
@@ -796,31 +769,98 @@ async fn read_command_exec_delta(
decode_delta_notification(notification)
}
async fn read_command_exec_output_until_contains(
async fn wait_for_command_exec_output_contains(
mcp: &mut McpProcess,
process_id: &str,
stream: CommandExecOutputStream,
expected: &str,
) -> Result<String> {
) -> Result<()> {
let stream_name = match stream {
CommandExecOutputStream::Stdout => "stdout",
CommandExecOutputStream::Stderr => "stderr",
};
collect_command_exec_output_until(
CommandExecDeltaReader::Mcp(mcp),
process_id,
format!("{stream_name} containing {expected:?}"),
|output, _delta| match stream {
CommandExecOutputStream::Stdout => output.stdout.contains(expected),
CommandExecOutputStream::Stderr => output.stderr.contains(expected),
},
)
.await?;
Ok(())
}
async fn wait_for_command_exec_outputs_contains(
mcp: &mut McpProcess,
process_id: &str,
stdout_expected: &str,
stderr_expected: &str,
) -> Result<()> {
collect_command_exec_output_until(
CommandExecDeltaReader::Mcp(mcp),
process_id,
format!("stdout containing {stdout_expected:?} and stderr containing {stderr_expected:?}"),
|output, _delta| {
output.stdout.contains(stdout_expected) && output.stderr.contains(stderr_expected)
},
)
.await?;
Ok(())
}
enum CommandExecDeltaReader<'a> {
Mcp(&'a mut McpProcess),
Websocket(&'a mut super::connection_handling_websocket::WsClient),
}
#[derive(Default)]
struct CollectedCommandExecOutput {
stdout: String,
stderr: String,
}
async fn collect_command_exec_output_until(
mut reader: CommandExecDeltaReader<'_>,
process_id: &str,
waiting_for: impl Into<String>,
mut should_stop: impl FnMut(
&CollectedCommandExecOutput,
&CommandExecOutputDeltaNotification,
) -> bool,
) -> Result<CollectedCommandExecOutput> {
let waiting_for = waiting_for.into();
let deadline = Instant::now() + DEFAULT_READ_TIMEOUT;
let mut collected = String::new();
let mut output = CollectedCommandExecOutput::default();
loop {
let remaining = deadline.saturating_duration_since(Instant::now());
let delta = timeout(remaining, read_command_exec_delta(mcp))
.await
.with_context(|| {
format!(
"timed out waiting for {expected:?} in command/exec output for {process_id}; collected {collected:?}"
)
})??;
let delta = timeout(remaining, async {
match &mut reader {
CommandExecDeltaReader::Mcp(mcp) => read_command_exec_delta(mcp).await,
CommandExecDeltaReader::Websocket(stream) => {
read_command_exec_delta_ws(stream).await
}
}
})
.await
.with_context(|| {
format!(
"timed out waiting for {waiting_for} in command/exec output for {process_id}; collected stdout={:?}, stderr={:?}",
output.stdout, output.stderr
)
})??;
assert_eq!(delta.process_id, process_id);
assert_eq!(delta.stream, stream);
let delta_text = String::from_utf8(STANDARD.decode(&delta.delta_base64)?)?;
collected.push_str(&delta_text.replace('\r', ""));
if collected.contains(expected) {
return Ok(collected);
let delta_text = delta_text.replace('\r', "");
match delta.stream {
CommandExecOutputStream::Stdout => output.stdout.push_str(&delta_text),
CommandExecOutputStream::Stderr => output.stderr.push_str(&delta_text),
}
if should_stop(&output, &delta) {
return Ok(output);
}
}
}
codex-rs/app-server/tests/suite/v2/compaction.rs
+5
View File
@@ -38,6 +38,11 @@ use std::collections::BTreeMap;
use tempfile::TempDir;
use tokio::time::timeout;
// macOS and Windows Bazel CI can spend tens of seconds starting app-server
// subprocesses or processing test RPCs under load.
#[cfg(any(target_os = "macos", windows))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
#[cfg(not(any(target_os = "macos", windows)))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
const AUTO_COMPACT_LIMIT: i64 = 1_000;
const COMPACT_PROMPT: &str = "Summarize the conversation.";
codex-rs/app-server/tests/suite/v2/config_rpc.rs
+3 -1
View File
@@ -33,7 +33,9 @@ use serde_json::json;
use tempfile::TempDir;
use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
// Bazel CI can spend tens of seconds starting app-server subprocesses or
// processing config RPCs under load.
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
fn write_config(codex_home: &TempDir, contents: &str) -> Result<()> {
Ok(std::fs::write(
codex-rs/app-server/tests/suite/v2/connection_handling_websocket.rs
+10 -4
View File
@@ -47,6 +47,12 @@ use tokio_tungstenite::tungstenite::http::HeaderValue;
use tokio_tungstenite::tungstenite::http::header::AUTHORIZATION;
use tokio_tungstenite::tungstenite::http::header::ORIGIN;
// macOS and Windows CI can spend tens of seconds starting the app-server test
// binary under Bazel before it accepts JSON-RPC or reports its websocket bind
// address.
#[cfg(any(target_os = "macos", windows))]
pub(super) const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(60);
#[cfg(not(any(target_os = "macos", windows)))]
pub(super) const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(10);
pub(super) type WsClient = WebSocketStream<MaybeTlsStream<tokio::net::TcpStream>>;
@@ -399,7 +405,7 @@ pub(super) async fn spawn_websocket_server_with_args(
.take()
.context("failed to capture websocket app-server stderr")?;
let mut stderr_reader = BufReader::new(stderr).lines();
let deadline = Instant::now() + Duration::from_secs(10);
let deadline = Instant::now() + DEFAULT_READ_TIMEOUT;
let bind_addr = loop {
let line = timeout(
deadline.saturating_duration_since(Instant::now()),
@@ -457,7 +463,7 @@ pub(super) async fn connect_websocket_with_bearer(
) -> Result<WsClient> {
let url = format!("ws://{}", connectable_bind_addr(bind_addr));
let request = websocket_request(url.as_str(), bearer_token, /*origin*/ None)?;
let deadline = Instant::now() + Duration::from_secs(10);
let deadline = Instant::now() + DEFAULT_READ_TIMEOUT;
loop {
match connect_async(request.clone()).await {
Ok((stream, _response)) => return Ok(stream),
@@ -524,7 +530,7 @@ async fn run_websocket_server_to_completion_with_args(
.stderr(Stdio::piped())
.env("CODEX_HOME", codex_home)
.env("RUST_LOG", "debug");
timeout(Duration::from_secs(10), cmd.output())
timeout(DEFAULT_READ_TIMEOUT, cmd.output())
.await
.context("timed out waiting for websocket app-server to exit")?
.context("failed to run websocket app-server")
@@ -536,7 +542,7 @@ async fn http_get(
path: &str,
) -> Result<reqwest::Response> {
let connectable_bind_addr = connectable_bind_addr(bind_addr);
let deadline = Instant::now() + Duration::from_secs(10);
let deadline = Instant::now() + DEFAULT_READ_TIMEOUT;
loop {
match client
.get(format!("http://{connectable_bind_addr}{path}"))
codex-rs/app-server/tests/suite/v2/dynamic_tools.rs
+5
View File
@@ -34,6 +34,11 @@ use tempfile::TempDir;
use tokio::time::timeout;
use wiremock::MockServer;
// macOS and Windows Bazel CI can spend tens of seconds starting app-server
// subprocesses or processing test RPCs under load.
#[cfg(any(target_os = "macos", windows))]
const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(60);
#[cfg(not(any(target_os = "macos", windows)))]
const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(10);
/// Ensures dynamic tool specs are serialized into the model request payload.
codex-rs/app-server/tests/suite/v2/fs.rs
+5
View File
@@ -27,6 +27,11 @@ use std::os::unix::fs::symlink;
#[cfg(unix)]
use std::process::Command;
// macOS and Windows Bazel CI can spend tens of seconds starting app-server
// subprocesses or processing test RPCs under load.
#[cfg(any(target_os = "macos", windows))]
const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(60);
#[cfg(not(any(target_os = "macos", windows)))]
const DEFAULT_READ_TIMEOUT: Duration = Duration::from_secs(10);
async fn initialized_mcp(codex_home: &TempDir) -> Result<McpProcess> {
codex-rs/app-server/tests/suite/v2/mcp_server_elicitation.rs
+5 -3
View File
@@ -66,8 +66,9 @@ use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
const CONNECTOR_ID: &str = "calendar";
const CONNECTOR_NAME: &str = "Calendar";
const TOOL_NAMESPACE: &str = "mcp__codex_apps__calendar";
const CALLABLE_TOOL_NAME: &str = "_confirm_action";
const TOOL_NAME: &str = "calendar_confirm_action";
const QUALIFIED_TOOL_NAME: &str = "mcp__codex_apps__calendar_confirm_action";
const TOOL_CALL_ID: &str = "call-calendar-confirm";
const ELICITATION_MESSAGE: &str = "Allow this request?";
@@ -85,9 +86,10 @@ async fn mcp_server_elicitation_round_trip() -> Result<()> {
]),
responses::sse(vec![
responses::ev_response_created("resp-1"),
responses::ev_function_call(
responses::ev_function_call_with_namespace(
TOOL_CALL_ID,
QUALIFIED_TOOL_NAME,
TOOL_NAMESPACE,
CALLABLE_TOOL_NAME,
&tool_call_arguments,
),
responses::ev_completed("resp-1"),
codex-rs/app-server/tests/suite/v2/memory_reset.rs
+145
View File
@@ -0,0 +1,145 @@
use anyhow::Result;
use app_test_support::McpProcess;
use app_test_support::to_response;
use chrono::Utc;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::MemoryResetResponse;
use codex_app_server_protocol::RequestId;
use codex_protocol::ThreadId;
use codex_protocol::protocol::SessionSource;
use codex_state::Stage1JobClaimOutcome;
use codex_state::StateRuntime;
use codex_state::ThreadMetadataBuilder;
use pretty_assertions::assert_eq;
use std::path::Path;
use std::sync::Arc;
use tempfile::TempDir;
use tokio::time::timeout;
use uuid::Uuid;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
#[tokio::test]
async fn memory_reset_clears_memory_files_and_rows_preserves_threads() -> Result<()> {
let codex_home = TempDir::new()?;
create_config_toml(codex_home.path())?;
let state_db = init_state_db(codex_home.path()).await?;
let memory_root = codex_home.path().join("memories");
tokio::fs::create_dir_all(memory_root.join("rollout_summaries")).await?;
tokio::fs::write(memory_root.join("MEMORY.md"), "stale memory\n").await?;
tokio::fs::write(
memory_root.join("rollout_summaries").join("stale.md"),
"stale rollout summary\n",
)
.await?;
let thread_id = seed_stage1_output(&state_db, codex_home.path()).await?;
let mut mcp = McpProcess::new(codex_home.path()).await?;
timeout(DEFAULT_READ_TIMEOUT, mcp.initialize()).await??;
let request_id = mcp
.send_raw_request("memory/reset", /*params*/ None)
.await?;
let response: JSONRPCResponse = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(request_id)),
)
.await??;
let _: MemoryResetResponse = to_response::<MemoryResetResponse>(response)?;
let stage1_outputs = state_db.list_stage1_outputs_for_global(/*n*/ 10).await?;
assert_eq!(stage1_outputs, Vec::new());
assert_eq!(
state_db.get_thread_memory_mode(thread_id).await?.as_deref(),
Some("enabled")
);
let mut remaining_entries = tokio::fs::read_dir(&memory_root).await?;
assert!(
remaining_entries.next_entry().await?.is_none(),
"memory root should be empty after reset"
);
Ok(())
}
async fn seed_stage1_output(state_db: &Arc<StateRuntime>, codex_home: &Path) -> Result<ThreadId> {
let now = Utc::now();
let thread_id = ThreadId::from_string(&Uuid::new_v4().to_string())?;
let worker_id = ThreadId::from_string(&Uuid::new_v4().to_string())?;
let mut builder = ThreadMetadataBuilder::new(
thread_id,
codex_home.join("sessions").join("test.jsonl"),
now,
SessionSource::Cli,
);
builder.updated_at = Some(now);
builder.cwd = codex_home.to_path_buf();
let metadata = builder.build("mock_provider");
state_db.upsert_thread(&metadata).await?;
let claim = state_db
.try_claim_stage1_job(
thread_id,
worker_id,
now.timestamp(),
/*lease_seconds*/ 3600,
/*max_running_jobs*/ 64,
)
.await?;
let Stage1JobClaimOutcome::Claimed { ownership_token } = claim else {
anyhow::bail!("unexpected stage1 claim outcome: {claim:?}");
};
assert!(
state_db
.mark_stage1_job_succeeded(
thread_id,
ownership_token.as_str(),
now.timestamp(),
"raw memory",
"rollout summary",
/*rollout_slug*/ None,
)
.await?,
"stage1 success should be recorded"
);
state_db
.enqueue_global_consolidation(now.timestamp())
.await?;
Ok(thread_id)
}
async fn init_state_db(codex_home: &Path) -> Result<Arc<StateRuntime>> {
let state_db = StateRuntime::init(codex_home.to_path_buf(), "mock_provider".into()).await?;
state_db
.mark_backfill_complete(/*last_watermark*/ None)
.await?;
Ok(state_db)
}
fn create_config_toml(codex_home: &Path) -> std::io::Result<()> {
let config_toml = codex_home.join("config.toml");
std::fs::write(
config_toml,
r#"
model = "mock-model"
approval_policy = "never"
sandbox_mode = "read-only"
model_provider = "mock_provider"
suppress_unstable_features_warning = true
[features]
sqlite = true
[model_providers.mock_provider]
name = "Mock provider for test"
base_url = "http://127.0.0.1:9/v1"
wire_api = "responses"
request_max_retries = 0
stream_max_retries = 0
"#,
)
}
codex-rs/app-server/tests/suite/v2/mod.rs
+1
View File
@@ -20,6 +20,7 @@ mod mcp_resource;
mod mcp_server_elicitation;
mod mcp_server_status;
mod mcp_tool;
mod memory_reset;
mod model_list;
mod output_schema;
mod plan_item;
codex-rs/app-server/tests/suite/v2/plugin_install.rs
+3 -1
View File
@@ -51,7 +51,9 @@ use wiremock::matchers::header;
use wiremock::matchers::method;
use wiremock::matchers::path;
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(10);
// Plugin install tests wait on connector discovery after the install response path
// starts, which is noticeably slower on Windows CI.
const DEFAULT_TIMEOUT: Duration = Duration::from_secs(60);
#[tokio::test]
async fn plugin_install_rejects_relative_marketplace_paths() -> Result<()> {
codex-rs/app-server/tests/suite/v2/realtime_conversation.rs
+23 -8
View File
@@ -75,6 +75,8 @@ const DEFAULT_TIMEOUT: Duration = Duration::from_secs(10);
const STARTUP_CONTEXT_HEADER: &str = "Startup context from Codex.";
const V2_STEERING_ACKNOWLEDGEMENT: &str =
"This was sent to steer the previous background agent task.";
const V2_HANDOFF_COMPLETE_ACKNOWLEDGEMENT: &str =
"Background agent finished. Use the preceding [BACKEND] messages as the result.";
#[derive(Debug, Clone, Copy)]
enum StartupContextConfig<'a> {
@@ -1359,7 +1361,8 @@ async fn webrtc_v2_forwards_audio_and_text_between_client_and_sideband() -> Resu
request["type"] == "conversation.item.create"
&& request["item"]["type"] == "message"
&& request["item"]["role"] == "user"
&& request["item"]["content"][0]["text"] == "hello"
&& request["item"]["content"][0]["type"] == "input_text"
&& request["item"]["content"][0]["text"] == "[USER] hello"
}),
"sideband requests should include user text item: {requests:?}"
);
@@ -1558,7 +1561,7 @@ async fn webrtc_v2_background_agent_tool_call_delegates_and_returns_function_out
assert_v2_progress_update(&progress, "delegated from v2");
let tool_output = harness.sideband_outbound_request(/*request_index*/ 2).await;
assert_v2_function_call_output(&tool_output, "call_v2", "delegated from v2");
assert_v2_function_call_output(&tool_output, "call_v2", V2_HANDOFF_COMPLETE_ACKNOWLEDGEMENT);
assert_eq!(
function_call_output_sideband_requests(&harness.realtime_server).len(),
1
@@ -1693,7 +1696,11 @@ async fn webrtc_v2_background_agent_progress_is_sent_before_function_output() ->
assert_v2_progress_update(&progress, "progress before final");
let tool_output = harness.sideband_outbound_request(/*request_index*/ 2).await;
assert_v2_function_call_output(&tool_output, "call_progress_order", "progress before final");
assert_v2_function_call_output(
&tool_output,
"call_progress_order",
V2_HANDOFF_COMPLETE_ACKNOWLEDGEMENT,
);
harness.shutdown().await;
Ok(())
@@ -1777,7 +1784,11 @@ async fn webrtc_v2_tool_call_delegated_turn_can_execute_shell_tool() -> Result<(
assert_v2_progress_update(&progress, "shell tool finished");
let tool_output = harness.sideband_outbound_request(/*request_index*/ 2).await;
assert_v2_function_call_output(&tool_output, "call_shell", "shell tool finished");
assert_v2_function_call_output(
&tool_output,
"call_shell",
V2_HANDOFF_COMPLETE_ACKNOWLEDGEMENT,
);
assert_eq!(
function_call_output_sideband_requests(&harness.realtime_server).len(),
1
@@ -1857,7 +1868,11 @@ async fn webrtc_v2_tool_call_does_not_block_sideband_audio() -> Result<()> {
assert_v2_progress_update(&progress, "late delegated result");
let tool_output = harness.sideband_outbound_request(/*request_index*/ 2).await;
assert_v2_function_call_output(&tool_output, "call_audio", "late delegated result");
assert_v2_function_call_output(
&tool_output,
"call_audio",
V2_HANDOFF_COMPLETE_ACKNOWLEDGEMENT,
);
harness.shutdown().await;
Ok(())
@@ -2090,7 +2105,7 @@ fn assert_v2_function_call_output(request: &Value, call_id: &str, expected_outpu
"item": {
"type": "function_call_output",
"call_id": call_id,
"output": format!("\"Agent Final Message\":\n\n{expected_output}"),
"output": expected_output,
}
})
);
@@ -2106,7 +2121,7 @@ fn assert_v2_progress_update(request: &Value, expected_text: &str) {
"role": "user",
"content": [{
"type": "input_text",
"text": format!("{expected_text}\n\nUpdate from background agent (task hasn't finished yet):")
"text": format!("[BACKEND] {expected_text}")
}]
}
})
@@ -2123,7 +2138,7 @@ fn assert_v2_user_text_item(request: &Value, expected_text: &str) {
"role": "user",
"content": [{
"type": "input_text",
"text": expected_text
"text": format!("[USER] {expected_text}")
}]
}
})
codex-rs/app-server/tests/suite/v2/thread_fork.rs
+3
View File
@@ -43,6 +43,9 @@ use super::analytics::enable_analytics_capture;
use super::analytics::thread_initialized_event;
use super::analytics::wait_for_analytics_payload;
#[cfg(windows)]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(25);
#[cfg(not(windows))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
#[tokio::test]
codex-rs/app-server/tests/suite/v2/thread_list.rs
+22 -3
View File
@@ -563,15 +563,34 @@ sqlite = true
/*git_info*/ None,
)?;
// `thread/list` only applies `search_term` on the sqlite path. In this test we
// create rollouts manually, so we must also create the sqlite DB and mark backfill
// complete; otherwise app-server will permanently use filesystem fallback.
// `thread/list` applies `search_term` on the sqlite fast path. This test creates
// rollouts manually, so mark the DB backfill complete and then run an unsearched
// list large enough to repair every rollout the searched list should find.
let state_db =
codex_state::StateRuntime::init(codex_home.path().to_path_buf(), "mock_provider".into())
.await?;
state_db
.mark_backfill_complete(/*last_watermark*/ None)
.await?;
let rollout_config = codex_rollout::RolloutConfig {
codex_home: codex_home.path().to_path_buf(),
sqlite_home: codex_home.path().to_path_buf(),
cwd: codex_home.path().to_path_buf(),
model_provider_id: "mock_provider".to_string(),
generate_memories: false,
};
let repaired_page = codex_core::RolloutRecorder::list_threads(
&rollout_config,
/*page_size*/ 10,
/*cursor*/ None,
codex_core::ThreadSortKey::CreatedAt,
&[],
/*model_providers*/ None,
"mock_provider",
/*search_term*/ None,
)
.await?;
assert_eq!(repaired_page.items.len(), 3);
let mut mcp = init_mcp(codex_home.path()).await?;
let request_id = mcp
codex-rs/app-server/tests/suite/v2/thread_read.rs
+3
View File
@@ -36,6 +36,9 @@ use std::path::Path;
use tempfile::TempDir;
use tokio::time::timeout;
#[cfg(windows)]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(25);
#[cfg(not(windows))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
#[tokio::test]
codex-rs/app-server/tests/suite/v2/thread_resume.rs
+3
View File
@@ -76,6 +76,9 @@ use super::analytics::enable_analytics_capture;
use super::analytics::thread_initialized_event;
use super::analytics::wait_for_analytics_payload;
#[cfg(windows)]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(25);
#[cfg(not(windows))]
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
const CODEX_5_2_INSTRUCTIONS_TEMPLATE_DEFAULT: &str = "You are Codex, a coding agent based on GPT-5. You and the user share the same workspace and collaborate to achieve the user's goals.";
codex-rs/app-server/tests/suite/v2/thread_start.rs
+4 -2
View File
@@ -717,6 +717,7 @@ model_reasoning_effort = "high"
let config_toml = std::fs::read_to_string(codex_home.path().join("config.toml"))?;
let trusted_root = resolve_root_git_project_for_trust(workspace.path())
.await
.unwrap_or_else(|| workspace.path().to_path_buf());
assert!(config_toml.contains(&persisted_trust_path(&trusted_root)));
assert!(config_toml.contains("trust_level = \"trusted\""));
@@ -753,8 +754,9 @@ async fn thread_start_with_nested_git_cwd_trusts_repo_root() -> Result<()> {
.await??;
let config_toml = std::fs::read_to_string(codex_home.path().join("config.toml"))?;
let trusted_root =
resolve_root_git_project_for_trust(&nested).expect("git root should resolve");
let trusted_root = resolve_root_git_project_for_trust(&nested)
.await
.expect("git root should resolve");
assert!(config_toml.contains(&persisted_trust_path(&trusted_root)));
assert!(!config_toml.contains(&persisted_trust_path(&nested)));
codex-rs/app-server/tests/suite/v2/thread_unsubscribe.rs
+9 -4
View File
@@ -31,13 +31,17 @@ use tempfile::TempDir;
use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
// Windows CI can spend most of the default read timeout launching PowerShell
// before the command finishes and the follow-up model request is sent.
const TURN_COMPLETION_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
async fn wait_for_responses_request_count_to_stabilize(
server: &wiremock::MockServer,
expected_count: usize,
settle_duration: std::time::Duration,
timeout_duration: std::time::Duration,
) -> Result<()> {
timeout(DEFAULT_READ_TIMEOUT, async {
timeout(timeout_duration, async {
let mut stable_since: Option<tokio::time::Instant> = None;
loop {
let requests = server
@@ -130,9 +134,9 @@ async fn thread_unsubscribe_keeps_thread_loaded_until_idle_timeout() -> Result<(
async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
#[cfg(target_os = "windows")]
let shell_command = vec![
"powershell".to_string(),
"-Command".to_string(),
"Start-Sleep -Seconds 1".to_string(),
"Start-Sleep".to_string(),
"-Seconds".to_string(),
"1".to_string(),
];
#[cfg(not(target_os = "windows"))]
let shell_command = vec!["sleep".to_string(), "1".to_string()];
@@ -210,6 +214,7 @@ async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
&server,
/*expected_count*/ 2,
std::time::Duration::from_millis(200),
TURN_COMPLETION_TIMEOUT,
)
.await?;
codex-rs/app-server/tests/suite/v2/turn_interrupt.rs
+12 -3
View File
@@ -127,10 +127,17 @@ async fn turn_interrupt_aborts_running_turn() -> Result<()> {
#[tokio::test]
async fn turn_interrupt_resolves_pending_command_approval_request() -> Result<()> {
#[cfg(target_os = "windows")]
let shell_command = vec![
"powershell".to_string(),
"-Command".to_string(),
"Start-Sleep -Seconds 10".to_string(),
];
#[cfg(not(target_os = "windows"))]
let shell_command = vec![
"python3".to_string(),
"-c".to_string(),
"print(42)".to_string(),
"import time; time.sleep(10)".to_string(),
];
let tmp = TempDir::new()?;
@@ -143,7 +150,7 @@ async fn turn_interrupt_resolves_pending_command_approval_request() -> Result<()
shell_command.clone(),
Some(&working_directory),
Some(10_000),
"call_python_approval",
"call_sleep_approval",
)?])
.await;
create_config_toml(&codex_home, &server.uri(), "untrusted", "read-only")?;
@@ -172,6 +179,7 @@ async fn turn_interrupt_resolves_pending_command_approval_request() -> Result<()
text_elements: Vec::new(),
}],
cwd: Some(working_directory),
approval_policy: Some(codex_app_server_protocol::AskForApproval::UnlessTrusted),
..Default::default()
})
.await?;
@@ -190,7 +198,7 @@ async fn turn_interrupt_resolves_pending_command_approval_request() -> Result<()
let ServerRequest::CommandExecutionRequestApproval { request_id, params } = request else {
panic!("expected CommandExecutionRequestApproval request");
};
assert_eq!(params.item_id, "call_python_approval");
assert_eq!(params.item_id, "call_sleep_approval");
assert_eq!(params.thread_id, thread.id);
assert_eq!(params.turn_id, turn.id);
@@ -251,6 +259,7 @@ fn create_config_toml(
r#"
model = "mock-model"
approval_policy = "{approval_policy}"
approvals_reviewer = "user"
sandbox_mode = "{sandbox_mode}"
model_provider = "mock_provider"
codex-rs/backend-client/src/client.rs
+15
View File
@@ -119,6 +119,7 @@ pub struct Client {
bearer_token: Option<String>,
user_agent: Option<HeaderValue>,
chatgpt_account_id: Option<String>,
chatgpt_account_is_fedramp: bool,
path_style: PathStyle,
}
@@ -144,6 +145,7 @@ impl Client {
bearer_token: None,
user_agent: None,
chatgpt_account_id: None,
chatgpt_account_is_fedramp: false,
path_style,
})
}
@@ -156,6 +158,9 @@ impl Client {
if let Some(account_id) = auth.get_account_id() {
client = client.with_chatgpt_account_id(account_id);
}
if auth.is_fedramp_account() {
client = client.with_fedramp_routing_header();
}
Ok(client)
}
@@ -176,6 +181,11 @@ impl Client {
self
}
pub fn with_fedramp_routing_header(mut self) -> Self {
self.chatgpt_account_is_fedramp = true;
self
}
pub fn with_path_style(mut self, style: PathStyle) -> Self {
self.path_style = style;
self
@@ -200,6 +210,11 @@ impl Client {
{
h.insert(name, hv);
}
if self.chatgpt_account_is_fedramp
&& let Ok(name) = HeaderName::from_bytes(b"X-OpenAI-Fedramp")
{
h.insert(name, HeaderValue::from_static("true"));
}
h
}
codex-rs/chatgpt/Cargo.toml
+1
View File
@@ -10,6 +10,7 @@ workspace = true
[dependencies]
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
codex-app-server-protocol = { workspace = true }
codex-connectors = { workspace = true }
codex-config = { workspace = true }
codex-core = { workspace = true }
codex-rs/chatgpt/src/connectors.rs
+40 -57
View File
@@ -1,7 +1,3 @@
use codex_core::config::Config;
use codex_login::AuthManager;
use codex_login::CodexAuth;
use codex_login::token_data::TokenData;
use std::collections::HashSet;
use std::time::Duration;
@@ -9,21 +5,24 @@ use crate::chatgpt_client::chatgpt_get_request_with_timeout;
use crate::chatgpt_token::get_chatgpt_token_data;
use crate::chatgpt_token::init_chatgpt_token_from_auth;
use codex_app_server_protocol::AppInfo;
use codex_connectors::AllConnectorsCacheKey;
use codex_connectors::DirectoryListResponse;
pub use codex_core::connectors::AppInfo;
pub use codex_core::connectors::connector_display_label;
use codex_core::connectors::filter_disallowed_connectors;
use codex_connectors::filter::filter_disallowed_connectors;
use codex_connectors::merge::merge_connectors;
use codex_connectors::merge::merge_plugin_connectors;
use codex_core::config::Config;
pub use codex_core::connectors::list_accessible_connectors_from_mcp_tools;
pub use codex_core::connectors::list_accessible_connectors_from_mcp_tools_with_options;
pub use codex_core::connectors::list_accessible_connectors_from_mcp_tools_with_options_and_status;
pub use codex_core::connectors::list_cached_accessible_connectors_from_mcp_tools;
use codex_core::connectors::merge_connectors;
use codex_core::connectors::merge_plugin_apps;
pub use codex_core::connectors::with_app_enabled_state;
use codex_core::plugins::AppConnectorId;
use codex_core::plugins::PluginsManager;
use codex_login::AuthManager;
use codex_login::CodexAuth;
use codex_login::default_client::originator;
use codex_login::token_data::TokenData;
const DIRECTORY_CONNECTORS_TIMEOUT: Duration = Duration::from_secs(60);
@@ -74,8 +73,17 @@ pub async fn list_cached_all_connectors(config: &Config) -> Option<Vec<AppInfo>>
let token_data = get_chatgpt_token_data()?;
let cache_key = all_connectors_cache_key(config, &token_data);
let connectors = codex_connectors::cached_all_connectors(&cache_key)?;
let connectors = merge_plugin_apps(connectors, plugin_apps_for_config(config).await);
Some(filter_disallowed_connectors(connectors))
let connectors = merge_plugin_connectors(
connectors,
plugin_apps_for_config(config)
.await
.into_iter()
.map(|connector_id| connector_id.0),
);
Some(filter_disallowed_connectors(
connectors,
originator().value.as_str(),
))
}
pub async fn list_all_connectors_with_options(
@@ -105,8 +113,17 @@ pub async fn list_all_connectors_with_options(
},
)
.await?;
let connectors = merge_plugin_apps(connectors, plugin_apps_for_config(config).await);
Ok(filter_disallowed_connectors(connectors))
let connectors = merge_plugin_connectors(
connectors,
plugin_apps_for_config(config)
.await
.into_iter()
.map(|connector_id| connector_id.0),
);
Ok(filter_disallowed_connectors(
connectors,
originator().value.as_str(),
))
}
fn all_connectors_cache_key(config: &Config, token_data: &TokenData) -> AllConnectorsCacheKey {
@@ -134,7 +151,13 @@ pub fn connectors_for_plugin_apps(
.map(|connector_id| connector_id.0.as_str())
.collect::<HashSet<_>>();
filter_disallowed_connectors(merge_plugin_apps(connectors, plugin_apps.to_vec()))
let connectors = merge_plugin_connectors(
connectors,
plugin_apps
.iter()
.map(|connector_id| connector_id.0.clone()),
);
filter_disallowed_connectors(connectors, originator().value.as_str())
.into_iter()
.filter(|connector| plugin_app_ids.contains(connector.id.as_str()))
.collect()
@@ -158,13 +181,13 @@ pub fn merge_connectors_with_accessible(
accessible_connectors
};
let merged = merge_connectors(connectors, accessible_connectors);
filter_disallowed_connectors(merged)
filter_disallowed_connectors(merged, originator().value.as_str())
}
#[cfg(test)]
mod tests {
use super::*;
use codex_core::connectors::connector_install_url;
use codex_connectors::metadata::connector_install_url;
use codex_core::plugins::AppConnectorId;
use pretty_assertions::assert_eq;
@@ -186,46 +209,6 @@ mod tests {
}
}
#[test]
fn allows_asdk_connectors() {
let filtered = filter_disallowed_connectors(vec![app("asdk_app_hidden"), app("alpha")]);
assert_eq!(filtered, vec![app("asdk_app_hidden"), app("alpha")]);
}
#[test]
fn allows_whitelisted_asdk_connectors() {
let filtered = filter_disallowed_connectors(vec![
app("asdk_app_69781557cc1481919cf5e9824fa2e792"),
app("beta"),
]);
assert_eq!(
filtered,
vec![
app("asdk_app_69781557cc1481919cf5e9824fa2e792"),
app("beta")
]
);
}
#[test]
fn filters_openai_prefixed_connectors() {
let filtered = filter_disallowed_connectors(vec![
app("connector_openai_foo"),
app("connector_openai_bar"),
app("gamma"),
]);
assert_eq!(filtered, vec![app("gamma")]);
}
#[test]
fn filters_disallowed_connector_ids() {
let filtered = filter_disallowed_connectors(vec![
app("asdk_app_6938a94a61d881918ef32cb999ff937c"),
app("delta"),
]);
assert_eq!(filtered, vec![app("delta")]);
}
fn merged_app(id: &str, is_accessible: bool) -> AppInfo {
AppInfo {
id: id.to_string(),
codex-rs/cli/Cargo.toml
+1
View File
@@ -45,6 +45,7 @@ codex-state = { workspace = true }
codex-stdio-to-uds = { workspace = true }
codex-terminal-detection = { workspace = true }
codex-tui = { workspace = true }
codex-utils-absolute-path = { workspace = true }
codex-utils-path = { workspace = true }
libc = { workspace = true }
owo-colors = { workspace = true }
codex-rs/cli/src/debug_sandbox.rs
+18 -8
View File
@@ -18,7 +18,10 @@ use codex_protocol::config_types::SandboxMode;
use codex_protocol::permissions::NetworkSandboxPolicy;
use codex_sandboxing::landlock::create_linux_sandbox_command_args_for_policies;
#[cfg(target_os = "macos")]
use codex_sandboxing::seatbelt::create_seatbelt_command_args_for_policies;
use codex_sandboxing::seatbelt::CreateSeatbeltCommandArgsParams;
#[cfg(target_os = "macos")]
use codex_sandboxing::seatbelt::create_seatbelt_command_args;
use codex_utils_absolute_path::AbsolutePathBuf;
use codex_utils_cli::CliConfigOverrides;
use tokio::process::Child;
use tokio::process::Command as TokioCommand;
@@ -39,6 +42,7 @@ pub async fn run_command_under_seatbelt(
) -> anyhow::Result<()> {
let SeatbeltCommand {
full_auto,
allow_unix_sockets,
log_denials,
config_overrides,
command,
@@ -50,6 +54,7 @@ pub async fn run_command_under_seatbelt(
codex_linux_sandbox_exe,
SandboxType::Seatbelt,
log_denials,
&allow_unix_sockets,
)
.await
}
@@ -78,6 +83,7 @@ pub async fn run_command_under_landlock(
codex_linux_sandbox_exe,
SandboxType::Landlock,
/*log_denials*/ false,
&[],
)
.await
}
@@ -98,6 +104,7 @@ pub async fn run_command_under_windows(
codex_linux_sandbox_exe,
SandboxType::Windows,
/*log_denials*/ false,
&[],
)
.await
}
@@ -116,6 +123,8 @@ async fn run_command_under_sandbox(
codex_linux_sandbox_exe: Option<PathBuf>,
sandbox_type: SandboxType,
log_denials: bool,
#[cfg_attr(not(target_os = "macos"), allow(unused_variables))]
allow_unix_sockets: &[AbsolutePathBuf],
) -> anyhow::Result<()> {
let config = load_debug_sandbox_config(
config_overrides
@@ -252,14 +261,15 @@ async fn run_command_under_sandbox(
let mut child = match sandbox_type {
#[cfg(target_os = "macos")]
SandboxType::Seatbelt => {
let args = create_seatbelt_command_args_for_policies(
let args = create_seatbelt_command_args(CreateSeatbeltCommandArgsParams {
command,
&config.permissions.file_system_sandbox_policy,
config.permissions.network_sandbox_policy,
sandbox_policy_cwd.as_path(),
/*enforce_managed_network*/ false,
network.as_ref(),
);
file_system_sandbox_policy: &config.permissions.file_system_sandbox_policy,
network_sandbox_policy: config.permissions.network_sandbox_policy,
sandbox_policy_cwd: sandbox_policy_cwd.as_path(),
enforce_managed_network: false,
network: network.as_ref(),
extra_allow_unix_sockets: allow_unix_sockets,
});
let network_policy = config.permissions.network_sandbox_policy;
spawn_debug_sandbox_child(
PathBuf::from("/usr/bin/sandbox-exec"),
codex-rs/cli/src/lib.rs
+10
View File
@@ -3,6 +3,7 @@ mod exit_status;
pub(crate) mod login;
use clap::Parser;
use codex_utils_absolute_path::AbsolutePathBuf;
use codex_utils_cli::CliConfigOverrides;
pub use debug_sandbox::run_command_under_landlock;
@@ -22,6 +23,10 @@ pub struct SeatbeltCommand {
#[arg(long = "full-auto", default_value_t = false)]
pub full_auto: bool,
/// Allow the sandboxed command to bind/connect AF_UNIX sockets rooted at this path. Relative paths are resolved against the current directory. Repeat to allow multiple paths.
#[arg(long = "allow-unix-socket", value_parser = parse_allow_unix_socket_path)]
pub allow_unix_sockets: Vec<AbsolutePathBuf>,
/// While the command runs, capture macOS sandbox denials via `log stream` and print them after exit
#[arg(long = "log-denials", default_value_t = false)]
pub log_denials: bool,
@@ -34,6 +39,11 @@ pub struct SeatbeltCommand {
pub command: Vec<String>,
}
fn parse_allow_unix_socket_path(raw: &str) -> Result<AbsolutePathBuf, String> {
AbsolutePathBuf::relative_to_current_dir(raw)
.map_err(|err| format!("invalid path {raw}: {err}"))
}
#[derive(Debug, Parser)]
pub struct LandlockCommand {
/// Convenience alias for low-friction sandboxed automatic execution (network-disabled sandbox that can write to cwd and TMPDIR)
codex-rs/cli/src/main.rs
+20 -20
View File
@@ -49,6 +49,7 @@ use crate::mcp_cmd::McpCli;
use crate::responses_cmd::ResponsesCommand;
use crate::responses_cmd::run_responses_command;
use codex_core::clear_memory_roots_contents;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::config::edit::ConfigEditsBuilder;
@@ -516,10 +517,19 @@ fn run_update_action(action: UpdateAction) -> anyhow::Result<()> {
let status = {
#[cfg(windows)]
{
// On Windows, run via cmd.exe so .CMD/.BAT are correctly resolved (PATHEXT semantics).
std::process::Command::new("cmd")
.args(["/C", &cmd_str])
.status()?
if action == UpdateAction::StandaloneWindows {
let (cmd, args) = action.command_args();
// Run the standalone PowerShell installer with PowerShell
// itself. Routing this through `cmd.exe /C` would parse
// PowerShell metacharacters like `|` before PowerShell sees
// the installer command.
std::process::Command::new(cmd).args(args).status()?
} else {
// On Windows, run via cmd.exe so .CMD/.BAT are correctly resolved (PATHEXT semantics).
std::process::Command::new("cmd")
.args(["/C", &cmd_str])
.status()?
}
}
#[cfg(not(windows))]
{
@@ -1269,31 +1279,21 @@ async fn run_debug_clear_memories_command(
let state_db =
StateRuntime::init(config.sqlite_home.clone(), config.model_provider_id.clone())
.await?;
state_db.reset_memory_data_for_fresh_start().await?;
state_db.clear_memory_data().await?;
cleared_state_db = true;
}
let memory_root = config.codex_home.join("memories");
let removed_memory_root = match tokio::fs::remove_dir_all(&memory_root).await {
Ok(()) => true,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => false,
Err(err) => return Err(err.into()),
};
clear_memory_roots_contents(&config.codex_home).await?;
let mut message = if cleared_state_db {
format!("Cleared memory state from {}.", state_path.display())
} else {
format!("No state db found at {}.", state_path.display())
};
if removed_memory_root {
message.push_str(&format!(" Removed {}.", memory_root.display()));
} else {
message.push_str(&format!(
" No memory directory found at {}.",
memory_root.display()
));
}
message.push_str(&format!(
" Cleared memory directories under {}.",
config.codex_home.display()
));
println!("{message}");
codex-rs/cli/src/marketplace_cmd.rs
+75 -6
View File
@@ -1,8 +1,12 @@
use anyhow::Context;
use anyhow::Result;
use anyhow::bail;
use clap::Parser;
use codex_core::config::Config;
use codex_core::config::find_codex_home;
use codex_core::plugins::MarketplaceAddRequest;
use codex_core::plugins::PluginMarketplaceUpgradeOutcome;
use codex_core::plugins::PluginsManager;
use codex_core::plugins::add_marketplace;
use codex_utils_cli::CliConfigOverrides;
@@ -17,8 +21,8 @@ pub struct MarketplaceCli {
#[derive(Debug, clap::Subcommand)]
enum MarketplaceSubcommand {
/// Add a remote marketplace repository.
Add(AddMarketplaceArgs),
Upgrade(UpgradeMarketplaceArgs),
}
#[derive(Debug, Parser)]
@@ -27,11 +31,9 @@ struct AddMarketplaceArgs {
/// or local marketplace root directories.
source: String,
/// Git ref to check out. Overrides any @ref or #ref suffix in SOURCE.
#[arg(long = "ref", value_name = "REF")]
ref_name: Option<String>,
/// Sparse-checkout path to use while cloning git sources. Repeat to include multiple paths.
#[arg(
long = "sparse",
value_name = "PATH",
@@ -40,6 +42,11 @@ struct AddMarketplaceArgs {
sparse_paths: Vec<String>,
}
#[derive(Debug, Parser)]
struct UpgradeMarketplaceArgs {
marketplace_name: Option<String>,
}
impl MarketplaceCli {
pub async fn run(self) -> Result<()> {
let MarketplaceCli {
@@ -47,14 +54,13 @@ impl MarketplaceCli {
subcommand,
} = self;
// Validate overrides now. This command writes to CODEX_HOME only; marketplace discovery
// happens from that cache root after the next plugin/list or app-server start.
config_overrides
let overrides = config_overrides
.parse_overrides()
.map_err(anyhow::Error::msg)?;
match subcommand {
MarketplaceSubcommand::Add(args) => run_add(args).await?,
MarketplaceSubcommand::Upgrade(args) => run_upgrade(overrides, args).await?,
}
Ok(())
@@ -98,6 +104,60 @@ async fn run_add(args: AddMarketplaceArgs) -> Result<()> {
Ok(())
}
async fn run_upgrade(
overrides: Vec<(String, toml::Value)>,
args: UpgradeMarketplaceArgs,
) -> Result<()> {
let UpgradeMarketplaceArgs { marketplace_name } = args;
let config = Config::load_with_cli_overrides(overrides)
.await
.context("failed to load configuration")?;
let codex_home = find_codex_home().context("failed to resolve CODEX_HOME")?;
let manager = PluginsManager::new(codex_home.to_path_buf());
let outcome = manager
.upgrade_configured_marketplaces_for_config(&config, marketplace_name.as_deref())
.map_err(anyhow::Error::msg)?;
print_upgrade_outcome(&outcome, marketplace_name.as_deref())
}
fn print_upgrade_outcome(
outcome: &PluginMarketplaceUpgradeOutcome,
marketplace_name: Option<&str>,
) -> Result<()> {
for error in &outcome.errors {
eprintln!(
"Failed to upgrade marketplace `{}`: {}",
error.marketplace_name, error.message
);
}
if !outcome.all_succeeded() {
bail!("{} upgrade failure(s) occurred.", outcome.errors.len());
}
let selection_label = marketplace_name.unwrap_or("all configured Git marketplaces");
if outcome.selected_marketplaces.is_empty() {
println!("No configured Git marketplaces to upgrade.");
} else if outcome.upgraded_roots.is_empty() {
if marketplace_name.is_some() {
println!("Marketplace `{selection_label}` is already up to date.");
} else {
println!("All configured Git marketplaces are already up to date.");
}
} else if marketplace_name.is_some() {
println!("Upgraded marketplace `{selection_label}` to the latest configured revision.");
for root in &outcome.upgraded_roots {
println!("Installed marketplace root: {}", root.display());
}
} else {
println!("Upgraded {} marketplace(s).", outcome.upgraded_roots.len());
for root in &outcome.upgraded_roots {
println!("Installed marketplace root: {}", root.display());
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -132,4 +192,13 @@ mod tests {
vec!["plugins/foo", "skills/bar"]
);
}
#[test]
fn upgrade_subcommand_parses_optional_marketplace_name() {
let upgrade_all = UpgradeMarketplaceArgs::try_parse_from(["upgrade"]).unwrap();
assert_eq!(upgrade_all.marketplace_name, None);
let upgrade_one = UpgradeMarketplaceArgs::try_parse_from(["upgrade", "debug"]).unwrap();
assert_eq!(upgrade_one.marketplace_name.as_deref(), Some("debug"));
}
}
codex-rs/cli/src/mcp_cmd.rs
+11
View File
@@ -6,6 +6,7 @@ use anyhow::Result;
use anyhow::anyhow;
use anyhow::bail;
use clap::ArgGroup;
use codex_config::types::AppToolApproval;
use codex_config::types::McpServerConfig;
use codex_config::types::McpServerTransportConfig;
use codex_core::McpManager;
@@ -297,12 +298,14 @@ async fn run_add(config_overrides: &CliConfigOverrides, add_args: AddArgs) -> Re
let new_entry = McpServerConfig {
transport: transport.clone(),
experimental_environment: None,
enabled: true,
required: false,
supports_parallel_tool_calls: false,
disabled_reason: None,
startup_timeout_sec: None,
tool_timeout_sec: None,
default_tools_approval_mode: None,
enabled_tools: None,
disabled_tools: None,
scopes: None,
@@ -878,6 +881,14 @@ async fn run_get(config_overrides: &CliConfigOverrides, get_args: GetArgs) -> Re
if let Some(timeout) = server.tool_timeout_sec {
println!(" tool_timeout_sec: {}", timeout.as_secs_f64());
}
if let Some(approval_mode) = server.default_tools_approval_mode {
let approval_mode = match approval_mode {
AppToolApproval::Auto => "auto",
AppToolApproval::Prompt => "prompt",
AppToolApproval::Approve => "approve",
};
println!(" default_tools_approval_mode: {approval_mode}");
}
println!(" remove: codex mcp remove {}", get_args.name);
Ok(())
codex-rs/cli/tests/debug_clear_memories.rs
+2 -7
View File
@@ -125,13 +125,8 @@ INSERT INTO jobs (
.fetch_one(&pool)
.await?;
assert_eq!(memory_jobs_count, 0);
let memory_mode: String = sqlx::query_scalar("SELECT memory_mode FROM threads WHERE id = ?")
.bind(thread_id)
.fetch_one(&pool)
.await?;
assert_eq!(memory_mode, "disabled");
assert!(!memory_root.exists());
assert!(memory_root.exists());
assert_eq!(std::fs::read_dir(memory_root)?.count(), 0);
Ok(())
}
codex-rs/code-mode/Cargo.toml
+1
View File
@@ -14,6 +14,7 @@ workspace = true
[dependencies]
async-trait = { workspace = true }
codex-protocol = { workspace = true }
deno_core_icudata = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
codex-rs/code-mode/src/description.rs
+67 -32
View File
@@ -1,3 +1,4 @@
use codex_protocol::ToolName;
use serde::Deserialize;
use serde::Serialize;
use serde_json::Value as JsonValue;
@@ -8,6 +9,8 @@ use crate::PUBLIC_TOOL_NAME;
const MAX_JS_SAFE_INTEGER: u64 = (1_u64 << 53) - 1;
const CODE_MODE_ONLY_PREFACE: &str =
"Use `exec/wait` tool to run all other tools, do not attempt to use any other tools directly";
const DEFERRED_NESTED_TOOLS_GUIDANCE: &str = r#"Some nested MCP/app tools may be omitted from this description. They are still available on the global `tools` object and listed in `ALL_TOOLS`.
To find one, filter `ALL_TOOLS` by `name` and `description`; do not print the full `ALL_TOOLS` array. Print only a small set of relevant matches if you need to inspect them."#;
const EXEC_DESCRIPTION_TEMPLATE: &str = r#"Run JavaScript code to orchestrate/compose tool calls
- Evaluates the provided JavaScript code in a fresh V8 isolate as an async module.
- All nested tools are available on the global `tools` object, for example `await tools.exec_command(...)`. Tool names are exposed as normalized JavaScript identifiers, for example `await tools.mcp__ologs__get_profile(...)`.
@@ -23,7 +26,7 @@ const EXEC_DESCRIPTION_TEMPLATE: &str = r#"Run JavaScript code to orchestrate/co
- Global helpers:
- `exit()`: Immediately ends the current script successfully (like an early return from the top level).
- `text(value: string | number | boolean | undefined | null)`: Appends a text item. Non-string values are stringified with `JSON.stringify(...)` when possible.
- `image(imageUrlOrItem: string | { image_url: string; detail?: "auto" | "low" | "high" | "original" | null })`: Appends an image item. `image_url` can be an HTTPS URL or a base64-encoded `data:` URL.
- `image(imageUrlOrItem: string | { image_url: string; detail?: "auto" | "low" | "high" | "original" | null } | ImageContent, detail?: "auto" | "low" | "high" | "original" | null)`: Appends an image item. `image_url` can be an HTTPS URL or a base64-encoded `data:` URL. To forward an MCP tool image, pass an individual `ImageContent` block from `result.content`, for example `image(result.content[0])`. MCP image blocks may request original detail with `_meta: { "codex/imageDetail": "original" }`. When provided, the second `detail` argument overrides any detail embedded in the first argument.
- `store(key: string, value: any)`: stores a serializable value under a string key for later `exec` calls in the same session.
- `load(key: string)`: returns the stored value for a string key, or `undefined` if it is missing.
- `notify(value: string | number | boolean | undefined | null)`: immediately injects an extra `custom_tool_call_output` for the current `exec` call. Values are stringified like `text(...)`.
@@ -129,6 +132,7 @@ pub enum CodeModeToolKind {
#[derive(Clone, Debug, PartialEq)]
pub struct ToolDefinition {
pub name: String,
pub tool_name: ToolName,
pub description: String,
pub kind: CodeModeToolKind,
pub input_schema: Option<JsonValue>,
@@ -249,15 +253,19 @@ pub fn build_exec_tool_description(
enabled_tools: &[ToolDefinition],
namespace_descriptions: &BTreeMap<String, ToolNamespaceDescription>,
code_mode_only: bool,
deferred_tools_available: bool,
) -> String {
if !code_mode_only {
return EXEC_DESCRIPTION_TEMPLATE.to_string();
let mut sections = Vec::new();
if code_mode_only {
sections.push(CODE_MODE_ONLY_PREFACE.to_string());
}
sections.push(EXEC_DESCRIPTION_TEMPLATE.to_string());
if deferred_tools_available {
sections.push(DEFERRED_NESTED_TOOLS_GUIDANCE.to_string());
}
if !code_mode_only {
return sections.join("\n\n");
}
let mut sections = vec![
CODE_MODE_ONLY_PREFACE.to_string(),
EXEC_DESCRIPTION_TEMPLATE.to_string(),
];
if !enabled_tools.is_empty() {
let mut current_namespace: Option<&str> = None;
@@ -269,11 +277,15 @@ pub fn build_exec_tool_description(
for tool in enabled_tools {
let name = tool.name.as_str();
let nested_description = render_code_mode_sample_for_definition(tool);
let next_namespace = namespace_descriptions
.get(name)
let namespace_description = tool
.tool_name
.namespace
.as_ref()
.and_then(|namespace| namespace_descriptions.get(namespace));
let next_namespace = namespace_description
.map(|namespace_description| namespace_description.name.as_str());
if next_namespace != current_namespace {
if let Some(namespace_description) = namespace_descriptions.get(name) {
if let Some(namespace_description) = namespace_description {
let namespace_description_text = namespace_description.description.trim();
if !namespace_description_text.is_empty() {
nested_tool_sections.push(format!(
@@ -346,7 +358,7 @@ pub fn augment_tool_definition(mut definition: ToolDefinition) -> ToolDefinition
pub fn enabled_tool_metadata(definition: &ToolDefinition) -> EnabledToolMetadata {
EnabledToolMetadata {
tool_name: definition.name.clone(),
tool_name: definition.tool_name.clone(),
global_name: normalize_code_mode_identifier(&definition.name),
description: definition.description.clone(),
kind: definition.kind,
@@ -355,7 +367,7 @@ pub fn enabled_tool_metadata(definition: &ToolDefinition) -> EnabledToolMetadata
#[derive(Clone, Debug, Eq, PartialEq, Serialize)]
pub struct EnabledToolMetadata {
pub tool_name: String,
pub tool_name: ToolName,
pub global_name: String,
pub description: String,
pub kind: CodeModeToolKind,
@@ -706,6 +718,7 @@ mod tests {
use super::build_exec_tool_description;
use super::normalize_code_mode_identifier;
use super::parse_exec_source;
use codex_protocol::ToolName;
use pretty_assertions::assert_eq;
use serde_json::Value as JsonValue;
use serde_json::json;
@@ -770,6 +783,7 @@ mod tests {
fn augment_tool_definition_appends_typed_declaration() {
let definition = ToolDefinition {
name: "hidden_dynamic_tool".to_string(),
tool_name: ToolName::plain("hidden_dynamic_tool"),
description: "Test tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -798,6 +812,7 @@ mod tests {
fn augment_tool_definition_includes_property_descriptions_as_comments() {
let definition = ToolDefinition {
name: "weather_tool".to_string(),
tool_name: ToolName::plain("weather_tool"),
description: "Weather tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -846,6 +861,7 @@ mod tests {
let description = build_exec_tool_description(
&[ToolDefinition {
name: "foo".to_string(),
tool_name: ToolName::plain("foo"),
description: "bar".to_string(),
kind: CodeModeToolKind::Function,
input_schema: None,
@@ -853,6 +869,7 @@ mod tests {
}],
&BTreeMap::new(),
/*code_mode_only*/ true,
/*deferred_tools_available*/ false,
);
assert!(description.contains(
"### `foo`
@@ -862,34 +879,30 @@ bar"
#[test]
fn exec_description_mentions_timeout_helpers() {
let description =
build_exec_tool_description(&[], &BTreeMap::new(), /*code_mode_only*/ false);
let description = build_exec_tool_description(
&[],
&BTreeMap::new(),
/*code_mode_only*/ false,
/*deferred_tools_available*/ false,
);
assert!(description.contains("`setTimeout(callback: () => void, delayMs?: number)`"));
assert!(description.contains("`clearTimeout(timeoutId?: number)`"));
}
#[test]
fn code_mode_only_description_groups_namespace_instructions_once() {
let namespace_descriptions = BTreeMap::from([
(
"mcp__sample__alpha".to_string(),
ToolNamespaceDescription {
name: "mcp__sample".to_string(),
description: "Shared namespace guidance.".to_string(),
},
),
(
"mcp__sample__beta".to_string(),
ToolNamespaceDescription {
name: "mcp__sample".to_string(),
description: "Shared namespace guidance.".to_string(),
},
),
]);
let namespace_descriptions = BTreeMap::from([(
"mcp__sample__".to_string(),
ToolNamespaceDescription {
name: "mcp__sample".to_string(),
description: "Shared namespace guidance.".to_string(),
},
)]);
let description = build_exec_tool_description(
&[
ToolDefinition {
name: "mcp__sample__alpha".to_string(),
tool_name: ToolName::namespaced("mcp__sample__", "alpha"),
description: "First tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -905,6 +918,7 @@ bar"
},
ToolDefinition {
name: "mcp__sample__beta".to_string(),
tool_name: ToolName::namespaced("mcp__sample__", "beta"),
description: "Second tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -921,6 +935,7 @@ bar"
],
&namespace_descriptions,
/*code_mode_only*/ true,
/*deferred_tools_available*/ false,
);
assert_eq!(description.matches("## mcp__sample").count(), 1);
assert!(description.contains("## mcp__sample\nShared namespace guidance."));
@@ -935,7 +950,7 @@ bar"
#[test]
fn code_mode_only_description_omits_empty_namespace_sections() {
let namespace_descriptions = BTreeMap::from([(
"mcp__sample__alpha".to_string(),
"mcp__sample__".to_string(),
ToolNamespaceDescription {
name: "mcp__sample".to_string(),
description: String::new(),
@@ -944,6 +959,7 @@ bar"
let description = build_exec_tool_description(
&[ToolDefinition {
name: "mcp__sample__alpha".to_string(),
tool_name: ToolName::namespaced("mcp__sample__", "alpha"),
description: "First tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -959,6 +975,7 @@ bar"
}],
&namespace_descriptions,
/*code_mode_only*/ true,
/*deferred_tools_available*/ false,
);
assert!(!description.contains("## mcp__sample"));
@@ -969,6 +986,7 @@ bar"
fn code_mode_only_description_renders_shared_mcp_types_once() {
let first_tool = augment_tool_definition(ToolDefinition {
name: "mcp__sample__alpha".to_string(),
tool_name: ToolName::namespaced("mcp__sample__", "alpha"),
description: "First tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -1002,6 +1020,7 @@ bar"
});
let second_tool = augment_tool_definition(ToolDefinition {
name: "mcp__sample__beta".to_string(),
tool_name: ToolName::namespaced("mcp__sample__", "beta"),
description: "Second tool".to_string(),
kind: CodeModeToolKind::Function,
input_schema: Some(json!({
@@ -1038,6 +1057,7 @@ bar"
&[
ToolDefinition {
name: first_tool.name,
tool_name: first_tool.tool_name,
description: "First tool".to_string(),
kind: first_tool.kind,
input_schema: first_tool.input_schema,
@@ -1045,6 +1065,7 @@ bar"
},
ToolDefinition {
name: second_tool.name,
tool_name: second_tool.tool_name,
description: "Second tool".to_string(),
kind: second_tool.kind,
input_schema: second_tool.input_schema,
@@ -1053,6 +1074,7 @@ bar"
],
&BTreeMap::new(),
/*code_mode_only*/ true,
/*deferred_tools_available*/ false,
);
assert_eq!(
@@ -1063,4 +1085,17 @@ bar"
);
assert_eq!(description.matches("Shared MCP Types:").count(), 1);
}
#[test]
fn exec_description_mentions_deferred_nested_tools_when_available() {
let description = build_exec_tool_description(
&[],
&BTreeMap::new(),
/*code_mode_only*/ false,
/*deferred_tools_available*/ true,
);
assert!(description.contains("Some nested MCP/app tools may be omitted"));
assert!(description.contains("filter `ALL_TOOLS` by `name` and `description`"));
}
}
codex-rs/code-mode/src/runtime/callbacks.rs
+37 -2
View File
@@ -15,7 +15,13 @@ pub(super) fn tool_callback(
args: v8::FunctionCallbackArguments,
mut retval: v8::ReturnValue<v8::Value>,
) {
let tool_name = args.data().to_rust_string_lossy(scope);
let tool_index = match args.data().to_rust_string_lossy(scope).parse::<usize>() {
Ok(tool_index) => tool_index,
Err(_) => {
throw_type_error(scope, "invalid tool callback data");
return;
}
};
let input = if args.length() == 0 {
Ok(None)
} else {
@@ -36,6 +42,22 @@ pub(super) fn tool_callback(
let promise = resolver.get_promise(scope);
let resolver = v8::Global::new(scope, resolver);
let tool_name = {
let Some(state) = scope.get_slot::<RuntimeState>() else {
throw_type_error(scope, "runtime state unavailable");
return;
};
let Some(tool_name) = state
.enabled_tools
.get(tool_index)
.map(|tool| tool.tool_name.clone())
else {
throw_type_error(scope, "tool callback data is out of range");
return;
};
tool_name
};
let Some(state) = scope.get_slot_mut::<RuntimeState>() else {
throw_type_error(scope, "runtime state unavailable");
return;
@@ -87,7 +109,20 @@ pub(super) fn image_callback(
} else {
args.get(0)
};
let image_item = match normalize_output_image(scope, value) {
let detail_override = if args.length() < 2 {
None
} else {
let detail = args.get(1);
if detail.is_string() {
Some(detail.to_rust_string_lossy(scope))
} else if detail.is_null() || detail.is_undefined() {
None
} else {
throw_type_error(scope, "image detail must be a string when provided");
return;
}
};
let image_item = match normalize_output_image(scope, value, detail_override) {
Ok(image_item) => image_item,
Err(()) => return,
};
codex-rs/code-mode/src/runtime/globals.rs
+4 -4
View File
@@ -53,10 +53,10 @@ fn build_tools_object<'s>(
.map(|state| state.enabled_tools.clone())
.unwrap_or_default();
for tool in enabled_tools {
for (tool_index, tool) in enabled_tools.iter().enumerate() {
let name = v8::String::new(scope, &tool.global_name)
.ok_or_else(|| "failed to allocate tool name".to_string())?;
let function = tool_function(scope, &tool.tool_name)?;
let function = tool_function(scope, tool_index)?;
tools.set(scope, name.into(), function.into());
}
Ok(tools)
@@ -116,9 +116,9 @@ where
fn tool_function<'s>(
scope: &mut v8::PinScope<'s, '_>,
tool_name: &str,
tool_index: usize,
) -> Result<v8::Local<'s, v8::Function>, String> {
let data = v8::String::new(scope, tool_name)
let data = v8::String::new(scope, &tool_index.to_string())
.ok_or_else(|| "failed to allocate tool callback data".to_string())?;
let template = v8::FunctionTemplate::builder(tool_callback)
.data(data.into())
codex-rs/code-mode/src/runtime/mod.rs
+3 -2
View File
@@ -9,6 +9,7 @@ use std::sync::OnceLock;
use std::sync::mpsc as std_mpsc;
use std::thread;
use codex_protocol::ToolName;
use serde_json::Value as JsonValue;
use tokio::sync::mpsc;
@@ -62,7 +63,7 @@ pub(crate) enum TurnMessage {
ToolCall {
cell_id: String,
id: String,
name: String,
name: ToolName,
input: Option<JsonValue>,
},
Notify {
@@ -87,7 +88,7 @@ pub(crate) enum RuntimeEvent {
YieldRequested,
ToolCall {
id: String,
name: String,
name: ToolName,
input: Option<JsonValue>,
},
Notify {
codex-rs/code-mode/src/runtime/value.rs
+94 -30
View File
@@ -3,6 +3,9 @@ use serde_json::Value as JsonValue;
use crate::response::FunctionCallOutputContentItem;
use crate::response::ImageDetail;
const IMAGE_HELPER_EXPECTS_MESSAGE: &str = "image expects a non-empty image URL string, an object with image_url and optional detail, or a raw MCP image block";
const CODEX_IMAGE_DETAIL_META_KEY: &str = "codex/imageDetail";
pub(super) fn serialize_output_text(
scope: &mut v8::PinScope<'_, '_>,
value: v8::Local<'_, v8::Value>,
@@ -34,45 +37,25 @@ pub(super) fn serialize_output_text(
pub(super) fn normalize_output_image(
scope: &mut v8::PinScope<'_, '_>,
value: v8::Local<'_, v8::Value>,
detail_override: Option<String>,
) -> Result<FunctionCallOutputContentItem, ()> {
let result = (|| -> Result<FunctionCallOutputContentItem, String> {
let (image_url, detail) = if value.is_string() {
(value.to_rust_string_lossy(scope), None)
} else if value.is_object() && !value.is_array() {
let object = v8::Local::<v8::Object>::try_from(value).map_err(|_| {
"image expects a non-empty image URL string or an object with image_url and optional detail".to_string()
})?;
let image_url_key = v8::String::new(scope, "image_url")
.ok_or_else(|| "failed to allocate image helper keys".to_string())?;
let detail_key = v8::String::new(scope, "detail")
.ok_or_else(|| "failed to allocate image helper keys".to_string())?;
let image_url = object
.get(scope, image_url_key.into())
.filter(|value| value.is_string())
.map(|value| value.to_rust_string_lossy(scope))
.ok_or_else(|| {
"image expects a non-empty image URL string or an object with image_url and optional detail"
.to_string()
})?;
let detail = match object.get(scope, detail_key.into()) {
Some(value) if value.is_string() => Some(value.to_rust_string_lossy(scope)),
Some(value) if value.is_null() || value.is_undefined() => None,
Some(_) => return Err("image detail must be a string when provided".to_string()),
None => None,
};
(image_url, detail)
let object = v8::Local::<v8::Object>::try_from(value)
.map_err(|_| IMAGE_HELPER_EXPECTS_MESSAGE.to_string())?;
if let Some(image) = parse_non_mcp_output_image(scope, object)? {
image
} else {
parse_mcp_output_image(scope, value)?
}
} else {
return Err(
"image expects a non-empty image URL string or an object with image_url and optional detail"
.to_string(),
);
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
};
if image_url.is_empty() {
return Err(
"image expects a non-empty image URL string or an object with image_url and optional detail"
.to_string(),
);
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
}
let lower = image_url.to_ascii_lowercase();
if !(lower.starts_with("http://")
@@ -82,6 +65,7 @@ pub(super) fn normalize_output_image(
return Err("image expects an http(s) or data URL".to_string());
}
let detail = detail_override.or(detail);
let detail = match detail {
Some(detail) => {
let normalized = detail.to_ascii_lowercase();
@@ -112,6 +96,86 @@ pub(super) fn normalize_output_image(
}
}
fn parse_non_mcp_output_image(
scope: &mut v8::PinScope<'_, '_>,
object: v8::Local<'_, v8::Object>,
) -> Result<Option<(String, Option<String>)>, String> {
let image_url_key = v8::String::new(scope, "image_url")
.ok_or_else(|| "failed to allocate image helper keys".to_string())?;
let Some(image_url) = object.get(scope, image_url_key.into()) else {
return Ok(None);
};
if image_url.is_undefined() {
return Ok(None);
}
if !image_url.is_string() {
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
}
let detail_key = v8::String::new(scope, "detail")
.ok_or_else(|| "failed to allocate image helper keys".to_string())?;
let detail = parse_image_detail_value(scope, object.get(scope, detail_key.into()))?;
Ok(Some((image_url.to_rust_string_lossy(scope), detail)))
}
fn parse_mcp_output_image(
scope: &mut v8::PinScope<'_, '_>,
value: v8::Local<'_, v8::Value>,
) -> Result<(String, Option<String>), String> {
let Some(result) = v8_value_to_json(scope, value)? else {
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
};
let JsonValue::Object(result) = result else {
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
};
let Some(item_type) = result.get("type").and_then(JsonValue::as_str) else {
return Err(IMAGE_HELPER_EXPECTS_MESSAGE.to_string());
};
if item_type != "image" {
return Err(format!(
"image only accepts MCP image blocks, got \"{item_type}\""
));
}
let data = result
.get("data")
.and_then(JsonValue::as_str)
.ok_or_else(|| "image expected MCP image data".to_string())?;
if data.is_empty() {
return Err("image expected MCP image data".to_string());
}
let image_url = if data.to_ascii_lowercase().starts_with("data:") {
data.to_string()
} else {
let mime_type = result
.get("mimeType")
.or_else(|| result.get("mime_type"))
.and_then(JsonValue::as_str)
.filter(|mime_type| !mime_type.is_empty())
.unwrap_or("application/octet-stream");
format!("data:{mime_type};base64,{data}")
};
let detail = result
.get("_meta")
.and_then(JsonValue::as_object)
.and_then(|meta| meta.get(CODEX_IMAGE_DETAIL_META_KEY))
.and_then(JsonValue::as_str)
.filter(|detail| *detail == "original")
.map(str::to_string);
Ok((image_url, detail))
}
fn parse_image_detail_value<'s>(
scope: &mut v8::PinScope<'s, '_>,
value: Option<v8::Local<'s, v8::Value>>,
) -> Result<Option<String>, String> {
match value {
Some(value) if value.is_string() => Ok(Some(value.to_rust_string_lossy(scope))),
Some(value) if value.is_null() || value.is_undefined() => Ok(None),
Some(_) => Err("image detail must be a string when provided".to_string()),
None => Ok(None),
}
}
pub(super) fn v8_value_to_json(
scope: &mut v8::PinScope<'_, '_>,
value: v8::Local<'_, v8::Value>,
codex-rs/code-mode/src/service.rs
+150 -1
View File
@@ -5,6 +5,7 @@ use std::sync::atomic::Ordering;
use std::time::Duration;
use async_trait::async_trait;
use codex_protocol::ToolName;
use serde_json::Value as JsonValue;
use tokio::sync::Mutex;
use tokio::sync::mpsc;
@@ -26,7 +27,7 @@ use crate::runtime::spawn_runtime;
pub trait CodeModeTurnHost: Send + Sync {
async fn invoke_tool(
&self,
tool_name: String,
tool_name: ToolName,
input: Option<JsonValue>,
cancellation_token: CancellationToken,
) -> Result<JsonValue, String>;
@@ -683,6 +684,154 @@ text(JSON.stringify(returnsUndefined));
);
}
#[tokio::test]
async fn image_helper_accepts_raw_mcp_image_block_with_original_detail() {
let service = CodeModeService::new();
let response = service
.execute(ExecuteRequest {
source: r#"
image({
type: "image",
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4z8DwHwAFAAH/iZk9HQAAAABJRU5ErkJggg==",
mimeType: "image/png",
_meta: { "codex/imageDetail": "original" },
});
"#
.to_string(),
yield_time_ms: None,
..execute_request("")
})
.await
.unwrap();
assert_eq!(
response,
RuntimeResponse::Result {
cell_id: "1".to_string(),
content_items: vec![FunctionCallOutputContentItem::InputImage {
image_url: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4z8DwHwAFAAH/iZk9HQAAAABJRU5ErkJggg==".to_string(),
detail: Some(crate::ImageDetail::Original),
}],
stored_values: HashMap::new(),
error_text: None,
}
);
}
#[tokio::test]
async fn image_helper_second_arg_overrides_explicit_object_detail() {
let service = CodeModeService::new();
let response = service
.execute(ExecuteRequest {
source: r#"
image(
{
image_url: "https://example.com/image.jpg",
detail: "low",
},
"original",
);
"#
.to_string(),
yield_time_ms: None,
..execute_request("")
})
.await
.unwrap();
assert_eq!(
response,
RuntimeResponse::Result {
cell_id: "1".to_string(),
content_items: vec![FunctionCallOutputContentItem::InputImage {
image_url: "https://example.com/image.jpg".to_string(),
detail: Some(crate::ImageDetail::Original),
}],
stored_values: HashMap::new(),
error_text: None,
}
);
}
#[tokio::test]
async fn image_helper_second_arg_overrides_raw_mcp_image_detail() {
let service = CodeModeService::new();
let response = service
.execute(ExecuteRequest {
source: r#"
image(
{
type: "image",
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4z8DwHwAFAAH/iZk9HQAAAABJRU5ErkJggg==",
mimeType: "image/png",
_meta: { "codex/imageDetail": "original" },
},
"low",
);
"#
.to_string(),
yield_time_ms: None,
..execute_request("")
})
.await
.unwrap();
assert_eq!(
response,
RuntimeResponse::Result {
cell_id: "1".to_string(),
content_items: vec![FunctionCallOutputContentItem::InputImage {
image_url: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4z8DwHwAFAAH/iZk9HQAAAABJRU5ErkJggg==".to_string(),
detail: Some(crate::ImageDetail::Low),
}],
stored_values: HashMap::new(),
error_text: None,
}
);
}
#[tokio::test]
async fn image_helper_rejects_raw_mcp_result_container() {
let service = CodeModeService::new();
let response = service
.execute(ExecuteRequest {
source: r#"
image({
content: [
{
type: "image",
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4z8DwHwAFAAH/iZk9HQAAAABJRU5ErkJggg==",
mimeType: "image/png",
_meta: { "codex/imageDetail": "original" },
},
],
isError: false,
});
"#
.to_string(),
yield_time_ms: None,
..execute_request("")
})
.await
.unwrap();
assert_eq!(
response,
RuntimeResponse::Result {
cell_id: "1".to_string(),
content_items: Vec::new(),
stored_values: HashMap::new(),
error_text: Some(
"image expects a non-empty image URL string, an object with image_url and optional detail, or a raw MCP image block".to_string(),
),
}
);
}
#[tokio::test]
async fn terminate_waits_for_runtime_shutdown_before_responding() {
let inner = test_inner();
codex-rs/codex-api/src/api_bridge.rs
+17 -6
View File
@@ -12,6 +12,7 @@ use codex_protocol::error::RetryLimitReachedError;
use codex_protocol::error::UnexpectedResponseError;
use codex_protocol::error::UsageLimitReachedError;
use http::HeaderMap;
use http::HeaderValue;
use serde::Deserialize;
use serde_json::Value;
@@ -178,6 +179,7 @@ struct UsageErrorBody {
pub struct CoreAuthProvider {
pub token: Option<String>,
pub account_id: Option<String>,
pub is_fedramp_account: bool,
}
impl CoreAuthProvider {
@@ -195,16 +197,25 @@ impl CoreAuthProvider {
Self {
token: token.map(str::to_string),
account_id: account_id.map(str::to_string),
is_fedramp_account: false,
}
}
}
impl ApiAuthProvider for CoreAuthProvider {
fn bearer_token(&self) -> Option<String> {
self.token.clone()
}
fn account_id(&self) -> Option<String> {
self.account_id.clone()
fn add_auth_headers(&self, headers: &mut HeaderMap) {
if let Some(token) = self.token.as_ref()
&& let Ok(header) = HeaderValue::from_str(&format!("Bearer {token}"))
{
let _ = headers.insert(http::header::AUTHORIZATION, header);
}
if let Some(account_id) = self.account_id.as_ref()
&& let Ok(header) = HeaderValue::from_str(account_id)
{
let _ = headers.insert("ChatGPT-Account-ID", header);
}
if self.is_fedramp_account {
crate::auth::add_fedramp_routing_header(headers);
}
}
}
codex-rs/codex-api/src/api_bridge_tests.rs
+41
View File
@@ -136,8 +136,49 @@ fn core_auth_provider_reports_when_auth_header_will_attach() {
let auth = CoreAuthProvider {
token: Some("access-token".to_string()),
account_id: None,
is_fedramp_account: false,
};
assert!(auth.auth_header_attached());
assert_eq!(auth.auth_header_name(), Some("authorization"));
}
#[test]
fn core_auth_provider_adds_auth_headers() {
let auth = CoreAuthProvider::for_test(Some("access-token"), Some("workspace-123"));
let mut headers = HeaderMap::new();
crate::AuthProvider::add_auth_headers(&auth, &mut headers);
assert_eq!(
headers
.get(http::header::AUTHORIZATION)
.and_then(|value| value.to_str().ok()),
Some("Bearer access-token")
);
assert_eq!(
headers
.get("ChatGPT-Account-ID")
.and_then(|value| value.to_str().ok()),
Some("workspace-123")
);
}
#[test]
fn core_auth_provider_adds_fedramp_routing_header_for_fedramp_accounts() {
let auth = CoreAuthProvider {
token: Some("access-token".to_string()),
account_id: Some("workspace-123".to_string()),
is_fedramp_account: true,
};
let mut headers = HeaderMap::new();
crate::AuthProvider::add_auth_headers(&auth, &mut headers);
assert_eq!(
headers
.get("X-OpenAI-Fedramp")
.and_then(|value| value.to_str().ok()),
Some("true")
);
}
codex-rs/codex-api/src/auth.rs
+21 -20
View File
@@ -1,33 +1,34 @@
use codex_client::Request;
use http::HeaderMap;
use http::HeaderValue;
/// Provides bearer and account identity information for API requests.
/// Adds authentication headers to API requests.
///
/// Implementations should be cheap and non-blocking; any asynchronous
/// refresh or I/O should be handled by higher layers before requests
/// reach this interface.
pub trait AuthProvider: Send + Sync {
fn bearer_token(&self) -> Option<String>;
fn account_id(&self) -> Option<String> {
None
}
fn add_auth_headers(&self, headers: &mut HeaderMap);
}
pub(crate) fn add_auth_headers_to_header_map<A: AuthProvider>(auth: &A, headers: &mut HeaderMap) {
if let Some(token) = auth.bearer_token()
&& let Ok(header) = HeaderValue::from_str(&format!("Bearer {token}"))
{
let _ = headers.insert(http::header::AUTHORIZATION, header);
}
if let Some(account_id) = auth.account_id()
&& let Ok(header) = HeaderValue::from_str(&account_id)
{
let _ = headers.insert("ChatGPT-Account-ID", header);
}
pub(crate) fn add_fedramp_routing_header(headers: &mut HeaderMap) {
headers.insert("X-OpenAI-Fedramp", HeaderValue::from_static("true"));
}
pub(crate) fn add_auth_headers<A: AuthProvider>(auth: &A, mut req: Request) -> Request {
add_auth_headers_to_header_map(auth, &mut req.headers);
req
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn add_fedramp_routing_header_sets_header() {
let mut headers = HeaderMap::new();
add_fedramp_routing_header(&mut headers);
assert_eq!(
headers
.get("X-OpenAI-Fedramp")
.and_then(|v| v.to_str().ok()),
Some("true")
);
}
}
codex-rs/codex-api/src/endpoint/compact.rs
+1 -3
View File
@@ -90,9 +90,7 @@ mod tests {
struct DummyAuth;
impl AuthProvider for DummyAuth {
fn bearer_token(&self) -> Option<String> {
None
}
fn add_auth_headers(&self, _headers: &mut HeaderMap) {}
}
#[test]
codex-rs/codex-api/src/endpoint/memories.rs
+1 -3
View File
@@ -103,9 +103,7 @@ mod tests {
struct DummyAuth;
impl AuthProvider for DummyAuth {
fn bearer_token(&self) -> Option<String> {
None
}
fn add_auth_headers(&self, _headers: &mut HeaderMap) {}
}
#[derive(Clone)]
codex-rs/codex-api/src/endpoint/models.rs
+1 -3
View File
@@ -132,9 +132,7 @@ mod tests {
struct DummyAuth;
impl AuthProvider for DummyAuth {
fn bearer_token(&self) -> Option<String> {
None
}
fn add_auth_headers(&self, _headers: &mut HeaderMap) {}
}
fn provider(base_url: &str) -> Provider {
codex-rs/codex-api/src/endpoint/realtime_call.rs
+5 -2
View File
@@ -284,8 +284,11 @@ mod tests {
struct DummyAuth;
impl AuthProvider for DummyAuth {
fn bearer_token(&self) -> Option<String> {
Some("test-token".to_string())
fn add_auth_headers(&self, headers: &mut HeaderMap) {
headers.insert(
http::header::AUTHORIZATION,
HeaderValue::from_static("Bearer test-token"),
);
}
}
codex-rs/codex-api/src/endpoint/realtime_websocket/methods.rs
+1 -1
View File
@@ -1670,7 +1670,7 @@ mod tests {
);
assert_eq!(
third_json["item"]["output"],
Value::String("\"Agent Final Message\":\n\ndelegated result".to_string())
Value::String("delegated result".to_string())
);
});
codex-rs/codex-api/src/endpoint/realtime_websocket/methods_common.rs
+4 -2
View File
@@ -45,9 +45,11 @@ pub(super) fn conversation_handoff_append_message(
handoff_id: String,
output_text: String,
) -> RealtimeOutboundMessage {
let output_text = format!("{AGENT_FINAL_MESSAGE_PREFIX}{output_text}");
match event_parser {
RealtimeEventParser::V1 => v1_conversation_handoff_append_message(handoff_id, output_text),
RealtimeEventParser::V1 => v1_conversation_handoff_append_message(
handoff_id,
format!("{AGENT_FINAL_MESSAGE_PREFIX}{output_text}"),
),
RealtimeEventParser::RealtimeV2 => {
v2_conversation_handoff_append_message(handoff_id, output_text)
}
codex-rs/codex-api/src/endpoint/responses_websocket.rs
+1 -2
View File
@@ -1,5 +1,4 @@
use crate::auth::AuthProvider;
use crate::auth::add_auth_headers_to_header_map;
use crate::common::ResponseEvent;
use crate::common::ResponseStream;
use crate::common::ResponsesWsRequest;
@@ -310,7 +309,7 @@ impl<A: AuthProvider> ResponsesWebsocketClient<A> {
let mut headers =
merge_request_headers(&self.provider.headers, extra_headers, default_headers);
add_auth_headers_to_header_map(&self.auth, &mut headers);
self.auth.add_auth_headers(&mut headers);
let (stream, server_reasoning_included, models_etag, server_model) =
connect_websocket(ws_url, headers, turn_state.clone()).await?;
codex-rs/codex-api/src/endpoint/session.rs
+2 -2
View File
@@ -1,5 +1,4 @@
use crate::auth::AuthProvider;
use crate::auth::add_auth_headers;
use crate::error::ApiError;
use crate::provider::Provider;
use crate::telemetry::run_with_request_telemetry;
@@ -56,7 +55,8 @@ impl<T: HttpTransport, A: AuthProvider> EndpointSession<T, A> {
if let Some(body) = body {
req.body = Some(RequestBody::Json(body.clone()));
}
add_auth_headers(&self.auth, req)
self.auth.add_auth_headers(&mut req.headers);
req
}
pub(crate) async fn execute(
codex-rs/codex-api/src/files.rs
+6 -9
View File
@@ -256,17 +256,14 @@ fn authorized_request(
method: reqwest::Method,
url: &str,
) -> reqwest::RequestBuilder {
let mut headers = http::HeaderMap::new();
auth.add_auth_headers(&mut headers);
let client = build_reqwest_client();
let mut request = client
client
.request(method, url)
.timeout(OPENAI_FILE_REQUEST_TIMEOUT);
if let Some(token) = auth.bearer_token() {
request = request.bearer_auth(token);
}
if let Some(account_id) = auth.account_id() {
request = request.header("chatgpt-account-id", account_id);
}
request
.timeout(OPENAI_FILE_REQUEST_TIMEOUT)
.headers(headers)
}
fn build_reqwest_client() -> reqwest::Client {

Some files were not shown because too many files have changed in this diff Show More