diff --git a/codex-rs/Cargo.lock b/codex-rs/Cargo.lock index a54222a835..bd501bd7ab 100644 --- a/codex-rs/Cargo.lock +++ b/codex-rs/Cargo.lock @@ -573,6 +573,18 @@ version = "0.8.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e9b18233253483ce2f65329a24072ec414db782531bdbb7d0bbc4bd2ce6b7e21" +[[package]] +name = "codex-agent" +version = "0.0.0" +dependencies = [ + "anyhow", + "codex-protocol", + "portable-pty", + "serde", + "tokio", + "wildmatch", +] + [[package]] name = "codex-ansi-escape" version = "0.0.0" @@ -677,6 +689,7 @@ dependencies = [ "base64", "bytes", "chrono", + "codex-agent", "codex-apply-patch", "codex-file-search", "codex-mcp-client", diff --git a/codex-rs/Cargo.toml b/codex-rs/Cargo.toml index 237f5ea05c..cc67bac0bf 100644 --- a/codex-rs/Cargo.toml +++ b/codex-rs/Cargo.toml @@ -4,6 +4,7 @@ members = [ "apply-patch", "arg0", "cli", + "codex-agent", "common", "core", "exec", @@ -36,6 +37,7 @@ edition = "2024" codex-ansi-escape = { path = "ansi-escape" } codex-apply-patch = { path = "apply-patch" } codex-arg0 = { path = "arg0" } +codex-agent = { path = "codex-agent" } codex-chatgpt = { path = "chatgpt" } codex-common = { path = "common" } codex-core = { path = "core" } diff --git a/codex-rs/README.md b/codex-rs/README.md index 043d872afe..923927212a 100644 --- a/codex-rs/README.md +++ b/codex-rs/README.md @@ -97,6 +97,7 @@ The same setting can be persisted in `~/.codex/config.toml` via the top-level `s This folder is the root of a Cargo workspace. It contains quite a bit of experimental code, but here are the key crates: - [`core/`](./core) contains the business logic for Codex. Ultimately, we hope this to be a library crate that is generally useful for building other Rust/native applications that use Codex. +- [`docs/agent_runtime_baseline.md`](./docs/agent_runtime_baseline.md) documents the current agent runtime interfaces (`Codex`, `Session`, `SessionTask`) and links to the ongoing refactor plan in `agent_refactor.md`. - [`exec/`](./exec) "headless" CLI for use in automation. - [`tui/`](./tui) CLI that launches a fullscreen TUI built with [Ratatui](https://ratatui.rs/). - [`cli/`](./cli) CLI multitool that provides the aforementioned CLIs via subcommands. diff --git a/codex-rs/agent_refactor.md b/codex-rs/agent_refactor.md new file mode 100644 index 0000000000..904519b353 --- /dev/null +++ b/codex-rs/agent_refactor.md @@ -0,0 +1,108 @@ +# Agent Runtime Refactor + +## Goals +- Decouple the Codex agent loop from CLI-specific wiring so it can run as a reusable library or standalone binary. +- Preserve the current behaviour of `codex-core` (tooling, approvals, sandboxing, MCP integration) while providing a cleaner embedding surface. +- Enable specialised hosts—CLI, training harnesses, response API bridges—to share the same runtime with minimal glue code. + +## Proposed Architecture +### 1. `codex-agent` crate (new) +- Owns the session runtime: `AgentRuntime`, `AgentHandle`, the states, and the task runners now under `core/src/tasks`. +- Exposes a queue-like API: `AgentHandle::submit(Op/Submission)` and `AgentHandle::next_event()` mirroring today’s behaviour. +- Re-exports protocol types from `codex-protocol` so consumers do not depend on the entire `codex-core` tree. +- Houses the agent loop (`run_task`, `run_turn`, exec/safety plumbing) together with the sandbox planner (`ExecPlan`, `PreparedExec`, etc.). + +### 2. Shared configuration surface +- Introduce `AgentConfig` as the minimal runtime configuration (model, provider, approvals, sandbox defaults, cwd, user/base instructions, feature flags relevant to the loop). +- Provide `From<&Config>` for CLI compatibility; training/other hosts construct `AgentConfig` directly. +- CLI-only concerns (logging, auth prompts, workspace presets) stay inside `codex-core` and are translated before spawning the runtime. + +### 3. Service abstraction layer +- Define traits that the runtime depends on instead of concrete CLI structs: + - `CredentialsProvider` (wraps `AuthManager`). + - `Notifier` (reuses `UserNotifier` contract). + - `McpInterface` (start/list tools, dispatch tool calls). + - `SandboxManager` (wraps `BackendRegistry`/`prepare_exec_invocation` wiring). + - `RolloutSink` (write/flush rollout items; default no-op). +- Provide default implementations in `codex-core` that simply wrap the existing services (`SessionServices`). + +### 4. Task subsystem consolidation +- Keep the new `SessionTask` trait and concrete tasks (`RegularTask`, `ReviewTask`, `CompactTask`) inside `codex-agent` so custom hosts can opt into additional tasks without touching CLI crates. +- Ensure task lifecycle management (`spawn_task`, `abort_all_tasks`, `ActiveTurn`) stays encapsulated in the runtime and surfaces only high-level signals (events, cancellation APIs). + +### 5. Sandbox execution layer +- Move the recently created `core/src/sandbox` module into `codex-agent` (or re-export) so runtime owns exec planning. +- Runtime exposes an injectable `SandboxRuntimeConfig` (paths, seatbelt binary, stdout streaming choice) and calls into `SandboxManager` to execute plans. +- Respect existing environment variables and approval policies; no semantic changes to seatbelt handling. + +### 6. Host integrations +- CLI crate: replaces direct usage of `Codex::spawn` with `AgentRuntime::spawn`, adapting CLI config/auth providers to runtime traits. Behaviour remains identical. +- Training binary (`codex-agent-bin`): thin crate that parses CLI flags (Response API URL, auth token, optional instructions) and bridges remote Ops/Events to the runtime via chosen transport (MCP channel, HTTP/WebSocket bridge). +- Additional hosts can embed the runtime by implementing the service traits and providing transport glue. + +### 7. Transport adapters +- Internally keep `async_channel` for runtime queues. +- Provide helper adapters (`AgentTransport` trait) so callers can hook streams (local channel, TCP bridge, etc.) while keeping backpressure and graceful shutdown semantics consistent. + +## Guidelines +- **Config boundary**: new code must depend on `AgentConfig`; only CLI/front-ends may use the broader `Config` struct. Avoid adding CLI-specific fields to the runtime config. +- **Trait-based services**: any runtime dependency that could vary across hosts (MCP, rollout persistence, sandbox execution, notifications) should be expressed as a trait with a default implementation living in `codex-core`. +- **Task authoring**: additional tasks must implement `SessionTask`; tasks are responsible for calling `run_task`/`exit_review_mode` helpers and returning final assistant output for `TaskComplete` events. +- **Sandbox safety**: all exec/patch calls must flow through `plan_exec`/`plan_apply_patch` (now under `codex-agent::sandbox`) to preserve approval semantics. Never bypass `SandboxManager`. +- **MCP usage**: runtime talks only through `McpInterface`; hosts provide concrete connectors (existing CLI manager, lightweight training stub, etc.). +- **Rollout handling**: default `RolloutSink` should no-op; hosts that require persistence (CLI, evaluation harness) supply an implementation that wraps existing recorder. +- **Transport/backpressure**: treat the runtime queue as bounded and handle cancellations; adapters must propagate `Op::Shutdown` promptly. +- **Observability**: keep tracing instrumentation intact; new modules should use existing `tracing` spans for start/end of tasks, exec calls, and MCP interactions. +- **Code quality**: write minimalist idiomatic code. Leverage the capacity of Rust + + +## Implementation Plan +1. **Baseline & documentation** + - Capture current interfaces (`Codex`, `Session`, `SessionTask`) and update developer docs to reference this refactor plan. + - Add smoke tests covering multi-task scenarios (regular + review + compact) to guard against regressions during extraction. + +2. **Introduce `AgentConfig`** + - Define struct + conversion helpers inside `codex-core`. + - Refactor internal `Session::new` / `TurnContext` builders to accept `AgentConfig` without changing external behaviour. + +3. **Service trait extraction** + - Carve out trait definitions (`CredentialsProvider`, `McpInterface`, `SandboxManager`, `RolloutSink`, `Notifier`). + - Provide adapters backed by existing `SessionServices`. + - Update `Session` and helper modules to depend on traits rather than concrete structs. + +4. **Create `codex-agent` crate** + - Scaffold crate, move runtime modules (`codex.rs`, `state`, `tasks`, `sandbox`) while keeping module paths stable via `pub use` re-exports. + - Resolve module imports to reference trait abstractions / helper crates (e.g., `codex_protocol`, `codex-apply-patch`). + - Ensure crate exposes `AgentRuntime`, `AgentHandle`, and service traits. + +5. **Adapt `codex-core`** + - Replace `Codex::spawn` with thin wrapper that constructs `AgentConfig`, runtime service adapters, and delegates to `codex-agent`. + - Update public API to re-export runtime types if downstream crates expect them. + - Confirm unit tests continue to pass. + +6. **Update front-ends** + - CLI crate: switch to new runtime API; verify login/auth flows, approvals, and sandbox invocations. + - Other binaries (`chatgpt`, etc.) migrate similarly, adjusting imports/config conversions. + +7. **Add training binary** + - Implement new `codex-agent-bin` crate providing CLI for Response API URL + auth. + - Reuse existing MCP client logic where possible; otherwise, provide minimal HTTP bridge translating Ops/Events. + - Add integration tests using mocked Response API. + +8. **Refine transport adapters** + - Add optional helper module offering channel/TCP/WebSocket adapters along with graceful shutdown behaviour. + - Document how hosts select or implement transports. + +9. **Finalize rollout persistence strategy** + - Implement `RolloutSink` adapters (file-based, in-memory, disabled). + - Ensure CLI wires existing recorder; training binary can opt in/out via flags. + +10. **Docs & polish** + - Update repository documentation (`README`, architecture docs) to reference the new crates and APIs. + - Record migration notes for downstream consumers. + - Run `just fmt`, scoped `just fix -p`, and targeted tests for touched crates before merging. + +11. **Validation** + - Execute `cargo test -p codex-agent`, `cargo test -p codex-core`, and full suite (`cargo test --all-features`) once shared crates change. + - Perform manual verification: CLI session, review task, training binary against mock Response API, ensuring approvals and sandboxing behave identically. + diff --git a/codex-rs/codex-agent/Cargo.toml b/codex-rs/codex-agent/Cargo.toml new file mode 100644 index 0000000000..bfa52f3fde --- /dev/null +++ b/codex-rs/codex-agent/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "codex-agent" +version.workspace = true +edition.workspace = true + +[dependencies] +anyhow = { workspace = true } +portable-pty = { workspace = true } +tokio = { workspace = true, features = ["macros", "rt-multi-thread", "sync", "time"] } +codex-protocol = { workspace = true } +serde = { workspace = true, features = ["derive"] } +wildmatch = { workspace = true } + +[lints] +workspace = true diff --git a/codex-rs/codex-agent/src/config_types.rs b/codex-rs/codex-agent/src/config_types.rs new file mode 100644 index 0000000000..7febb3ebc3 --- /dev/null +++ b/codex-rs/codex-agent/src/config_types.rs @@ -0,0 +1,305 @@ +//! Shared configuration data structures for Codex runtime and hosts. +// +// This module intentionally focuses on simple data containers without +// business logic so they can be reused across crates. + +use std::collections::HashMap; +use std::path::PathBuf; +use std::time::Duration; +use wildmatch::WildMatchPattern; + +use serde::Deserialize; +use serde::Deserializer; +use serde::Serialize; +use serde::de::Error as SerdeError; + +#[derive(Serialize, Debug, Clone, PartialEq)] +pub struct McpServerConfig { + pub command: String, + + #[serde(default)] + pub args: Vec, + + #[serde(default)] + pub env: Option>, + + /// Startup timeout in seconds for initializing MCP server & initially listing tools. + #[serde( + default, + with = "option_duration_secs", + skip_serializing_if = "Option::is_none" + )] + pub startup_timeout_sec: Option, + + /// Default timeout for MCP tool calls initiated via this server. + #[serde(default, with = "option_duration_secs")] + pub tool_timeout_sec: Option, +} + +impl<'de> Deserialize<'de> for McpServerConfig { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + #[derive(Deserialize)] + struct RawMcpServerConfig { + command: String, + #[serde(default)] + args: Vec, + #[serde(default)] + env: Option>, + #[serde(default)] + startup_timeout_sec: Option, + #[serde(default)] + startup_timeout_ms: Option, + #[serde(default, with = "option_duration_secs")] + tool_timeout_sec: Option, + } + + let raw = RawMcpServerConfig::deserialize(deserializer)?; + + let startup_timeout_sec = match (raw.startup_timeout_sec, raw.startup_timeout_ms) { + (Some(sec), _) => { + let duration = Duration::try_from_secs_f64(sec).map_err(SerdeError::custom)?; + Some(duration) + } + (None, Some(ms)) => Some(Duration::from_millis(ms)), + (None, None) => None, + }; + + Ok(Self { + command: raw.command, + args: raw.args, + env: raw.env, + startup_timeout_sec, + tool_timeout_sec: raw.tool_timeout_sec, + }) + } +} + +mod option_duration_secs { + use serde::Deserialize; + use serde::Deserializer; + use serde::Serializer; + use std::time::Duration; + + pub fn serialize(value: &Option, serializer: S) -> Result + where + S: Serializer, + { + match value { + Some(duration) => serializer.serialize_some(&duration.as_secs_f64()), + None => serializer.serialize_none(), + } + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let secs = Option::::deserialize(deserializer)?; + secs.map(|secs| Duration::try_from_secs_f64(secs).map_err(serde::de::Error::custom)) + .transpose() + } +} + +#[derive(Deserialize, Debug, Copy, Clone, PartialEq)] +pub enum UriBasedFileOpener { + #[serde(rename = "vscode")] + VsCode, + + #[serde(rename = "vscode-insiders")] + VsCodeInsiders, + + #[serde(rename = "windsurf")] + Windsurf, + + #[serde(rename = "cursor")] + Cursor, + + /// Option to disable the URI-based file opener. + #[serde(rename = "none")] + None, +} + +impl UriBasedFileOpener { + pub fn get_scheme(&self) -> Option<&str> { + match self { + UriBasedFileOpener::VsCode => Some("vscode"), + UriBasedFileOpener::VsCodeInsiders => Some("vscode-insiders"), + UriBasedFileOpener::Windsurf => Some("windsurf"), + UriBasedFileOpener::Cursor => Some("cursor"), + UriBasedFileOpener::None => None, + } + } +} + +/// Settings that govern if and what will be written to `~/.codex/history.jsonl`. +#[derive(Deserialize, Debug, Clone, PartialEq, Default)] +pub struct History { + /// If true, history entries will not be written to disk. + pub persistence: HistoryPersistence, + + /// If set, the maximum size of the history file in bytes. + /// TODO(mbolin): Not currently honored. + pub max_bytes: Option, +} + +#[derive(Deserialize, Debug, Copy, Clone, PartialEq, Default)] +#[serde(rename_all = "kebab-case")] +pub enum HistoryPersistence { + /// Save all history entries to disk. + #[default] + SaveAll, + /// Do not write history to disk. + None, +} + +#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] +#[serde(untagged)] +pub enum Notifications { + Enabled(bool), + Custom(Vec), +} + +impl Default for Notifications { + fn default() -> Self { + Self::Enabled(false) + } +} + +/// Collection of settings that are specific to the TUI. +#[derive(Deserialize, Debug, Clone, PartialEq, Default)] +pub struct Tui { + /// Enable desktop notifications from the TUI when the terminal is unfocused. + /// Defaults to `false`. + #[serde(default)] + pub notifications: Notifications, +} + +#[derive(Deserialize, Debug, Clone, PartialEq, Default)] +pub struct SandboxWorkspaceWrite { + #[serde(default)] + pub writable_roots: Vec, + #[serde(default)] + pub network_access: bool, + #[serde(default)] + pub exclude_tmpdir_env_var: bool, + #[serde(default)] + pub exclude_slash_tmp: bool, +} + +impl From for codex_protocol::mcp_protocol::SandboxSettings { + fn from(sandbox_workspace_write: SandboxWorkspaceWrite) -> Self { + Self { + writable_roots: sandbox_workspace_write.writable_roots, + network_access: Some(sandbox_workspace_write.network_access), + exclude_tmpdir_env_var: Some(sandbox_workspace_write.exclude_tmpdir_env_var), + exclude_slash_tmp: Some(sandbox_workspace_write.exclude_slash_tmp), + } + } +} + +#[derive(Deserialize, Debug, Clone, PartialEq, Default)] +#[serde(rename_all = "kebab-case")] +pub enum ShellEnvironmentPolicyInherit { + /// "Core" environment variables for the platform. On UNIX, this would + /// include HOME, LOGNAME, PATH, SHELL, and USER, among others. + Core, + + /// Inherits the full environment from the parent process. + #[default] + All, + + /// Do not inherit any environment variables from the parent process. + None, +} + +/// Policy for building the `env` when spawning a process via either the +/// `shell` or `local_shell` tool. +#[derive(Deserialize, Debug, Clone, PartialEq, Default)] +pub struct ShellEnvironmentPolicyToml { + pub inherit: Option, + + pub ignore_default_excludes: Option, + + /// List of regular expressions. + pub exclude: Option>, + + pub r#set: Option>, + + /// List of regular expressions. + pub include_only: Option>, + + pub experimental_use_profile: Option, +} + +pub type EnvironmentVariablePattern = WildMatchPattern<'*', '?'>; + +/// Deriving the `env` based on this policy works as follows: +/// 1. Create an initial map based on the `inherit` policy. +/// 2. If `ignore_default_excludes` is false, filter the map using the default +/// exclude pattern(s), which are: `"*KEY*"` and `"*TOKEN*"`. +/// 3. If `exclude` is not empty, filter the map using the provided patterns. +/// 4. Insert any entries from `r#set` into the map. +/// 5. If non-empty, filter the map using the `include_only` patterns. +#[derive(Debug, Clone, PartialEq, Default)] +pub struct ShellEnvironmentPolicy { + /// Starting point when building the environment. + pub inherit: ShellEnvironmentPolicyInherit, + + /// True to skip the check to exclude default environment variables that + /// contain "KEY" or "TOKEN" in their name. + pub ignore_default_excludes: bool, + + /// Environment variable names to exclude from the environment. + pub exclude: Vec, + + /// (key, value) pairs to insert in the environment. + pub r#set: HashMap, + + /// Environment variable names to retain in the environment. + pub include_only: Vec, + + /// If true, the shell profile will be used to run the command. + pub use_profile: bool, +} + +impl From for ShellEnvironmentPolicy { + fn from(toml: ShellEnvironmentPolicyToml) -> Self { + // Default to inheriting the full environment when not specified. + let inherit = toml.inherit.unwrap_or(ShellEnvironmentPolicyInherit::All); + let ignore_default_excludes = toml.ignore_default_excludes.unwrap_or(false); + let exclude = toml + .exclude + .unwrap_or_default() + .into_iter() + .map(|s| EnvironmentVariablePattern::new_case_insensitive(&s)) + .collect(); + let r#set = toml.r#set.unwrap_or_default(); + let include_only = toml + .include_only + .unwrap_or_default() + .into_iter() + .map(|s| EnvironmentVariablePattern::new_case_insensitive(&s)) + .collect(); + let use_profile = toml.experimental_use_profile.unwrap_or(false); + + Self { + inherit, + ignore_default_excludes, + exclude, + r#set, + include_only, + use_profile, + } + } +} + +#[derive(Deserialize, Debug, Clone, PartialEq, Eq, Default, Hash)] +#[serde(rename_all = "kebab-case")] +pub enum ReasoningSummaryFormat { + #[default] + None, + Experimental, +} diff --git a/codex-rs/core/src/exec_command/exec_command_params.rs b/codex-rs/codex-agent/src/exec_command/exec_command_params.rs similarity index 100% rename from codex-rs/core/src/exec_command/exec_command_params.rs rename to codex-rs/codex-agent/src/exec_command/exec_command_params.rs diff --git a/codex-rs/core/src/exec_command/exec_command_session.rs b/codex-rs/codex-agent/src/exec_command/exec_command_session.rs similarity index 94% rename from codex-rs/core/src/exec_command/exec_command_session.rs rename to codex-rs/codex-agent/src/exec_command/exec_command_session.rs index 31b3c929b2..2f4d03aacf 100644 --- a/codex-rs/core/src/exec_command/exec_command_session.rs +++ b/codex-rs/codex-agent/src/exec_command/exec_command_session.rs @@ -5,7 +5,8 @@ use tokio::sync::mpsc; use tokio::task::JoinHandle; #[derive(Debug)] -pub(crate) struct ExecCommandSession { +#[allow(dead_code)] +pub struct ExecCommandSession { /// Queue for writing bytes to the process stdin (PTY master write side). writer_tx: mpsc::Sender>, /// Broadcast stream of output chunks read from the PTY. New subscribers @@ -29,8 +30,9 @@ pub(crate) struct ExecCommandSession { exit_status: std::sync::Arc, } +#[allow(dead_code)] impl ExecCommandSession { - pub(crate) fn new( + pub fn new( writer_tx: mpsc::Sender>, output_tx: broadcast::Sender>, killer: Box, @@ -54,7 +56,7 @@ impl ExecCommandSession { ) } - pub(crate) fn writer_sender(&self) -> mpsc::Sender> { + pub fn writer_sender(&self) -> mpsc::Sender> { self.writer_tx.clone() } @@ -62,7 +64,7 @@ impl ExecCommandSession { self.output_tx.subscribe() } - pub(crate) fn has_exited(&self) -> bool { + pub fn has_exited(&self) -> bool { self.exit_status.load(std::sync::atomic::Ordering::SeqCst) } } diff --git a/codex-rs/codex-agent/src/exec_command/mod.rs b/codex-rs/codex-agent/src/exec_command/mod.rs new file mode 100644 index 0000000000..24c59c49a3 --- /dev/null +++ b/codex-rs/codex-agent/src/exec_command/mod.rs @@ -0,0 +1,11 @@ +mod exec_command_params; +mod exec_command_session; +mod session_id; +mod session_manager; + +pub use exec_command_params::ExecCommandParams; +pub use exec_command_params::WriteStdinParams; +pub use exec_command_session::ExecCommandSession; +pub use session_id::SessionId; +pub use session_manager::ExecCommandOutput; +pub use session_manager::SessionManager as ExecSessionManager; diff --git a/codex-rs/core/src/exec_command/session_id.rs b/codex-rs/codex-agent/src/exec_command/session_id.rs similarity index 76% rename from codex-rs/core/src/exec_command/session_id.rs rename to codex-rs/codex-agent/src/exec_command/session_id.rs index c97c5d5440..3d884fdf58 100644 --- a/codex-rs/core/src/exec_command/session_id.rs +++ b/codex-rs/codex-agent/src/exec_command/session_id.rs @@ -2,4 +2,4 @@ use serde::Deserialize; use serde::Serialize; #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] -pub(crate) struct SessionId(pub u32); +pub struct SessionId(pub u32); diff --git a/codex-rs/core/src/exec_command/session_manager.rs b/codex-rs/codex-agent/src/exec_command/session_manager.rs similarity index 98% rename from codex-rs/core/src/exec_command/session_manager.rs rename to codex-rs/codex-agent/src/exec_command/session_manager.rs index cd1c5329a2..f3c18c4b35 100644 --- a/codex-rs/core/src/exec_command/session_manager.rs +++ b/codex-rs/codex-agent/src/exec_command/session_manager.rs @@ -5,6 +5,7 @@ use std::sync::Arc; use std::sync::Mutex as StdMutex; use std::sync::atomic::AtomicBool; use std::sync::atomic::AtomicU32; +use std::vec::Vec; use portable_pty::CommandBuilder; use portable_pty::PtySize; @@ -28,6 +29,7 @@ pub struct SessionManager { sessions: Mutex>, } +#[allow(dead_code)] #[derive(Debug)] pub struct ExecCommandOutput { wall_time: Duration, @@ -37,7 +39,7 @@ pub struct ExecCommandOutput { } impl ExecCommandOutput { - pub(crate) fn to_text_output(&self) -> String { + pub fn to_text_output(&self) -> String { let wall_time_secs = self.wall_time.as_secs_f32(); let termination_status = match self.exit_status { ExitStatus::Exited(code) => format!("Process exited with code {code}"), @@ -61,6 +63,7 @@ Output: } } +#[allow(dead_code)] #[derive(Debug)] pub enum ExitStatus { Exited(i32), @@ -79,7 +82,11 @@ impl SessionManager { .fetch_add(1, std::sync::atomic::Ordering::SeqCst), ); - let (session, mut output_rx, mut exit_rx) = create_exec_command_session(params.clone()) + let (session, mut output_rx, mut exit_rx): ( + ExecCommandSession, + tokio::sync::broadcast::Receiver>, + tokio::sync::oneshot::Receiver, + ) = create_exec_command_session(params.clone()) .await .map_err(|err| { format!( diff --git a/codex-rs/codex-agent/src/lib.rs b/codex-rs/codex-agent/src/lib.rs new file mode 100644 index 0000000000..6189b9b327 --- /dev/null +++ b/codex-rs/codex-agent/src/lib.rs @@ -0,0 +1,5 @@ +pub mod config_types; +pub mod exec_command; +pub mod truncate; + +pub use config_types::*; diff --git a/codex-rs/codex-agent/src/truncate.rs b/codex-rs/codex-agent/src/truncate.rs new file mode 100644 index 0000000000..1672bfcdec --- /dev/null +++ b/codex-rs/codex-agent/src/truncate.rs @@ -0,0 +1,180 @@ +//! Utilities for truncating large chunks of output while preserving a prefix +//! and suffix on UTF-8 boundaries. + +/// Truncate the middle of a UTF-8 string to at most `max_bytes` bytes, +/// preserving the beginning and the end. Returns the possibly truncated +/// string and `Some(original_token_count)` (estimated at 4 bytes/token) +/// if truncation occurred; otherwise returns the original string and `None`. +pub fn truncate_middle(s: &str, max_bytes: usize) -> (String, Option) { + if s.len() <= max_bytes { + return (s.to_string(), None); + } + + let est_tokens = (s.len() as u64).div_ceil(4); + if max_bytes == 0 { + return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); + } + + fn truncate_on_boundary(input: &str, max_len: usize) -> &str { + if input.len() <= max_len { + return input; + } + let mut end = max_len; + while end > 0 && !input.is_char_boundary(end) { + end -= 1; + } + &input[..end] + } + + fn pick_prefix_end(s: &str, left_budget: usize) -> usize { + if let Some(head) = s.get(..left_budget) + && let Some(i) = head.rfind('\n') + { + return i + 1; + } + truncate_on_boundary(s, left_budget).len() + } + + fn pick_suffix_start(s: &str, right_budget: usize) -> usize { + let start_tail = s.len().saturating_sub(right_budget); + if let Some(tail) = s.get(start_tail..) + && let Some(i) = tail.find('\n') + { + return start_tail + i + 1; + } + + let mut idx = start_tail.min(s.len()); + while idx < s.len() && !s.is_char_boundary(idx) { + idx += 1; + } + idx + } + + let mut guess_tokens = est_tokens; + for _ in 0..4 { + let marker = format!("…{guess_tokens} tokens truncated…"); + let marker_len = marker.len(); + let keep_budget = max_bytes.saturating_sub(marker_len); + if keep_budget == 0 { + return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); + } + + let left_budget = keep_budget / 2; + let right_budget = keep_budget - left_budget; + let prefix_end = pick_prefix_end(s, left_budget); + let mut suffix_start = pick_suffix_start(s, right_budget); + if suffix_start < prefix_end { + suffix_start = prefix_end; + } + + let kept_content_bytes = prefix_end + (s.len() - suffix_start); + let truncated_content_bytes = s.len().saturating_sub(kept_content_bytes); + let new_tokens = (truncated_content_bytes as u64).div_ceil(4); + + if new_tokens == guess_tokens { + let mut out = String::with_capacity(marker_len + kept_content_bytes + 1); + out.push_str(&s[..prefix_end]); + out.push_str(&marker); + out.push('\n'); + out.push_str(&s[suffix_start..]); + return (out, Some(est_tokens)); + } + + guess_tokens = new_tokens; + } + + let marker = format!("…{guess_tokens} tokens truncated…"); + let marker_len = marker.len(); + let keep_budget = max_bytes.saturating_sub(marker_len); + if keep_budget == 0 { + return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); + } + + let left_budget = keep_budget / 2; + let right_budget = keep_budget - left_budget; + let prefix_end = pick_prefix_end(s, left_budget); + let suffix_start = pick_suffix_start(s, right_budget); + + let mut out = String::with_capacity(marker_len + prefix_end + (s.len() - suffix_start) + 1); + out.push_str(&s[..prefix_end]); + out.push_str(&marker); + out.push('\n'); + out.push_str(&s[suffix_start..]); + (out, Some(est_tokens)) +} + +#[cfg(test)] +mod tests { + use super::truncate_middle; + + #[test] + fn truncate_middle_no_newlines_fallback() { + let s = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ*"; + let max_bytes = 32; + let (out, original) = truncate_middle(s, max_bytes); + assert!(out.starts_with("abc")); + assert!(out.contains("tokens truncated")); + assert!(out.ends_with("XYZ*")); + assert_eq!(original, Some((s.len() as u64).div_ceil(4))); + } + + #[test] + fn truncate_middle_prefers_newline_boundaries() { + let mut s = String::new(); + for i in 1..=20 { + s.push_str(&format!("{i:03}\n")); + } + assert_eq!(s.len(), 80); + + let max_bytes = 64; + let (out, tokens) = truncate_middle(&s, max_bytes); + assert!(out.starts_with("001\n002\n003\n004\n")); + assert!(out.contains("tokens truncated")); + assert!(out.ends_with("017\n018\n019\n020\n")); + assert_eq!(tokens, Some(20)); + } + + #[test] + fn truncate_middle_handles_utf8_content() { + let s = "😀😀😀😀😀😀😀😀😀😀\nsecond line with ascii text\n"; + let max_bytes = 32; + let (out, tokens) = truncate_middle(s, max_bytes); + + assert!(out.contains("tokens truncated")); + assert!(!out.contains('\u{fffd}')); + assert_eq!(tokens, Some((s.len() as u64).div_ceil(4))); + } + + #[test] + fn truncate_middle_prefers_newline_boundaries_2() { + // Build a multi-line string of 20 numbered lines (each "NNN\n"). + let mut s = String::new(); + for i in 1..=20 { + s.push_str(&format!("{i:03}\n")); + } + // Total length: 20 lines * 4 bytes per line = 80 bytes. + assert_eq!(s.len(), 80); + + // Choose a cap that forces truncation while leaving room for + // a few lines on each side after accounting for the marker. + let max_bytes = 64; + // Expect exact output: first 4 lines, marker, last 4 lines, and correct token estimate (80/4 = 20). + assert_eq!( + truncate_middle(&s, max_bytes), + ( + r#"001 +002 +003 +004 +…12 tokens truncated… +017 +018 +019 +020 +"# + .to_string(), + Some(20) + ) + ); + } +} diff --git a/codex-rs/core/Cargo.toml b/codex-rs/core/Cargo.toml index a1e7876a26..1225f966ae 100644 --- a/codex-rs/core/Cargo.toml +++ b/codex-rs/core/Cargo.toml @@ -20,6 +20,7 @@ base64 = { workspace = true } bytes = { workspace = true } chrono = { workspace = true, features = ["serde"] } codex-apply-patch = { workspace = true } +codex-agent = { workspace = true } codex-file-search = { workspace = true } codex-mcp-client = { workspace = true } codex-protocol = { workspace = true } diff --git a/codex-rs/core/src/agent_config.rs b/codex-rs/core/src/agent_config.rs new file mode 100644 index 0000000000..f7e3d78496 --- /dev/null +++ b/codex-rs/core/src/agent_config.rs @@ -0,0 +1,81 @@ +use std::collections::HashMap; +use std::path::PathBuf; + +use crate::config::Config; +use crate::config_types::History; +use crate::config_types::McpServerConfig; +use crate::config_types::ShellEnvironmentPolicy; +use crate::model_family::ModelFamily; +use crate::model_provider_info::ModelProviderInfo; +use codex_protocol::config_types::ReasoningEffort; +use codex_protocol::config_types::ReasoningSummary; +use codex_protocol::config_types::Verbosity; +use codex_protocol::protocol::AskForApproval; +use codex_protocol::protocol::SandboxPolicy; + +#[derive(Debug, Clone, PartialEq)] +pub struct AgentConfig { + pub model: String, + pub review_model: String, + pub model_family: ModelFamily, + pub model_context_window: Option, + pub model_auto_compact_token_limit: Option, + pub model_reasoning_effort: Option, + pub model_reasoning_summary: ReasoningSummary, + pub model_verbosity: Option, + pub model_provider: ModelProviderInfo, + pub approval_policy: AskForApproval, + pub sandbox_policy: SandboxPolicy, + pub shell_environment_policy: ShellEnvironmentPolicy, + pub user_instructions: Option, + pub base_instructions: Option, + pub notify: Option>, + pub cwd: PathBuf, + pub codex_home: PathBuf, + pub history: History, + pub mcp_servers: HashMap, + pub include_plan_tool: bool, + pub include_apply_patch_tool: bool, + pub include_view_image_tool: bool, + pub tools_web_search_request: bool, + pub use_experimental_streamable_shell_tool: bool, + pub use_experimental_unified_exec_tool: bool, + pub show_raw_agent_reasoning: bool, + pub codex_linux_sandbox_exe: Option, + pub project_doc_max_bytes: usize, +} + +impl From<&Config> for AgentConfig { + fn from(config: &Config) -> Self { + Self { + model: config.model.clone(), + review_model: config.review_model.clone(), + model_family: config.model_family.clone(), + model_context_window: config.model_context_window, + model_auto_compact_token_limit: config.model_auto_compact_token_limit, + model_reasoning_effort: config.model_reasoning_effort, + model_reasoning_summary: config.model_reasoning_summary, + model_verbosity: config.model_verbosity, + model_provider: config.model_provider.clone(), + approval_policy: config.approval_policy, + sandbox_policy: config.sandbox_policy.clone(), + shell_environment_policy: config.shell_environment_policy.clone(), + user_instructions: config.user_instructions.clone(), + base_instructions: config.base_instructions.clone(), + notify: config.notify.clone(), + cwd: config.cwd.clone(), + codex_home: config.codex_home.clone(), + history: config.history.clone(), + mcp_servers: config.mcp_servers.clone(), + include_plan_tool: config.include_plan_tool, + include_apply_patch_tool: config.include_apply_patch_tool, + include_view_image_tool: config.include_view_image_tool, + tools_web_search_request: config.tools_web_search_request, + use_experimental_streamable_shell_tool: config.use_experimental_streamable_shell_tool, + use_experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + show_raw_agent_reasoning: config.show_raw_agent_reasoning, + codex_linux_sandbox_exe: config.codex_linux_sandbox_exe.clone(), + project_doc_max_bytes: config.project_doc_max_bytes, + } + } +} diff --git a/codex-rs/core/src/agent_services.rs b/codex-rs/core/src/agent_services.rs new file mode 100644 index 0000000000..9f6f088d20 --- /dev/null +++ b/codex-rs/core/src/agent_services.rs @@ -0,0 +1,209 @@ +use anyhow::Result; +use async_trait::async_trait; +use mcp_types::CallToolResult; +use mcp_types::Tool; +use serde_json::Value; +use std::collections::HashMap; +use std::path::PathBuf; + +use crate::auth::AuthManager; +use crate::auth::CodexAuth; +use crate::exec_command::ExecCommandOutput; +use crate::exec_command::ExecCommandParams; +use crate::exec_command::ExecSessionManager; +use crate::exec_command::WriteStdinParams; +use crate::mcp_connection_manager::McpConnectionManager; +use crate::rollout::RolloutRecorder; +use crate::unified_exec::UnifiedExecError; +use crate::unified_exec::UnifiedExecRequest; +use crate::unified_exec::UnifiedExecResult; +use crate::unified_exec::UnifiedExecSessionManager; +use crate::user_notification::UserNotification; +use crate::user_notification::UserNotifier; + +/// Provides access to credentials required when talking to model providers. +#[async_trait] +pub trait CredentialsProvider: Send + Sync { + fn auth(&self) -> Option; + + async fn refresh_token(&self) -> std::io::Result>; +} + +#[async_trait] +impl CredentialsProvider for AuthManager { + fn auth(&self) -> Option { + AuthManager::auth(self) + } + + async fn refresh_token(&self) -> std::io::Result> { + AuthManager::refresh_token(self).await + } +} + +/// Emits user-facing notifications for turn completion or other events. +pub trait Notifier: Send + Sync { + fn notify(&self, notification: &UserNotification); +} + +impl Notifier for UserNotifier { + fn notify(&self, notification: &UserNotification) { + UserNotifier::notify(self, notification); + } +} + +/// Aggregates and dispatches MCP tool calls across configured servers. +#[async_trait] +pub trait McpInterface: Send + Sync { + fn list_all_tools(&self) -> HashMap; + + fn parse_tool_name(&self, tool_name: &str) -> Option<(String, String)>; + + async fn call_tool( + &self, + server: &str, + tool: &str, + arguments: Option, + ) -> Result; +} + +#[async_trait] +impl McpInterface for McpConnectionManager { + fn list_all_tools(&self) -> HashMap { + McpConnectionManager::list_all_tools(self) + } + + fn parse_tool_name(&self, tool_name: &str) -> Option<(String, String)> { + McpConnectionManager::parse_tool_name(self, tool_name) + } + + async fn call_tool( + &self, + server: &str, + tool: &str, + arguments: Option, + ) -> Result { + McpConnectionManager::call_tool(self, server, tool, arguments).await + } +} + +/// Persists rollout events for later inspection or replay. +#[async_trait] +pub trait RolloutSink: Send + Sync { + async fn record_items( + &self, + items: &[codex_protocol::protocol::RolloutItem], + ) -> std::io::Result<()>; + + async fn flush(&self) -> std::io::Result<()>; + + async fn shutdown(&self) -> std::io::Result<()>; + + fn get_rollout_path(&self) -> PathBuf; +} + +#[async_trait] +impl RolloutSink for RolloutRecorder { + async fn record_items( + &self, + items: &[codex_protocol::protocol::RolloutItem], + ) -> std::io::Result<()> { + RolloutRecorder::record_items(self, items).await + } + + async fn flush(&self) -> std::io::Result<()> { + RolloutRecorder::flush(self).await + } + + async fn shutdown(&self) -> std::io::Result<()> { + RolloutRecorder::shutdown(self).await + } + + fn get_rollout_path(&self) -> PathBuf { + RolloutRecorder::get_rollout_path(self) + } +} + +/// Handles sandboxed exec orchestration, including long-running sessions. +#[async_trait] +pub trait SandboxManager: Send + Sync { + async fn handle_exec_command_request( + &self, + params: ExecCommandParams, + ) -> Result; + + async fn handle_write_stdin_request( + &self, + params: WriteStdinParams, + ) -> Result; + + async fn handle_unified_exec_request( + &self, + request: UnifiedExecRequest<'_>, + ) -> Result; + + fn codex_linux_sandbox_exe(&self) -> &Option; + + fn user_shell(&self) -> &crate::shell::Shell; +} + +/// Default [`SandboxManager`] used by the CLI runtime. Wraps the existing exec +/// session managers and exposes their functionality via the trait-based +/// interface so other hosts can substitute different implementations. +pub struct DefaultSandboxManager { + exec_session_manager: ExecSessionManager, + unified_exec_manager: UnifiedExecSessionManager, + codex_linux_sandbox_exe: Option, + user_shell: crate::shell::Shell, +} + +impl DefaultSandboxManager { + pub fn new( + exec_session_manager: ExecSessionManager, + unified_exec_manager: UnifiedExecSessionManager, + codex_linux_sandbox_exe: Option, + user_shell: crate::shell::Shell, + ) -> Self { + Self { + exec_session_manager, + unified_exec_manager, + codex_linux_sandbox_exe, + user_shell, + } + } +} + +#[async_trait] +impl SandboxManager for DefaultSandboxManager { + async fn handle_exec_command_request( + &self, + params: ExecCommandParams, + ) -> Result { + self.exec_session_manager + .handle_exec_command_request(params) + .await + } + + async fn handle_write_stdin_request( + &self, + params: WriteStdinParams, + ) -> Result { + self.exec_session_manager + .handle_write_stdin_request(params) + .await + } + + async fn handle_unified_exec_request( + &self, + request: UnifiedExecRequest<'_>, + ) -> Result { + self.unified_exec_manager.handle_request(request).await + } + + fn codex_linux_sandbox_exe(&self) -> &Option { + &self.codex_linux_sandbox_exe + } + + fn user_shell(&self) -> &crate::shell::Shell { + &self.user_shell + } +} diff --git a/codex-rs/core/src/client.rs b/codex-rs/core/src/client.rs index aedea4887b..c0c4bad495 100644 --- a/codex-rs/core/src/client.rs +++ b/codex-rs/core/src/client.rs @@ -1,9 +1,10 @@ +use std::fmt; use std::io::BufRead; use std::path::Path; use std::sync::OnceLock; use std::time::Duration; -use crate::AuthManager; +use crate::agent_services::CredentialsProvider; use crate::auth::CodexAuth; use bytes::Bytes; use codex_protocol::mcp_protocol::AuthMode; @@ -23,6 +24,7 @@ use tracing::debug; use tracing::trace; use tracing::warn; +use crate::agent_config::AgentConfig; use crate::chat_completions::AggregateStreamExt; use crate::chat_completions::stream_chat_completions; use crate::client_common::Prompt; @@ -31,7 +33,6 @@ use crate::client_common::ResponseStream; use crate::client_common::ResponsesApiRequest; use crate::client_common::create_reasoning_param_for_request; use crate::client_common::create_text_param_for_request; -use crate::config::Config; use crate::default_client::create_client; use crate::error::CodexErr; use crate::error::Result; @@ -69,10 +70,10 @@ struct Error { resets_in_seconds: Option, } -#[derive(Debug, Clone)] +#[derive(Clone)] pub struct ModelClient { - config: Arc, - auth_manager: Option>, + config: Arc, + auth_manager: Option>, client: reqwest::Client, provider: ModelProviderInfo, conversation_id: ConversationId, @@ -80,10 +81,22 @@ pub struct ModelClient { summary: ReasoningSummaryConfig, } +impl fmt::Debug for ModelClient { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ModelClient") + .field("config", &self.config) + .field("provider", &self.provider) + .field("conversation_id", &self.conversation_id) + .field("effort", &self.effort) + .field("summary", &self.summary) + .finish() + } +} + impl ModelClient { pub fn new( - config: Arc, - auth_manager: Option>, + config: Arc, + auth_manager: Option>, provider: ModelProviderInfo, effort: Option, summary: ReasoningSummaryConfig, @@ -259,7 +272,7 @@ impl ModelClient { async fn attempt_stream_responses( &self, payload_json: &Value, - auth_manager: &Option>, + auth_manager: &Option>, ) -> std::result::Result { // Always fetch the latest auth in case a prior attempt refreshed the token. let auth = auth_manager.as_ref().and_then(|m| m.auth()); @@ -419,7 +432,7 @@ impl ModelClient { self.summary } - pub fn get_auth_manager(&self) -> Option> { + pub fn get_auth_manager(&self) -> Option> { self.auth_manager.clone() } } diff --git a/codex-rs/core/src/codex.rs b/codex-rs/core/src/codex.rs index 7092260369..3177429bdc 100644 --- a/codex-rs/core/src/codex.rs +++ b/codex-rs/core/src/codex.rs @@ -40,6 +40,13 @@ use tracing::trace; use tracing::warn; use crate::ModelProviderInfo; +use crate::agent_config::AgentConfig; +use crate::agent_services::CredentialsProvider; +use crate::agent_services::DefaultSandboxManager; +use crate::agent_services::McpInterface; +use crate::agent_services::Notifier; +use crate::agent_services::RolloutSink; +use crate::agent_services::SandboxManager; use crate::apply_patch; use crate::apply_patch::ApplyPatchExec; use crate::apply_patch::InternalApplyPatchInvocation; @@ -177,27 +184,28 @@ impl Codex { let (tx_sub, rx_sub) = async_channel::bounded(SUBMISSION_CHANNEL_CAPACITY); let (tx_event, rx_event) = async_channel::unbounded(); - let user_instructions = get_user_instructions(&config).await; + let agent_config = AgentConfig::from(&config); + let user_instructions = get_user_instructions(&agent_config).await; - let config = Arc::new(config); + let agent_config = Arc::new(agent_config); let configure_session = ConfigureSession { - provider: config.model_provider.clone(), - model: config.model.clone(), - model_reasoning_effort: config.model_reasoning_effort, - model_reasoning_summary: config.model_reasoning_summary, + provider: agent_config.model_provider.clone(), + model: agent_config.model.clone(), + model_reasoning_effort: agent_config.model_reasoning_effort, + model_reasoning_summary: agent_config.model_reasoning_summary, user_instructions, - base_instructions: config.base_instructions.clone(), - approval_policy: config.approval_policy, - sandbox_policy: config.sandbox_policy.clone(), - notify: UserNotifier::new(config.notify.clone()), - cwd: config.cwd.clone(), + base_instructions: agent_config.base_instructions.clone(), + approval_policy: agent_config.approval_policy, + sandbox_policy: agent_config.sandbox_policy.clone(), + notify: UserNotifier::new(agent_config.notify.clone()), + cwd: agent_config.cwd.clone(), }; // Generate a unique ID for the lifetime of this Codex session. let (session, turn_context) = Session::new( configure_session, - config.clone(), + agent_config.clone(), auth_manager.clone(), tx_event.clone(), conversation_history, @@ -210,7 +218,7 @@ impl Codex { let conversation_id = session.conversation_id; // This task will run until Op::Shutdown is received. - tokio::spawn(submission_loop(session, turn_context, config, rx_sub)); + tokio::spawn(submission_loop(session, turn_context, agent_config, rx_sub)); let codex = Codex { next_id: AtomicU64::new(0), tx_sub, @@ -265,6 +273,7 @@ pub(crate) struct Session { state: Mutex, pub(crate) active_turn: Mutex>, services: SessionServices, + agent_config: Arc, next_internal_sub_id: AtomicU64, } @@ -331,7 +340,7 @@ struct ConfigureSession { impl Session { async fn new( configure_session: ConfigureSession, - config: Arc, + agent_config: Arc, auth_manager: Arc, tx_event: Sender, initial_history: InitialHistory, @@ -376,11 +385,11 @@ impl Session { // - spin up MCP connection manager // - perform default shell discovery // - load history metadata - let rollout_fut = RolloutRecorder::new(&config, rollout_params); + let rollout_fut = RolloutRecorder::new(&agent_config, rollout_params); - let mcp_fut = McpConnectionManager::new(config.mcp_servers.clone()); + let mcp_fut = McpConnectionManager::new(agent_config.mcp_servers.clone()); let default_shell_fut = shell::default_user_shell(); - let history_meta_fut = crate::message_history::history_metadata(&config); + let history_meta_fut = crate::message_history::history_metadata(&agent_config); // Join all independent futures. let (rollout_recorder, mcp_res, default_shell, (history_log_id, history_entry_count)) = @@ -390,13 +399,14 @@ impl Session { error!("failed to initialize rollout recorder: {e:#}"); anyhow::anyhow!("failed to initialize rollout recorder: {e:#}") })?; - let rollout_path = rollout_recorder.rollout_path.clone(); + let rollout_path = rollout_recorder.get_rollout_path(); + let rollout_sink: Arc = Arc::new(rollout_recorder); // Create the mutable state for the Session. let state = SessionState::new(); // Handle MCP manager result and record any startup failures. let (mcp_connection_manager, failed_clients) = match mcp_res { - Ok((mgr, failures)) => (mgr, failures), + Ok((mgr, failures)) => (Arc::new(mgr) as Arc, failures), Err(e) => { let message = format!("Failed to create MCP connection manager: {e:#}"); error!("{message}"); @@ -404,7 +414,10 @@ impl Session { id: INITIAL_SUBMIT_ID.to_owned(), msg: EventMsg::Error(ErrorEvent { message }), }); - (McpConnectionManager::default(), Default::default()) + ( + Arc::new(McpConnectionManager::default()) as Arc, + Default::default(), + ) } }; @@ -422,9 +435,10 @@ impl Session { // Now that the conversation id is final (may have been updated by resume), // construct the model client. + let credentials_provider: Option> = Some(auth_manager.clone()); let client = ModelClient::new( - config.clone(), - Some(auth_manager.clone()), + agent_config.clone(), + credentials_provider, provider.clone(), model_reasoning_effort, model_reasoning_summary, @@ -433,32 +447,36 @@ impl Session { let turn_context = TurnContext { client, tools_config: ToolsConfig::new(&ToolsConfigParams { - model_family: &config.model_family, - include_plan_tool: config.include_plan_tool, - include_apply_patch_tool: config.include_apply_patch_tool, - include_web_search_request: config.tools_web_search_request, - use_streamable_shell_tool: config.use_experimental_streamable_shell_tool, - include_view_image_tool: config.include_view_image_tool, - experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + model_family: &agent_config.model_family, + include_plan_tool: agent_config.include_plan_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, + include_web_search_request: agent_config.tools_web_search_request, + use_streamable_shell_tool: agent_config.use_experimental_streamable_shell_tool, + include_view_image_tool: agent_config.include_view_image_tool, + experimental_unified_exec_tool: agent_config.use_experimental_unified_exec_tool, }), user_instructions, base_instructions, approval_policy, sandbox_policy, - shell_environment_policy: config.shell_environment_policy.clone(), + shell_environment_policy: agent_config.shell_environment_policy.clone(), cwd, is_review_mode: false, final_output_json_schema: None, }; + let notifier: Arc = Arc::new(notify); + let sandbox: Arc = Arc::new(DefaultSandboxManager::new( + ExecSessionManager::default(), + UnifiedExecSessionManager::default(), + agent_config.codex_linux_sandbox_exe.clone(), + default_shell, + )); let services = SessionServices { - mcp_connection_manager, - session_manager: ExecSessionManager::default(), - unified_exec_manager: UnifiedExecSessionManager::default(), - notifier: notify, - rollout: Mutex::new(Some(rollout_recorder)), - codex_linux_sandbox_exe: config.codex_linux_sandbox_exe.clone(), - user_shell: default_shell, - show_raw_agent_reasoning: config.show_raw_agent_reasoning, + mcp: mcp_connection_manager, + notifier, + sandbox, + rollout: Mutex::new(Some(rollout_sink)), + show_raw_agent_reasoning: agent_config.show_raw_agent_reasoning, }; let sess = Arc::new(Session { @@ -467,6 +485,7 @@ impl Session { state: Mutex::new(state), active_turn: Mutex::new(None), services, + agent_config: agent_config.clone(), next_internal_sub_id: AtomicU64::new(0), }); @@ -1021,10 +1040,7 @@ impl Session { tool: &str, arguments: Option, ) -> anyhow::Result { - self.services - .mcp_connection_manager - .call_tool(server, tool, arguments) - .await + self.services.mcp.call_tool(server, tool, arguments).await } pub async fn interrupt_task(self: &Arc) { @@ -1045,12 +1061,12 @@ impl Session { } } - pub(crate) fn notifier(&self) -> &UserNotifier { - &self.services.notifier + pub(crate) fn notifier(&self) -> &dyn Notifier { + self.services.notifier.as_ref() } fn user_shell(&self) -> &shell::Shell { - &self.services.user_shell + self.services.sandbox.user_shell() } fn show_raw_agent_reasoning(&self) -> bool { @@ -1082,7 +1098,7 @@ pub(crate) struct ApplyPatchCommandContext { async fn submission_loop( sess: Arc, turn_context: TurnContext, - config: Arc, + agent_config: Arc, rx_sub: Receiver, ) { // Wrap once to avoid cloning TurnContext for each task. @@ -1108,8 +1124,8 @@ async fn submission_loop( // Effective model + family let (effective_model, effective_family) = if let Some(ref m) = model { - let fam = - find_family_for_model(m).unwrap_or_else(|| config.model_family.clone()); + let fam = find_family_for_model(m) + .unwrap_or_else(|| agent_config.model_family.clone()); (m.clone(), fam) } else { (prev.client.get_model(), prev.client.get_model_family()) @@ -1122,7 +1138,7 @@ async fn submission_loop( let auth_manager = prev.client.get_auth_manager(); // Build updated config for the client - let mut updated_config = (*config).clone(); + let mut updated_config = (*agent_config).clone(); updated_config.model = effective_model.clone(); updated_config.model_family = effective_family.clone(); if let Some(model_info) = get_model_info(&effective_family) { @@ -1146,12 +1162,12 @@ async fn submission_loop( let tools_config = ToolsConfig::new(&ToolsConfigParams { model_family: &effective_family, - include_plan_tool: config.include_plan_tool, - include_apply_patch_tool: config.include_apply_patch_tool, - include_web_search_request: config.tools_web_search_request, - use_streamable_shell_tool: config.use_experimental_streamable_shell_tool, - include_view_image_tool: config.include_view_image_tool, - experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + include_plan_tool: agent_config.include_plan_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, + include_web_search_request: agent_config.tools_web_search_request, + use_streamable_shell_tool: agent_config.use_experimental_streamable_shell_tool, + include_view_image_tool: agent_config.include_view_image_tool, + experimental_unified_exec_tool: agent_config.use_experimental_unified_exec_tool, }); let new_turn_context = TurnContext { @@ -1208,10 +1224,10 @@ async fn submission_loop( // Derive a model family for the requested model; fall back to the session's. let model_family = find_family_for_model(&model) - .unwrap_or_else(|| config.model_family.clone()); + .unwrap_or_else(|| agent_config.model_family.clone()); // Create a per‑turn Config clone with the requested model/family. - let mut per_turn_config = (*config).clone(); + let mut per_turn_config = (*agent_config).clone(); per_turn_config.model = model.clone(); per_turn_config.model_family = model_family.clone(); if let Some(model_info) = get_model_info(&model_family) { @@ -1233,13 +1249,13 @@ async fn submission_loop( client, tools_config: ToolsConfig::new(&ToolsConfigParams { model_family: &model_family, - include_plan_tool: config.include_plan_tool, - include_apply_patch_tool: config.include_apply_patch_tool, - include_web_search_request: config.tools_web_search_request, - use_streamable_shell_tool: config + include_plan_tool: agent_config.include_plan_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, + include_web_search_request: agent_config.tools_web_search_request, + use_streamable_shell_tool: agent_config .use_experimental_streamable_shell_tool, - include_view_image_tool: config.include_view_image_tool, - experimental_unified_exec_tool: config + include_view_image_tool: agent_config.include_view_image_tool, + experimental_unified_exec_tool: agent_config .use_experimental_unified_exec_tool, }), user_instructions: turn_context.user_instructions.clone(), @@ -1282,9 +1298,10 @@ async fn submission_loop( }, Op::AddToHistory { text } => { let id = sess.conversation_id; - let config = config.clone(); + let agent_config_clone = agent_config.clone(); tokio::spawn(async move { - if let Err(e) = crate::message_history::append_entry(&text, &id, &config).await + if let Err(e) = + crate::message_history::append_entry(&text, &id, &agent_config_clone).await { warn!("failed to append to message history: {e}"); } @@ -1292,14 +1309,14 @@ async fn submission_loop( } Op::GetHistoryEntryRequest { offset, log_id } => { - let config = config.clone(); + let agent_config_clone = agent_config.clone(); let sess_clone = sess.clone(); let sub_id = sub.id.clone(); tokio::spawn(async move { // Run lookup in blocking thread because it does file IO + locking. let entry_opt = tokio::task::spawn_blocking(move || { - crate::message_history::lookup(log_id, offset, &config) + crate::message_history::lookup(log_id, offset, &agent_config_clone) }) .await .unwrap_or(None); @@ -1328,7 +1345,7 @@ async fn submission_loop( let sub_id = sub.id.clone(); // This is a cheap lookup from the connection manager's cache. - let tools = sess.services.mcp_connection_manager.list_all_tools(); + let tools = sess.services.mcp.list_all_tools(); let event = Event { id: sub_id, msg: EventMsg::McpListToolsResponse( @@ -1427,7 +1444,7 @@ async fn submission_loop( Op::Review { review_request } => { spawn_review_thread( sess.clone(), - config.clone(), + sess.agent_config.clone(), turn_context.clone(), sub.id, review_request, @@ -1445,22 +1462,22 @@ async fn submission_loop( /// Spawn a review thread using the given prompt. async fn spawn_review_thread( sess: Arc, - config: Arc, + agent_config: Arc, parent_turn_context: Arc, sub_id: String, review_request: ReviewRequest, ) { - let model = config.review_model.clone(); + let model = agent_config.review_model.clone(); let review_model_family = find_family_for_model(&model) .unwrap_or_else(|| parent_turn_context.client.get_model_family()); let tools_config = ToolsConfig::new(&ToolsConfigParams { model_family: &review_model_family, include_plan_tool: false, - include_apply_patch_tool: config.include_apply_patch_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, include_web_search_request: false, use_streamable_shell_tool: false, include_view_image_tool: false, - experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + experimental_unified_exec_tool: agent_config.use_experimental_unified_exec_tool, }); let base_instructions = REVIEW_PROMPT.to_string(); @@ -1470,7 +1487,7 @@ async fn spawn_review_thread( let model_family = review_model_family.clone(); // Build per‑turn client with the requested model/family. - let mut per_turn_config = (*config).clone(); + let mut per_turn_config = (*agent_config).clone(); per_turn_config.model = model.clone(); per_turn_config.model_family = model_family.clone(); per_turn_config.model_reasoning_effort = Some(ReasoningEffortConfig::Low); @@ -1853,7 +1870,7 @@ async fn run_turn( ) -> CodexResult { let tools = get_openai_tools( &turn_context.tools_config, - Some(sess.services.mcp_connection_manager.list_all_tools()), + Some(sess.services.mcp.list_all_tools()), ); let prompt = Prompt { @@ -2133,9 +2150,7 @@ async fn handle_response_item( .. } => { info!("FunctionCall: {name}({arguments})"); - if let Some((server, tool_name)) = - sess.services.mcp_connection_manager.parse_tool_name(&name) - { + if let Some((server, tool_name)) = sess.services.mcp.parse_tool_name(&name) { let resp = handle_mcp_tool_call( sess, sub_id, @@ -2314,8 +2329,8 @@ async fn handle_unified_exec_tool_call( let value = sess .services - .unified_exec_manager - .handle_request(request) + .sandbox + .handle_unified_exec_request(request) .await .map_err(|err| { FunctionCallError::RespondToModel(format!("unified exec failed: {err:?}")) @@ -2433,7 +2448,7 @@ async fn handle_function_call( })?; let result = sess .services - .session_manager + .sandbox .handle_exec_command_request(exec_params) .await; match result { @@ -2451,7 +2466,7 @@ async fn handle_function_call( let result = sess .services - .session_manager + .sandbox .handle_write_stdin_request(write_stdin_params) .await .map_err(FunctionCallError::RespondToModel)?; @@ -2646,7 +2661,7 @@ async fn handle_container_exec_with_params( plan: plan_for_invocation, sandbox_policy: &turn_context.sandbox_policy, sandbox_cwd: &turn_context.cwd, - codex_linux_sandbox_exe: &sess.services.codex_linux_sandbox_exe, + codex_linux_sandbox_exe: sess.services.sandbox.codex_linux_sandbox_exe(), stdout_stream: if exec_command_context.apply_patch.is_some() { None } else { @@ -2768,7 +2783,7 @@ async fn handle_sandbox_error( plan: ExecPlan::approved(SandboxType::None, false, true), sandbox_policy: &turn_context.sandbox_policy, sandbox_cwd: &turn_context.cwd, - codex_linux_sandbox_exe: &sess.services.codex_linux_sandbox_exe, + codex_linux_sandbox_exe: sess.services.sandbox.codex_linux_sandbox_exe(), stdout_stream: if exec_command_context.apply_patch.is_some() { None } else { @@ -3323,45 +3338,48 @@ mod tests { ) .expect("load default test config"); let config = Arc::new(config); + let agent_config = Arc::new(AgentConfig::from(config.as_ref())); let conversation_id = ConversationId::default(); let client = ModelClient::new( - config.clone(), + agent_config.clone(), None, - config.model_provider.clone(), - config.model_reasoning_effort, - config.model_reasoning_summary, + agent_config.model_provider.clone(), + agent_config.model_reasoning_effort, + agent_config.model_reasoning_summary, conversation_id, ); let tools_config = ToolsConfig::new(&ToolsConfigParams { - model_family: &config.model_family, - include_plan_tool: config.include_plan_tool, - include_apply_patch_tool: config.include_apply_patch_tool, - include_web_search_request: config.tools_web_search_request, - use_streamable_shell_tool: config.use_experimental_streamable_shell_tool, - include_view_image_tool: config.include_view_image_tool, - experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + model_family: &agent_config.model_family, + include_plan_tool: agent_config.include_plan_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, + include_web_search_request: agent_config.tools_web_search_request, + use_streamable_shell_tool: agent_config.use_experimental_streamable_shell_tool, + include_view_image_tool: agent_config.include_view_image_tool, + experimental_unified_exec_tool: agent_config.use_experimental_unified_exec_tool, }); let turn_context = TurnContext { client, - cwd: config.cwd.clone(), - base_instructions: config.base_instructions.clone(), - user_instructions: config.user_instructions.clone(), - approval_policy: config.approval_policy, - sandbox_policy: config.sandbox_policy.clone(), - shell_environment_policy: config.shell_environment_policy.clone(), + cwd: agent_config.cwd.clone(), + base_instructions: agent_config.base_instructions.clone(), + user_instructions: agent_config.user_instructions.clone(), + approval_policy: agent_config.approval_policy, + sandbox_policy: agent_config.sandbox_policy.clone(), + shell_environment_policy: agent_config.shell_environment_policy.clone(), tools_config, is_review_mode: false, final_output_json_schema: None, }; let services = SessionServices { - mcp_connection_manager: McpConnectionManager::default(), - session_manager: ExecSessionManager::default(), - unified_exec_manager: UnifiedExecSessionManager::default(), - notifier: UserNotifier::default(), + mcp: Arc::new(McpConnectionManager::default()), + notifier: Arc::new(UserNotifier::default()), + sandbox: Arc::new(DefaultSandboxManager::new( + ExecSessionManager::default(), + UnifiedExecSessionManager::default(), + None, + shell::Shell::Unknown, + )), rollout: Mutex::new(None), - codex_linux_sandbox_exe: None, - user_shell: shell::Shell::Unknown, - show_raw_agent_reasoning: config.show_raw_agent_reasoning, + show_raw_agent_reasoning: agent_config.show_raw_agent_reasoning, }; let session = Session { conversation_id, @@ -3369,6 +3387,7 @@ mod tests { state: Mutex::new(SessionState::new()), active_turn: Mutex::new(None), services, + agent_config, next_internal_sub_id: AtomicU64::new(0), }; (session, turn_context) @@ -3390,45 +3409,48 @@ mod tests { ) .expect("load default test config"); let config = Arc::new(config); + let agent_config = Arc::new(AgentConfig::from(config.as_ref())); let conversation_id = ConversationId::default(); let client = ModelClient::new( - config.clone(), + agent_config.clone(), None, - config.model_provider.clone(), - config.model_reasoning_effort, - config.model_reasoning_summary, + agent_config.model_provider.clone(), + agent_config.model_reasoning_effort, + agent_config.model_reasoning_summary, conversation_id, ); let tools_config = ToolsConfig::new(&ToolsConfigParams { - model_family: &config.model_family, - include_plan_tool: config.include_plan_tool, - include_apply_patch_tool: config.include_apply_patch_tool, - include_web_search_request: config.tools_web_search_request, - use_streamable_shell_tool: config.use_experimental_streamable_shell_tool, - include_view_image_tool: config.include_view_image_tool, - experimental_unified_exec_tool: config.use_experimental_unified_exec_tool, + model_family: &agent_config.model_family, + include_plan_tool: agent_config.include_plan_tool, + include_apply_patch_tool: agent_config.include_apply_patch_tool, + include_web_search_request: agent_config.tools_web_search_request, + use_streamable_shell_tool: agent_config.use_experimental_streamable_shell_tool, + include_view_image_tool: agent_config.include_view_image_tool, + experimental_unified_exec_tool: agent_config.use_experimental_unified_exec_tool, }); let turn_context = Arc::new(TurnContext { client, - cwd: config.cwd.clone(), - base_instructions: config.base_instructions.clone(), - user_instructions: config.user_instructions.clone(), - approval_policy: config.approval_policy, - sandbox_policy: config.sandbox_policy.clone(), - shell_environment_policy: config.shell_environment_policy.clone(), + cwd: agent_config.cwd.clone(), + base_instructions: agent_config.base_instructions.clone(), + user_instructions: agent_config.user_instructions.clone(), + approval_policy: agent_config.approval_policy, + sandbox_policy: agent_config.sandbox_policy.clone(), + shell_environment_policy: agent_config.shell_environment_policy.clone(), tools_config, is_review_mode: false, final_output_json_schema: None, }); let services = SessionServices { - mcp_connection_manager: McpConnectionManager::default(), - session_manager: ExecSessionManager::default(), - unified_exec_manager: UnifiedExecSessionManager::default(), - notifier: UserNotifier::default(), + mcp: Arc::new(McpConnectionManager::default()), + notifier: Arc::new(UserNotifier::default()), + sandbox: Arc::new(DefaultSandboxManager::new( + ExecSessionManager::default(), + UnifiedExecSessionManager::default(), + None, + shell::Shell::Unknown, + )), rollout: Mutex::new(None), - codex_linux_sandbox_exe: None, - user_shell: shell::Shell::Unknown, - show_raw_agent_reasoning: config.show_raw_agent_reasoning, + show_raw_agent_reasoning: agent_config.show_raw_agent_reasoning, }; let session = Arc::new(Session { conversation_id, @@ -3436,6 +3458,7 @@ mod tests { state: Mutex::new(SessionState::new()), active_turn: Mutex::new(None), services, + agent_config, next_internal_sub_id: AtomicU64::new(0), }); (session, turn_context, rx_event) diff --git a/codex-rs/core/src/config_types.rs b/codex-rs/core/src/config_types.rs index d273b23d69..045e4a3b08 100644 --- a/codex-rs/core/src/config_types.rs +++ b/codex-rs/core/src/config_types.rs @@ -1,305 +1,3 @@ -//! Types used to define the fields of [`crate::config::Config`]. +//! Re-exported configuration data structures now defined in `codex-agent`. -// Note this file should generally be restricted to simple struct/enum -// definitions that do not contain business logic. - -use std::collections::HashMap; -use std::path::PathBuf; -use std::time::Duration; -use wildmatch::WildMatchPattern; - -use serde::Deserialize; -use serde::Deserializer; -use serde::Serialize; -use serde::de::Error as SerdeError; - -#[derive(Serialize, Debug, Clone, PartialEq)] -pub struct McpServerConfig { - pub command: String, - - #[serde(default)] - pub args: Vec, - - #[serde(default)] - pub env: Option>, - - /// Startup timeout in seconds for initializing MCP server & initially listing tools. - #[serde( - default, - with = "option_duration_secs", - skip_serializing_if = "Option::is_none" - )] - pub startup_timeout_sec: Option, - - /// Default timeout for MCP tool calls initiated via this server. - #[serde(default, with = "option_duration_secs")] - pub tool_timeout_sec: Option, -} - -impl<'de> Deserialize<'de> for McpServerConfig { - fn deserialize(deserializer: D) -> Result - where - D: Deserializer<'de>, - { - #[derive(Deserialize)] - struct RawMcpServerConfig { - command: String, - #[serde(default)] - args: Vec, - #[serde(default)] - env: Option>, - #[serde(default)] - startup_timeout_sec: Option, - #[serde(default)] - startup_timeout_ms: Option, - #[serde(default, with = "option_duration_secs")] - tool_timeout_sec: Option, - } - - let raw = RawMcpServerConfig::deserialize(deserializer)?; - - let startup_timeout_sec = match (raw.startup_timeout_sec, raw.startup_timeout_ms) { - (Some(sec), _) => { - let duration = Duration::try_from_secs_f64(sec).map_err(SerdeError::custom)?; - Some(duration) - } - (None, Some(ms)) => Some(Duration::from_millis(ms)), - (None, None) => None, - }; - - Ok(Self { - command: raw.command, - args: raw.args, - env: raw.env, - startup_timeout_sec, - tool_timeout_sec: raw.tool_timeout_sec, - }) - } -} - -mod option_duration_secs { - use serde::Deserialize; - use serde::Deserializer; - use serde::Serializer; - use std::time::Duration; - - pub fn serialize(value: &Option, serializer: S) -> Result - where - S: Serializer, - { - match value { - Some(duration) => serializer.serialize_some(&duration.as_secs_f64()), - None => serializer.serialize_none(), - } - } - - pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> - where - D: Deserializer<'de>, - { - let secs = Option::::deserialize(deserializer)?; - secs.map(|secs| Duration::try_from_secs_f64(secs).map_err(serde::de::Error::custom)) - .transpose() - } -} - -#[derive(Deserialize, Debug, Copy, Clone, PartialEq)] -pub enum UriBasedFileOpener { - #[serde(rename = "vscode")] - VsCode, - - #[serde(rename = "vscode-insiders")] - VsCodeInsiders, - - #[serde(rename = "windsurf")] - Windsurf, - - #[serde(rename = "cursor")] - Cursor, - - /// Option to disable the URI-based file opener. - #[serde(rename = "none")] - None, -} - -impl UriBasedFileOpener { - pub fn get_scheme(&self) -> Option<&str> { - match self { - UriBasedFileOpener::VsCode => Some("vscode"), - UriBasedFileOpener::VsCodeInsiders => Some("vscode-insiders"), - UriBasedFileOpener::Windsurf => Some("windsurf"), - UriBasedFileOpener::Cursor => Some("cursor"), - UriBasedFileOpener::None => None, - } - } -} - -/// Settings that govern if and what will be written to `~/.codex/history.jsonl`. -#[derive(Deserialize, Debug, Clone, PartialEq, Default)] -pub struct History { - /// If true, history entries will not be written to disk. - pub persistence: HistoryPersistence, - - /// If set, the maximum size of the history file in bytes. - /// TODO(mbolin): Not currently honored. - pub max_bytes: Option, -} - -#[derive(Deserialize, Debug, Copy, Clone, PartialEq, Default)] -#[serde(rename_all = "kebab-case")] -pub enum HistoryPersistence { - /// Save all history entries to disk. - #[default] - SaveAll, - /// Do not write history to disk. - None, -} - -#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] -#[serde(untagged)] -pub enum Notifications { - Enabled(bool), - Custom(Vec), -} - -impl Default for Notifications { - fn default() -> Self { - Self::Enabled(false) - } -} - -/// Collection of settings that are specific to the TUI. -#[derive(Deserialize, Debug, Clone, PartialEq, Default)] -pub struct Tui { - /// Enable desktop notifications from the TUI when the terminal is unfocused. - /// Defaults to `false`. - #[serde(default)] - pub notifications: Notifications, -} - -#[derive(Deserialize, Debug, Clone, PartialEq, Default)] -pub struct SandboxWorkspaceWrite { - #[serde(default)] - pub writable_roots: Vec, - #[serde(default)] - pub network_access: bool, - #[serde(default)] - pub exclude_tmpdir_env_var: bool, - #[serde(default)] - pub exclude_slash_tmp: bool, -} - -impl From for codex_protocol::mcp_protocol::SandboxSettings { - fn from(sandbox_workspace_write: SandboxWorkspaceWrite) -> Self { - Self { - writable_roots: sandbox_workspace_write.writable_roots, - network_access: Some(sandbox_workspace_write.network_access), - exclude_tmpdir_env_var: Some(sandbox_workspace_write.exclude_tmpdir_env_var), - exclude_slash_tmp: Some(sandbox_workspace_write.exclude_slash_tmp), - } - } -} - -#[derive(Deserialize, Debug, Clone, PartialEq, Default)] -#[serde(rename_all = "kebab-case")] -pub enum ShellEnvironmentPolicyInherit { - /// "Core" environment variables for the platform. On UNIX, this would - /// include HOME, LOGNAME, PATH, SHELL, and USER, among others. - Core, - - /// Inherits the full environment from the parent process. - #[default] - All, - - /// Do not inherit any environment variables from the parent process. - None, -} - -/// Policy for building the `env` when spawning a process via either the -/// `shell` or `local_shell` tool. -#[derive(Deserialize, Debug, Clone, PartialEq, Default)] -pub struct ShellEnvironmentPolicyToml { - pub inherit: Option, - - pub ignore_default_excludes: Option, - - /// List of regular expressions. - pub exclude: Option>, - - pub r#set: Option>, - - /// List of regular expressions. - pub include_only: Option>, - - pub experimental_use_profile: Option, -} - -pub type EnvironmentVariablePattern = WildMatchPattern<'*', '?'>; - -/// Deriving the `env` based on this policy works as follows: -/// 1. Create an initial map based on the `inherit` policy. -/// 2. If `ignore_default_excludes` is false, filter the map using the default -/// exclude pattern(s), which are: `"*KEY*"` and `"*TOKEN*"`. -/// 3. If `exclude` is not empty, filter the map using the provided patterns. -/// 4. Insert any entries from `r#set` into the map. -/// 5. If non-empty, filter the map using the `include_only` patterns. -#[derive(Debug, Clone, PartialEq, Default)] -pub struct ShellEnvironmentPolicy { - /// Starting point when building the environment. - pub inherit: ShellEnvironmentPolicyInherit, - - /// True to skip the check to exclude default environment variables that - /// contain "KEY" or "TOKEN" in their name. - pub ignore_default_excludes: bool, - - /// Environment variable names to exclude from the environment. - pub exclude: Vec, - - /// (key, value) pairs to insert in the environment. - pub r#set: HashMap, - - /// Environment variable names to retain in the environment. - pub include_only: Vec, - - /// If true, the shell profile will be used to run the command. - pub use_profile: bool, -} - -impl From for ShellEnvironmentPolicy { - fn from(toml: ShellEnvironmentPolicyToml) -> Self { - // Default to inheriting the full environment when not specified. - let inherit = toml.inherit.unwrap_or(ShellEnvironmentPolicyInherit::All); - let ignore_default_excludes = toml.ignore_default_excludes.unwrap_or(false); - let exclude = toml - .exclude - .unwrap_or_default() - .into_iter() - .map(|s| EnvironmentVariablePattern::new_case_insensitive(&s)) - .collect(); - let r#set = toml.r#set.unwrap_or_default(); - let include_only = toml - .include_only - .unwrap_or_default() - .into_iter() - .map(|s| EnvironmentVariablePattern::new_case_insensitive(&s)) - .collect(); - let use_profile = toml.experimental_use_profile.unwrap_or(false); - - Self { - inherit, - ignore_default_excludes, - exclude, - r#set, - include_only, - use_profile, - } - } -} - -#[derive(Deserialize, Debug, Clone, PartialEq, Eq, Default, Hash)] -#[serde(rename_all = "kebab-case")] -pub enum ReasoningSummaryFormat { - #[default] - None, - Experimental, -} +pub use codex_agent::config_types::*; diff --git a/codex-rs/core/src/exec_command/responses_api.rs b/codex-rs/core/src/exec_command.rs similarity index 92% rename from codex-rs/core/src/exec_command/responses_api.rs rename to codex-rs/core/src/exec_command.rs index 70b90dd425..ddf5172f88 100644 --- a/codex-rs/core/src/exec_command/responses_api.rs +++ b/codex-rs/core/src/exec_command.rs @@ -3,6 +3,12 @@ use std::collections::BTreeMap; use crate::openai_tools::JsonSchema; use crate::openai_tools::ResponsesApiTool; +pub use codex_agent::exec_command::ExecCommandOutput; +pub use codex_agent::exec_command::ExecCommandParams; +pub use codex_agent::exec_command::ExecCommandSession; +pub use codex_agent::exec_command::ExecSessionManager; +pub use codex_agent::exec_command::WriteStdinParams; + pub const EXEC_COMMAND_TOOL_NAME: &str = "exec_command"; pub const WRITE_STDIN_TOOL_NAME: &str = "write_stdin"; diff --git a/codex-rs/core/src/exec_command/mod.rs b/codex-rs/core/src/exec_command/mod.rs deleted file mode 100644 index 2cfd022510..0000000000 --- a/codex-rs/core/src/exec_command/mod.rs +++ /dev/null @@ -1,14 +0,0 @@ -mod exec_command_params; -mod exec_command_session; -mod responses_api; -mod session_id; -mod session_manager; - -pub use exec_command_params::ExecCommandParams; -pub use exec_command_params::WriteStdinParams; -pub(crate) use exec_command_session::ExecCommandSession; -pub use responses_api::EXEC_COMMAND_TOOL_NAME; -pub use responses_api::WRITE_STDIN_TOOL_NAME; -pub use responses_api::create_exec_command_tool_for_responses_api; -pub use responses_api::create_write_stdin_tool_for_responses_api; -pub use session_manager::SessionManager as ExecSessionManager; diff --git a/codex-rs/core/src/lib.rs b/codex-rs/core/src/lib.rs index e78b546f6d..19d3cf0246 100644 --- a/codex-rs/core/src/lib.rs +++ b/codex-rs/core/src/lib.rs @@ -5,6 +5,8 @@ // the TUI or the tracing stack). #![deny(clippy::print_stdout, clippy::print_stderr)] +pub mod agent_config; +pub mod agent_services; mod apply_patch; pub mod auth; pub mod bash; @@ -92,6 +94,13 @@ pub use codex_protocol::protocol; // as those in the protocol crate when constructing protocol messages. pub use codex_protocol::config_types as protocol_config_types; +pub use agent_config::AgentConfig; +pub use agent_services::CredentialsProvider; +pub use agent_services::DefaultSandboxManager; +pub use agent_services::McpInterface; +pub use agent_services::Notifier; +pub use agent_services::RolloutSink; +pub use agent_services::SandboxManager; pub use client::ModelClient; pub use client_common::Prompt; pub use client_common::REVIEW_PROMPT; diff --git a/codex-rs/core/src/message_history.rs b/codex-rs/core/src/message_history.rs index bfd8bf0a9e..df704c8bcd 100644 --- a/codex-rs/core/src/message_history.rs +++ b/codex-rs/core/src/message_history.rs @@ -27,7 +27,7 @@ use std::time::Duration; use tokio::fs; use tokio::io::AsyncReadExt; -use crate::config::Config; +use crate::agent_config::AgentConfig; use crate::config_types::HistoryPersistence; use codex_protocol::mcp_protocol::ConversationId; @@ -49,7 +49,7 @@ pub struct HistoryEntry { pub text: String, } -fn history_filepath(config: &Config) -> PathBuf { +fn history_filepath(config: &AgentConfig) -> PathBuf { let mut path = config.codex_home.clone(); path.push(HISTORY_FILENAME); path @@ -61,7 +61,7 @@ fn history_filepath(config: &Config) -> PathBuf { pub(crate) async fn append_entry( text: &str, conversation_id: &ConversationId, - config: &Config, + config: &AgentConfig, ) -> Result<()> { match config.history.persistence { HistoryPersistence::SaveAll => { @@ -140,7 +140,7 @@ pub(crate) async fn append_entry( /// Asynchronously fetch the history file's *identifier* (inode on Unix) and /// the current number of entries by counting newline characters. -pub(crate) async fn history_metadata(config: &Config) -> (u64, usize) { +pub(crate) async fn history_metadata(config: &AgentConfig) -> (u64, usize) { let path = history_filepath(config); #[cfg(unix)] @@ -187,7 +187,7 @@ pub(crate) async fn history_metadata(config: &Config) -> (u64, usize) { /// Note this function is not async because it uses a sync advisory file /// locking API. #[cfg(unix)] -pub(crate) fn lookup(log_id: u64, offset: usize, config: &Config) -> Option { +pub(crate) fn lookup(log_id: u64, offset: usize, config: &AgentConfig) -> Option { use std::io::BufRead; use std::io::BufReader; use std::os::unix::fs::MetadataExt; diff --git a/codex-rs/core/src/project_doc.rs b/codex-rs/core/src/project_doc.rs index 549248fc11..f073e5682d 100644 --- a/codex-rs/core/src/project_doc.rs +++ b/codex-rs/core/src/project_doc.rs @@ -12,7 +12,7 @@ //! that order. //! 3. We do **not** walk past the Git root. -use crate::config::Config; +use crate::agent_config::AgentConfig; use std::path::PathBuf; use tokio::io::AsyncReadExt; use tracing::error; @@ -26,7 +26,7 @@ const PROJECT_DOC_SEPARATOR: &str = "\n\n--- project-doc ---\n\n"; /// Combines `Config::instructions` and `AGENTS.md` (if present) into a single /// string of instructions. -pub(crate) async fn get_user_instructions(config: &Config) -> Option { +pub(crate) async fn get_user_instructions(config: &AgentConfig) -> Option { match read_project_docs(config).await { Ok(Some(project_doc)) => match &config.user_instructions { Some(original_instructions) => Some(format!( @@ -48,7 +48,7 @@ pub(crate) async fn get_user_instructions(config: &Config) -> Option { /// concatenation of all discovered docs. If no documentation file is found the /// function returns `Ok(None)`. Unexpected I/O failures bubble up as `Err` so /// callers can decide how to handle them. -pub async fn read_project_docs(config: &Config) -> std::io::Result> { +pub async fn read_project_docs(config: &AgentConfig) -> std::io::Result> { let max_total = config.project_doc_max_bytes; if max_total == 0 { @@ -106,7 +106,7 @@ pub async fn read_project_docs(config: &Config) -> std::io::Result std::io::Result> { +pub fn discover_project_doc_paths(config: &AgentConfig) -> std::io::Result> { let mut dir = config.cwd.clone(); if let Ok(canon) = dir.canonicalize() { dir = canon; @@ -176,6 +176,7 @@ pub fn discover_project_doc_paths(config: &Config) -> std::io::Result) -> Config { + fn make_config(root: &TempDir, limit: usize, instructions: Option<&str>) -> AgentConfig { let codex_home = TempDir::new().unwrap(); let mut config = Config::load_from_base_config_with_overrides( ConfigToml::default(), @@ -199,7 +200,8 @@ mod tests { config.project_doc_max_bytes = limit; config.user_instructions = instructions.map(ToOwned::to_owned); - config + + AgentConfig::from(&config) } /// AGENTS.md missing – should yield `None`. diff --git a/codex-rs/core/src/rollout/recorder.rs b/codex-rs/core/src/rollout/recorder.rs index 23e645413f..4b6b5f60e5 100644 --- a/codex-rs/core/src/rollout/recorder.rs +++ b/codex-rs/core/src/rollout/recorder.rs @@ -23,7 +23,7 @@ use super::list::ConversationsPage; use super::list::Cursor; use super::list::get_conversations; use super::policy::is_persisted_response_item; -use crate::config::Config; +use crate::agent_config::AgentConfig; use crate::default_client::ORIGINATOR; use crate::git_info::collect_git_info; use codex_protocol::protocol::InitialHistory; @@ -96,7 +96,7 @@ impl RolloutRecorder { /// Attempt to create a new [`RolloutRecorder`]. If the sessions directory /// cannot be created or the rollout file cannot be opened we return the /// error so the caller can decide whether to disable persistence. - pub async fn new(config: &Config, params: RolloutRecorderParams) -> std::io::Result { + pub async fn new(config: &AgentConfig, params: RolloutRecorderParams) -> std::io::Result { let (file, rollout_path, meta) = match params { RolloutRecorderParams::Create { conversation_id, @@ -292,7 +292,7 @@ struct LogFileInfo { } fn create_log_file( - config: &Config, + config: &AgentConfig, conversation_id: ConversationId, ) -> std::io::Result { // Resolve ~/.codex/sessions/YYYY/MM/DD and create it if missing. diff --git a/codex-rs/core/src/state/service.rs b/codex-rs/core/src/state/service.rs index a67b9dda93..97c3184208 100644 --- a/codex-rs/core/src/state/service.rs +++ b/codex-rs/core/src/state/service.rs @@ -1,18 +1,14 @@ -use crate::RolloutRecorder; -use crate::exec_command::ExecSessionManager; -use crate::mcp_connection_manager::McpConnectionManager; -use crate::unified_exec::UnifiedExecSessionManager; -use crate::user_notification::UserNotifier; -use std::path::PathBuf; +use crate::agent_services::McpInterface; +use crate::agent_services::Notifier; +use crate::agent_services::RolloutSink; +use crate::agent_services::SandboxManager; +use std::sync::Arc; use tokio::sync::Mutex; pub(crate) struct SessionServices { - pub(crate) mcp_connection_manager: McpConnectionManager, - pub(crate) session_manager: ExecSessionManager, - pub(crate) unified_exec_manager: UnifiedExecSessionManager, - pub(crate) notifier: UserNotifier, - pub(crate) rollout: Mutex>, - pub(crate) codex_linux_sandbox_exe: Option, - pub(crate) user_shell: crate::shell::Shell, + pub(crate) mcp: Arc, + pub(crate) notifier: Arc, + pub(crate) sandbox: Arc, + pub(crate) rollout: Mutex>>, pub(crate) show_raw_agent_reasoning: bool, } diff --git a/codex-rs/core/src/truncate.rs b/codex-rs/core/src/truncate.rs index ab0158720b..3e62aa323f 100644 --- a/codex-rs/core/src/truncate.rs +++ b/codex-rs/core/src/truncate.rs @@ -1,180 +1 @@ -//! Utilities for truncating large chunks of output while preserving a prefix -//! and suffix on UTF-8 boundaries. - -/// Truncate the middle of a UTF-8 string to at most `max_bytes` bytes, -/// preserving the beginning and the end. Returns the possibly truncated -/// string and `Some(original_token_count)` (estimated at 4 bytes/token) -/// if truncation occurred; otherwise returns the original string and `None`. -pub(crate) fn truncate_middle(s: &str, max_bytes: usize) -> (String, Option) { - if s.len() <= max_bytes { - return (s.to_string(), None); - } - - let est_tokens = (s.len() as u64).div_ceil(4); - if max_bytes == 0 { - return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); - } - - fn truncate_on_boundary(input: &str, max_len: usize) -> &str { - if input.len() <= max_len { - return input; - } - let mut end = max_len; - while end > 0 && !input.is_char_boundary(end) { - end -= 1; - } - &input[..end] - } - - fn pick_prefix_end(s: &str, left_budget: usize) -> usize { - if let Some(head) = s.get(..left_budget) - && let Some(i) = head.rfind('\n') - { - return i + 1; - } - truncate_on_boundary(s, left_budget).len() - } - - fn pick_suffix_start(s: &str, right_budget: usize) -> usize { - let start_tail = s.len().saturating_sub(right_budget); - if let Some(tail) = s.get(start_tail..) - && let Some(i) = tail.find('\n') - { - return start_tail + i + 1; - } - - let mut idx = start_tail.min(s.len()); - while idx < s.len() && !s.is_char_boundary(idx) { - idx += 1; - } - idx - } - - let mut guess_tokens = est_tokens; - for _ in 0..4 { - let marker = format!("…{guess_tokens} tokens truncated…"); - let marker_len = marker.len(); - let keep_budget = max_bytes.saturating_sub(marker_len); - if keep_budget == 0 { - return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); - } - - let left_budget = keep_budget / 2; - let right_budget = keep_budget - left_budget; - let prefix_end = pick_prefix_end(s, left_budget); - let mut suffix_start = pick_suffix_start(s, right_budget); - if suffix_start < prefix_end { - suffix_start = prefix_end; - } - - let kept_content_bytes = prefix_end + (s.len() - suffix_start); - let truncated_content_bytes = s.len().saturating_sub(kept_content_bytes); - let new_tokens = (truncated_content_bytes as u64).div_ceil(4); - - if new_tokens == guess_tokens { - let mut out = String::with_capacity(marker_len + kept_content_bytes + 1); - out.push_str(&s[..prefix_end]); - out.push_str(&marker); - out.push('\n'); - out.push_str(&s[suffix_start..]); - return (out, Some(est_tokens)); - } - - guess_tokens = new_tokens; - } - - let marker = format!("…{guess_tokens} tokens truncated…"); - let marker_len = marker.len(); - let keep_budget = max_bytes.saturating_sub(marker_len); - if keep_budget == 0 { - return (format!("…{est_tokens} tokens truncated…"), Some(est_tokens)); - } - - let left_budget = keep_budget / 2; - let right_budget = keep_budget - left_budget; - let prefix_end = pick_prefix_end(s, left_budget); - let suffix_start = pick_suffix_start(s, right_budget); - - let mut out = String::with_capacity(marker_len + prefix_end + (s.len() - suffix_start) + 1); - out.push_str(&s[..prefix_end]); - out.push_str(&marker); - out.push('\n'); - out.push_str(&s[suffix_start..]); - (out, Some(est_tokens)) -} - -#[cfg(test)] -mod tests { - use super::truncate_middle; - - #[test] - fn truncate_middle_no_newlines_fallback() { - let s = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ*"; - let max_bytes = 32; - let (out, original) = truncate_middle(s, max_bytes); - assert!(out.starts_with("abc")); - assert!(out.contains("tokens truncated")); - assert!(out.ends_with("XYZ*")); - assert_eq!(original, Some((s.len() as u64).div_ceil(4))); - } - - #[test] - fn truncate_middle_prefers_newline_boundaries() { - let mut s = String::new(); - for i in 1..=20 { - s.push_str(&format!("{i:03}\n")); - } - assert_eq!(s.len(), 80); - - let max_bytes = 64; - let (out, tokens) = truncate_middle(&s, max_bytes); - assert!(out.starts_with("001\n002\n003\n004\n")); - assert!(out.contains("tokens truncated")); - assert!(out.ends_with("017\n018\n019\n020\n")); - assert_eq!(tokens, Some(20)); - } - - #[test] - fn truncate_middle_handles_utf8_content() { - let s = "😀😀😀😀😀😀😀😀😀😀\nsecond line with ascii text\n"; - let max_bytes = 32; - let (out, tokens) = truncate_middle(s, max_bytes); - - assert!(out.contains("tokens truncated")); - assert!(!out.contains('\u{fffd}')); - assert_eq!(tokens, Some((s.len() as u64).div_ceil(4))); - } - - #[test] - fn truncate_middle_prefers_newline_boundaries_2() { - // Build a multi-line string of 20 numbered lines (each "NNN\n"). - let mut s = String::new(); - for i in 1..=20 { - s.push_str(&format!("{i:03}\n")); - } - // Total length: 20 lines * 4 bytes per line = 80 bytes. - assert_eq!(s.len(), 80); - - // Choose a cap that forces truncation while leaving room for - // a few lines on each side after accounting for the marker. - let max_bytes = 64; - // Expect exact output: first 4 lines, marker, last 4 lines, and correct token estimate (80/4 = 20). - assert_eq!( - truncate_middle(&s, max_bytes), - ( - r#"001 -002 -003 -004 -…12 tokens truncated… -017 -018 -019 -020 -"# - .to_string(), - Some(20) - ) - ); - } -} +pub use codex_agent::truncate::*; diff --git a/codex-rs/core/src/unified_exec/errors.rs b/codex-rs/core/src/unified_exec/errors.rs index 6bf5bf7ec5..3b09f335fd 100644 --- a/codex-rs/core/src/unified_exec/errors.rs +++ b/codex-rs/core/src/unified_exec/errors.rs @@ -1,7 +1,7 @@ use thiserror::Error; #[derive(Debug, Error)] -pub(crate) enum UnifiedExecError { +pub enum UnifiedExecError { #[error("Failed to create unified exec session: {pty_error}")] CreateSession { #[source] diff --git a/codex-rs/core/src/unified_exec/mod.rs b/codex-rs/core/src/unified_exec/mod.rs index 48b82e3e8c..757e55220e 100644 --- a/codex-rs/core/src/unified_exec/mod.rs +++ b/codex-rs/core/src/unified_exec/mod.rs @@ -22,27 +22,27 @@ use crate::truncate::truncate_middle; mod errors; -pub(crate) use errors::UnifiedExecError; +pub use errors::UnifiedExecError; const DEFAULT_TIMEOUT_MS: u64 = 1_000; const MAX_TIMEOUT_MS: u64 = 60_000; const UNIFIED_EXEC_OUTPUT_MAX_BYTES: usize = 128 * 1024; // 128 KiB #[derive(Debug)] -pub(crate) struct UnifiedExecRequest<'a> { +pub struct UnifiedExecRequest<'a> { pub session_id: Option, pub input_chunks: &'a [String], pub timeout_ms: Option, } #[derive(Debug, Clone, PartialEq)] -pub(crate) struct UnifiedExecResult { +pub struct UnifiedExecResult { pub session_id: Option, pub output: String, } #[derive(Debug, Default)] -pub(crate) struct UnifiedExecSessionManager { +pub struct UnifiedExecSessionManager { next_session_id: AtomicI32, sessions: Mutex>, } diff --git a/codex-rs/core/src/user_notification.rs b/codex-rs/core/src/user_notification.rs index 5eb9e98058..7e25764711 100644 --- a/codex-rs/core/src/user_notification.rs +++ b/codex-rs/core/src/user_notification.rs @@ -46,7 +46,7 @@ impl UserNotifier { /// program. #[derive(Debug, Clone, PartialEq, Serialize)] #[serde(tag = "type", rename_all = "kebab-case")] -pub(crate) enum UserNotification { +pub enum UserNotification { #[serde(rename_all = "kebab-case")] AgentTurnComplete { turn_id: String, diff --git a/codex-rs/core/tests/chat_completions_payload.rs b/codex-rs/core/tests/chat_completions_payload.rs index ba3fe9de79..c4046184bb 100644 --- a/codex-rs/core/tests/chat_completions_payload.rs +++ b/codex-rs/core/tests/chat_completions_payload.rs @@ -1,5 +1,6 @@ use std::sync::Arc; +use codex_core::AgentConfig; use codex_core::ContentItem; use codex_core::LocalShellAction; use codex_core::LocalShellExecAction; @@ -69,9 +70,10 @@ async fn run_request(input: Vec) -> Value { let effort = config.model_reasoning_effort; let summary = config.model_reasoning_summary; let config = Arc::new(config); + let agent_config = Arc::new(AgentConfig::from(config.as_ref())); let client = ModelClient::new( - Arc::clone(&config), + Arc::clone(&agent_config), None, provider, effort, diff --git a/codex-rs/core/tests/chat_completions_sse.rs b/codex-rs/core/tests/chat_completions_sse.rs index 6155d15e62..bc1d4f7a9d 100644 --- a/codex-rs/core/tests/chat_completions_sse.rs +++ b/codex-rs/core/tests/chat_completions_sse.rs @@ -1,5 +1,6 @@ use std::sync::Arc; +use codex_core::AgentConfig; use codex_core::ContentItem; use codex_core::ModelClient; use codex_core::ModelProviderInfo; @@ -62,9 +63,10 @@ async fn run_stream(sse_body: &str) -> Vec { let effort = config.model_reasoning_effort; let summary = config.model_reasoning_summary; let config = Arc::new(config); + let agent_config = Arc::new(AgentConfig::from(config.as_ref())); let client = ModelClient::new( - Arc::clone(&config), + Arc::clone(&agent_config), None, provider, effort, diff --git a/codex-rs/core/tests/suite/client.rs b/codex-rs/core/tests/suite/client.rs index a9b0bb1b76..eb8fcd2f59 100644 --- a/codex-rs/core/tests/suite/client.rs +++ b/codex-rs/core/tests/suite/client.rs @@ -1,3 +1,4 @@ +use codex_core::AgentConfig; use codex_core::CodexAuth; use codex_core::ContentItem; use codex_core::ConversationManager; @@ -661,9 +662,10 @@ async fn azure_responses_request_includes_store_and_reasoning_ids() { let effort = config.model_reasoning_effort; let summary = config.model_reasoning_summary; let config = Arc::new(config); + let agent_config = Arc::new(AgentConfig::from(config.as_ref())); let client = ModelClient::new( - Arc::clone(&config), + Arc::clone(&agent_config), None, provider, effort, diff --git a/codex-rs/core/tests/suite/mod.rs b/codex-rs/core/tests/suite/mod.rs index 0e4e725c36..868145ded5 100644 --- a/codex-rs/core/tests/suite/mod.rs +++ b/codex-rs/core/tests/suite/mod.rs @@ -12,6 +12,7 @@ mod fork_conversation; mod json_result; mod live_cli; mod model_overrides; +mod multi_task_smoke; mod prompt_caching; mod review; mod rollout_list_find; diff --git a/codex-rs/core/tests/suite/multi_task_smoke.rs b/codex-rs/core/tests/suite/multi_task_smoke.rs new file mode 100644 index 0000000000..7c1a697c0c --- /dev/null +++ b/codex-rs/core/tests/suite/multi_task_smoke.rs @@ -0,0 +1,115 @@ +use codex_core::protocol::EventMsg; +use codex_core::protocol::InputItem; +use codex_core::protocol::Op; +use codex_core::protocol::ReviewRequest; +use core_test_support::responses::ev_assistant_message; +use core_test_support::responses::ev_completed; +use core_test_support::responses::mount_sse_sequence; +use core_test_support::responses::sse; +use core_test_support::responses::start_mock_server; +use core_test_support::skip_if_no_network; +use core_test_support::test_codex::test_codex; +use core_test_support::wait_for_event; +use pretty_assertions::assert_eq; + +/// Smoke test: ensure regular, review, and compact tasks can run sequentially in +/// a single session without leaving dangling state. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn regular_review_compact_sequence() { + skip_if_no_network!(); + + let server = start_mock_server().await; + + let regular_body = sse(vec![ + ev_assistant_message("regular-msg", "First turn complete."), + ev_completed("regular-resp"), + ]); + + let review_json = serde_json::json!({ + "findings": [], + "overall_correctness": "ok", + "overall_explanation": "Looks good overall.", + "overall_confidence_score": 0.5 + }) + .to_string(); + let review_body = sse(vec![ + ev_assistant_message("review-msg", &review_json), + ev_completed("review-resp"), + ]); + + let compact_body = sse(vec![ + ev_assistant_message("compact-msg", "Session summary."), + ev_completed("compact-resp"), + ]); + + mount_sse_sequence(&server, vec![regular_body, review_body, compact_body]).await; + + let codex = test_codex().build(&server).await.unwrap().codex; + + // 1. Regular user input turn. + codex + .submit(Op::UserInput { + items: vec![InputItem::Text { + text: "first turn".into(), + }], + }) + .await + .unwrap(); + let regular_complete = + wait_for_event(&codex, |ev| matches!(ev, EventMsg::TaskComplete(_))).await; + let regular_last = match regular_complete { + EventMsg::TaskComplete(ev) => ev.last_agent_message, + other => panic!("expected TaskComplete for regular turn, got {other:?}"), + }; + assert_eq!(regular_last.as_deref(), Some("First turn complete.")); + + // 2. Review task turn. + codex + .submit(Op::Review { + review_request: ReviewRequest { + prompt: "please review".to_string(), + user_facing_hint: "hint".to_string(), + }, + }) + .await + .unwrap(); + + let _entered = wait_for_event(&codex, |ev| matches!(ev, EventMsg::EnteredReviewMode(_))).await; + let exited = wait_for_event(&codex, |ev| matches!(ev, EventMsg::ExitedReviewMode(_))).await; + let review_output = match exited { + EventMsg::ExitedReviewMode(ev) => ev.review_output.expect("expected review output"), + other => panic!("expected ExitedReviewMode, got {other:?}"), + }; + assert_eq!(review_output.overall_correctness, "ok"); + assert_eq!(review_output.overall_explanation, "Looks good overall."); + assert!(review_output.findings.is_empty()); + + let review_complete = + wait_for_event(&codex, |ev| matches!(ev, EventMsg::TaskComplete(_))).await; + let review_last = match review_complete { + EventMsg::TaskComplete(ev) => ev.last_agent_message, + other => panic!("expected TaskComplete for review, got {other:?}"), + } + .expect("review task should emit last_agent_message"); + assert_eq!(review_last, review_json); + + // 3. Manual compact task. + codex.submit(Op::Compact).await.unwrap(); + let compact_complete = + wait_for_event(&codex, |ev| matches!(ev, EventMsg::TaskComplete(_))).await; + let compact_last = match compact_complete { + EventMsg::TaskComplete(ev) => ev.last_agent_message, + other => panic!("expected TaskComplete for compact, got {other:?}"), + }; + assert!( + compact_last.is_none(), + "compact task should not emit a trailing assistant message" + ); + + let requests = server.received_requests().await.unwrap(); + assert_eq!( + requests.len(), + 3, + "expected exactly three Responses API calls" + ); +} diff --git a/codex-rs/docs/agent_runtime_baseline.md b/codex-rs/docs/agent_runtime_baseline.md new file mode 100644 index 0000000000..f44dd8cd3a --- /dev/null +++ b/codex-rs/docs/agent_runtime_baseline.md @@ -0,0 +1,77 @@ +# Agent Runtime Baseline + +Codex currently exposes its agent runtime from the `codex-core` crate. The runtime is organised around three cooperating interfaces located in `core/src`: + +- `Codex` (`core/src/codex.rs`) is the public façade that hosts use. It owns the submission/event queues and spawns the asynchronous runtime. +- `Session` (`core/src/codex.rs`) encapsulates conversation-scoped state and orchestrates task lifecycles once a session has been configured. +- `SessionTask` (`core/src/tasks/mod.rs`) is the trait implemented by the concrete task runners (`RegularTask`, `ReviewTask`, `CompactTask`). + +The refactor tracked in `../agent_refactor.md` will extract these responsibilities into a dedicated `codex-agent` crate; this document captures the current layout before the extraction begins. + +## `Codex` + +`Codex` is the high-level queue API that front-ends interact with: + +- `spawn` initialises the runtime with a `Config`, `AuthManager`, and `InitialHistory` and returns a `CodexSpawnOk` containing the queue endpoints and generated `ConversationId`. +- `submit` wraps an `Op` in a `Submission`, generates a unique id, and pushes it onto the bounded submission channel (`SUBMISSION_CHANNEL_CAPACITY = 64`). +- `next_event` pulls the next `Event` from the unbounded event receiver, propagating `CodexErr::InternalAgentDied` if the channel is closed. + +Upon spawning, `Codex` constructs a `ConfigureSession` payload that gathers CLI-derived configuration (model details, approvals, sandbox policy, notify hooks, `cwd`) and delegates to `Session::new`. It also seeds the `TurnContext`, `SessionServices`, and kicks off the background `submission_loop` task that drives the agent. + +## `Session` + +`Session` bundles the pieces required to handle a configured conversation: + +- Identifiers: `conversation_id` (originating from `InitialHistory`) and an internal `next_internal_sub_id` counter for action-specific ids. +- Communication: the `tx_event` sender used to emit `Event` messages back to the host. +- State holders: a `Mutex` for persistent session data and a `Mutex>` tracking the currently running tasks. +- Services: `SessionServices` packages dependencies that are currently hard-wired to CLI types—`ExecSessionManager`, `UnifiedExecSessionManager`, `RolloutRecorder`, `McpConnectionManager`, etc. + +`Session::new` performs the upfront wiring: it initialises rollout recording, MCP connections, default shell discovery, and history metadata in parallel; constructs the `SessionServices`; emits the initial `SessionConfigured` event; and records any startup warnings so they can be surfaced after configuration. + +Operationally, `Session` is responsible for: + +- Translating incoming `Submission`s into task invocations via `run_task`, `run_turn`, and helper functions in `core/src/codex.rs` and `core/src/tasks`. +- Managing approvals and sandbox execution by calling into `sandbox::plan_*` helpers and dispatching events (`ExecApprovalRequestEvent`, `ApplyPatchApprovalRequestEvent`, etc.). +- Recording rollout items and forwarding MCP tool call updates. +- Tracking and cancelling running work when new input arrives or when approvals are rejected. + +## `SessionTask` + +Tasks implement the asynchronous work units that execute within a session. The trait lives in `core/src/tasks/mod.rs`: + +```rust +#[async_trait] +pub(crate) trait SessionTask: Send + Sync + 'static { + fn kind(&self) -> TaskKind; + async fn run( + self: Arc, + session: Arc, + ctx: Arc, + sub_id: String, + input: Vec, + ) -> Option; + + async fn abort(&self, session: Arc, sub_id: &str) { ... } +} +``` + +`SessionTaskContext` is a thin wrapper that hands tasks a clone of the `Session`, giving them access to helpers such as `send_event`, `plan_exec`, and `run_with_plan`. `Session::spawn_task` ensures only one task runs at a time by: + +1. Calling `abort_all_tasks` to cancel the current `ActiveTurn`. +2. Wrapping the concrete task in `Arc`. +3. Spawning a Tokio task that awaits `run` and then reports completion through `Session::on_task_finished`. + +`RunningTask`, `ActiveTurn`, and `TurnAbortReason` (from `core/src/state`) coordinate cancellation semantics and surface `TurnAborted`/`TaskComplete` events consistently. + +Today the concrete implementations are: + +- `RegularTask` (`core/src/tasks/regular.rs`) for the standard Codex workflow. +- `ReviewTask` (`core/src/tasks/review.rs`) used during review mode. +- `CompactTask` (`core/src/tasks/compact.rs`) which emits summarised history. + +Each uses shared utilities in `core/src/codex.rs` (e.g., `run_task`, `exit_review_mode`, sandbox planners) and relies on the CLI-flavoured services packaged in `SessionServices`. + +## Next Steps + +With this baseline documented, the next implementation steps are described in `../agent_refactor.md`. As we move work into the new `codex-agent` crate we should revisit this document to ensure the captured interfaces stay accurate and to outline any newly introduced abstractions (`AgentRuntime`, `AgentConfig`, service traits, etc.). diff --git a/codex-rs/tui/src/status/helpers.rs b/codex-rs/tui/src/status/helpers.rs index 46185e1c0f..ae6caef5b6 100644 --- a/codex-rs/tui/src/status/helpers.rs +++ b/codex-rs/tui/src/status/helpers.rs @@ -2,6 +2,7 @@ use crate::exec_command::relativize_to_home; use crate::text_formatting; use chrono::DateTime; use chrono::Local; +use codex_core::AgentConfig; use codex_core::auth::get_auth_file; use codex_core::auth::try_read_auth_json; use codex_core::config::Config; @@ -32,7 +33,8 @@ pub(crate) fn compose_model_display( } pub(crate) fn compose_agents_summary(config: &Config) -> String { - match discover_project_doc_paths(config) { + let agent_config = AgentConfig::from(config); + match discover_project_doc_paths(&agent_config) { Ok(paths) => { let mut rels: Vec = Vec::new(); for p in paths {