## Summary
- change compaction pre-check accounting to include all items added
after the last model-generated item, not only trailing codex-generated
outputs
- use that boundary consistently in get_total_token_usage() and
get_total_token_usage_breakdown()
- update history tests to cover user/tool-output items after the last
model item
## Why
last_token_usage.total_tokens is API-reported for the last successful
model response. After that point, local history may gain additional
items (user messages, injected context, tool outputs). Compaction
triggering must account for all of those items to avoid late compaction
attempts that can overflow context.
## Testing
- just fmt
- cargo test -p codex-core
We support requirements on Unix, loading from
`/etc/codex/requirements.toml`. On MacOS, we also support MDM.
Now, on Windows, we'll load requirements from
`%ProgramData%\OpenAI\Codex\requirements.toml`
```
FAIL [ 1.903s] (1926/3311) codex-core::all suite::tool_parallelism::mixed_parallel_tools_run_in_parallel
stdout ───
running 1 test
test suite::tool_parallelism::mixed_parallel_tools_run_in_parallel ... FAILED
failures:
failures:
suite::tool_parallelism::mixed_parallel_tools_run_in_parallel
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 684 filtered out; finished in 1.86s
stderr ───
thread 'suite::tool_parallelism::mixed_parallel_tools_run_in_parallel' (205083) panicked at core/tests/suite/tool_parallelism.rs:74:5:
expected parallel execution to finish quickly, got 1.406255993s
stack backtrace:
0: __rustc::rust_begin_unwind
at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
1: core::panicking::panic_fmt
at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
2: all::suite::tool_parallelism::assert_parallel_duration
at ./tests/suite/tool_parallelism.rs:74:5
3: all::suite::tool_parallelism::mixed_parallel_tools_run_in_parallel::{{closure}}
at ./tests/suite/tool_parallelism.rs:206:5
4: <core::pin::Pin<P> as core::future::future::Future>::poll
at /home/runner/.rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:133:9
5: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/park.rs:284:71
6: tokio::task::coop::with_budget
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/task/coop/mod.rs:167:5
7: tokio::task::coop::budget
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/task/coop/mod.rs:133:5
8: tokio::runtime::park::CachedParkThread::block_on
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/park.rs:284:31
9: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/context/blocking.rs:66:14
10: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/scheduler/multi_thread/mod.rs:89:22
11: tokio::runtime::context::runtime::enter_runtime
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/context/runtime.rs:65:16
12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/scheduler/multi_thread/mod.rs:88:9
13: tokio::runtime::runtime::Runtime::block_on_inner
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/runtime.rs:370:50
14: tokio::runtime::runtime::Runtime::block_on
at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.49.0/src/runtime/runtime.rs:342:18
15: all::suite::tool_parallelism::mixed_parallel_tools_run_in_parallel
at ./tests/suite/tool_parallelism.rs:208:7
16: all::suite::tool_parallelism::mixed_parallel_tools_run_in_parallel::{{closure}}
at ./tests/suite/tool_parallelism.rs:178:52
17: core::ops::function::FnOnce::call_once
at /home/runner/.rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
18: core::ops::function::FnOnce::call_once
at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
With this PR we do not close the unified exec processes (i.e. background
terminals) at the end of a turn unless:
* The user interrupt the turn
* The user decide to clean the processes through `app-server` or
`/clean`
I made sure that `codex exec` correctly kill all the processes
This PR adds the following field to `Config`:
```rust
pub network: Option<NetworkProxy>,
```
Though for the moment, it will always be initialized as `None` (this
will be addressed in a subsequent PR).
This PR does the work to thread `network` through to `execute_exec_env()`, `process_exec_tool_call()`, and `UnifiedExecRuntime.run()` to ensure it is available whenever we span a process.
There are two concepts of apps that we load in the harness:
- Directory apps, which is all the apps that the user can install.
- Accessible apps, which is what the user actually installed and can be
$ inserted and be used by the model. These are extracted from the tools
that are loaded through the gateway MCP.
Previously we wait for both sets of apps before returning the full apps
list. Which causes many issues because accessible apps won't be
available to the UI or the model if directory apps aren't loaded or
failed to load.
In this PR we are separating them so that accessible apps can be loaded
separately and are instantly available to be shown in the UI and to be
provided in model context. We also added an app-server event so that
clients can subscribe to also get accessible apps without being blocked
on the full app list.
- [x] Separate accessible apps and directory apps loading.
- [x] `app/list` request will also emit `app/list/updated` notifications
that app-server clients can subscribe. Which allows clients to get
accessible apps list to render in the $ menu without being blocked by
directory apps.
- [x] Cache both accessible and directory apps with 1 hour TTL to avoid
reloading them when creating new threads.
- [x] TUI improvements to redraw $ menu and /apps menu when app list is
updated.
If `NetworkConstraints` is set, then include the relevant settings on `<environment_context>`. Example:
```xml
<environment_context>
<cwd>/repo</cwd>
<shell>bash</shell>
<network enabled="true">
<allowed>api.example.com</allowed>
<allowed>*.openai.com</allowed>
<denied>blocked.example.com</denied>
</network>
</environment_context>
```
Currently, `codex-network-proxy` depends on `codex-core`, but this
should be the other way around. As a first step, refactor out
`ConfigReloader`, which should make it easier to move
`codex-rs/network-proxy/src/state.rs` to `codex-core` in a subsequent
commit.
Rasmus Rygaard