Make the app-server thread unload delay configurable (#42320)

## What changed

- Add the top-level `thread_unload_delay_secs` configuration key for the
  app-server, with a 60-second default instead of the previous 30-minute
  fixed delay.
- Allow zero-delay unloading and reject values that cannot fit in a
  monotonic-clock deadline.
- Reset the unload countdown when a thread gains a subscriber or becomes
  active, and preserve active turns even when the configured delay is zero.
- Document that unloading ephemeral threads discards their in-memory state.

## Testing

- Cover configuration parsing, defaults, overrides, and overflow rejection.
- Cover unsubscribe, resubscribe, delayed and immediate unloading,
  notifications, persisted-thread resume, and WebSocket disconnect behavior.

GitOrigin-RevId: edd46f6b49bbdafbf606bff74378e8f569e95977
This commit is contained in:
Eric Traut
2026-09-02 18:18:54 +00:00
committed by copyberry
parent a526f54b00
commit 5e26f7621c
11 changed files with 238 additions and 13 deletions
+12 -2
View File
@@ -208,7 +208,7 @@ Example with notification opt-out:
- `thread/status/changed` — notification emitted when a loaded thread’s status changes (`threadId` + new `status`).
- `thread/archive` — move a thread’s rollout file into the archived directory and attempt to move any spawned descendant thread rollout files; returns `{}` on success and emits `thread/archived` for each archived thread.
- `thread/delete` — hard-delete an active or archived thread and any spawned descendant threads; returns `{}` on success and emits `thread/deleted` for each deleted thread.
- `thread/unsubscribe` — unsubscribe this connection from thread turn/item events. If this was the last subscriber, the server keeps the thread loaded and unloads it only after it has had no subscribers and no thread activity for 30 minutes, runs `SessionEnd` hooks, then emits `thread/closed`.
- `thread/unsubscribe` — unsubscribe this connection from thread turn/item events. If this was the last subscriber, the server keeps the thread loaded and unloads it only after it has had no subscribers and no thread activity for 60 seconds by default (configured by `thread_unload_delay_secs`), runs `SessionEnd` hooks, then emits `thread/closed`.
- `thread/name/set` — set or update a thread’s user-facing name for either a loaded thread or a persisted rollout; returns `{}` on success and emits `thread/name/updated` to initialized, opted-in clients. Thread names are not required to be unique; name lookups resolve to the most recently updated thread.
- `thread/unarchive` — move an archived rollout file back into the sessions directory; returns the restored `thread` on success and emits `thread/unarchived`.
- `thread/compact/start` — trigger conversation history compaction for a thread; returns `{}` immediately while progress streams through standard turn/item notifications. Parent-owned Multi-Agent V2 subagents reject direct compaction requests.
@@ -638,7 +638,17 @@ Gauges register when first used. Depending on process activity, the snapshot can
- `notSubscribed` when the connection was not subscribed to that thread.
- `notLoaded` when the thread is not loaded.
If this was the last subscriber, the server does not unload the thread immediately. It unloads the thread after the thread has had no subscribers and no thread activity for 30 minutes, runs `SessionEnd` hooks, then emits `thread/closed` and a `thread/status/changed` transition to `notLoaded`.
If this was the last subscriber, the server unloads the thread after the thread has had no subscribers and no thread activity for 60 seconds by default, runs `SessionEnd` hooks, then emits `thread/closed` and a `thread/status/changed` transition to `notLoaded`. A new subscriber or thread activity resets the countdown.
Set the top-level `thread_unload_delay_secs` key in `config.toml` to change this timeout:
```toml
thread_unload_delay_secs = 60
```
The value must be a nonnegative integer in seconds that fits in a monotonic-clock deadline; excessively large values are rejected. Set it to `0` to unload as soon as the thread is inactive and has no subscribers. The app-server reads this setting at startup and applies it to all threads, so changes require a server restart. Set it to `1800` to retain the previous 30-minute timeout.
The timeout also applies to ephemeral threads. Unloading discards their in-memory state, and they cannot subsequently be resumed by ID.
`SessionEnd` also runs before archive, delete, and graceful app-server shutdown. It runs only for root threads, not `ThreadSpawn` children or internal subagents. Hooks are advisory: their output cannot block teardown. The default timeout is one second, configured timeouts are capped at three seconds, `async: true` runs synchronously with a configuration warning, and the hook input always reports `reason: "other"`. `SessionEnd` matchers are evaluated against that reason.
@@ -4,8 +4,6 @@ use codex_app_server_protocol::ThreadQueueChangedNotification;
use codex_extension_api::ThreadIdleCause;
use codex_protocol::config_types::MultiAgentMode;
pub(super) const THREAD_UNLOADING_DELAY: Duration = Duration::from_secs(30 * 60);
#[derive(Clone)]
pub(super) struct ListenerTaskContext {
pub(super) thread_manager: Arc<ThreadManager>,
@@ -16,6 +14,7 @@ pub(super) struct ListenerTaskContext {
pub(super) thread_list_state_permit: Arc<Semaphore>,
pub(super) fallback_model_provider: String,
pub(super) codex_home: PathBuf,
pub(super) thread_unload_delay: Duration,
pub(super) skills_watcher: Arc<SkillsWatcher>,
pub(super) turn_cost_worker: Option<crate::turn_cost_worker::TurnCostWorkerHandle>,
}
@@ -59,7 +58,7 @@ impl UnloadingState {
fn unloading_target(&self) -> Option<Instant> {
match (self.has_subscribers, self.is_active) {
((false, has_no_subscribers_since), (false, is_inactive_since)) => {
Some(std::cmp::max(has_no_subscribers_since, is_inactive_since) + self.delay)
std::cmp::max(has_no_subscribers_since, is_inactive_since).checked_add(self.delay)
}
_ => None,
}
@@ -224,7 +223,7 @@ pub(super) async fn ensure_listener_task_running(
let Some(mut unloading_state) = UnloadingState::new(
&listener_task_context,
conversation_id,
THREAD_UNLOADING_DELAY,
listener_task_context.thread_unload_delay,
)
.await
else {
@@ -1069,6 +1069,7 @@ impl ThreadRequestProcessor {
thread_list_state_permit: self.thread_list_state_permit.clone(),
fallback_model_provider: self.config.model_provider_id.clone(),
codex_home: self.config.codex_home.to_path_buf(),
thread_unload_delay: self.config.thread_unload_delay,
skills_watcher: Arc::clone(&self.skills_watcher),
turn_cost_worker: self.turn_cost_worker.clone(),
}
@@ -1200,6 +1201,7 @@ impl ThreadRequestProcessor {
thread_list_state_permit: self.thread_list_state_permit.clone(),
fallback_model_provider: self.config.model_provider_id.clone(),
codex_home: self.config.codex_home.to_path_buf(),
thread_unload_delay: self.config.thread_unload_delay,
skills_watcher: Arc::clone(&self.skills_watcher),
turn_cost_worker: self.turn_cost_worker.clone(),
};
@@ -1632,6 +1632,7 @@ impl TurnRequestProcessor {
thread_list_state_permit: self.thread_list_state_permit.clone(),
fallback_model_provider: self.config.model_provider_id.clone(),
codex_home: self.config.codex_home.to_path_buf(),
thread_unload_delay: self.config.thread_unload_delay,
skills_watcher: Arc::clone(&self.skills_watcher),
turn_cost_worker: self.turn_cost_worker.clone(),
}
@@ -456,7 +456,12 @@ async fn websocket_disconnect_keeps_last_subscribed_thread_loaded_until_idle_tim
let codex_home = TempDir::new()?;
create_config_toml(codex_home.path(), &server.uri(), "never")?;
let (mut process, bind_addr) = spawn_websocket_server(codex_home.path()).await?;
let (mut process, bind_addr) = spawn_websocket_server_with_args(
codex_home.path(),
"ws://127.0.0.1:0",
&["-c".to_string(), "thread_unload_delay_secs=2".to_string()],
)
.await?;
let mut ws1 = connect_websocket(bind_addr).await?;
send_initialize_request(&mut ws1, /*id*/ 1, "ws_thread_owner").await?;
@@ -465,14 +470,15 @@ async fn websocket_disconnect_keeps_last_subscribed_thread_loaded_until_idle_tim
let thread_id = start_thread(&mut ws1, /*id*/ 2).await?;
assert_loaded_threads(&mut ws1, /*id*/ 3, &[thread_id.as_str()]).await?;
ws1.close(None).await.context("failed to close websocket")?;
drop(ws1);
let mut ws2 = connect_websocket(bind_addr).await?;
send_initialize_request(&mut ws2, /*id*/ 4, "ws_reconnect_client").await?;
read_response_for_id(&mut ws2, /*id*/ 4).await?;
ws1.close(None).await.context("failed to close websocket")?;
drop(ws1);
wait_for_loaded_threads(&mut ws2, /*first_id*/ 5, &[thread_id.as_str()]).await?;
wait_for_loaded_threads(&mut ws2, /*first_id*/ 100, &[]).await?;
process
.kill()
@@ -10,6 +10,7 @@ use codex_app_server_protocol::DynamicToolFunctionSpec;
use codex_app_server_protocol::DynamicToolSpec;
use codex_app_server_protocol::ItemStartedNotification;
use codex_app_server_protocol::ServerRequest;
use codex_app_server_protocol::ThreadClosedNotification;
use codex_app_server_protocol::ThreadItem;
use codex_app_server_protocol::ThreadLoadedListParams;
use codex_app_server_protocol::ThreadLoadedListResponse;
@@ -20,6 +21,7 @@ use codex_app_server_protocol::ThreadResumeResponse;
use codex_app_server_protocol::ThreadStartParams;
use codex_app_server_protocol::ThreadStartResponse;
use codex_app_server_protocol::ThreadStatus;
use codex_app_server_protocol::ThreadStatusChangedNotification;
use codex_app_server_protocol::ThreadUnsubscribeParams;
use codex_app_server_protocol::ThreadUnsubscribeResponse;
use codex_app_server_protocol::ThreadUnsubscribeStatus;
@@ -32,6 +34,7 @@ use core_test_support::streaming_sse::start_streaming_sse_server;
use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::TempDir;
use test_case::test_case;
use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
@@ -41,6 +44,7 @@ async fn thread_unsubscribe_keeps_thread_loaded_until_idle_timeout() -> Result<(
let codex_home = TempDir::new()?;
MockResponsesConfig::new(&server.uri())
.with_sandbox_mode("danger-full-access")
.with_root_config("thread_unload_delay_secs = 2")
.write(codex_home.path())?;
let mut mcp = TestAppServer::builder()
@@ -56,6 +60,20 @@ async fn thread_unsubscribe_keeps_thread_loaded_until_idle_timeout() -> Result<(
.await?;
let thread_id = thread.id;
// Persist a rollout so both warm and cold resumes can find the thread.
timeout(
DEFAULT_READ_TIMEOUT,
mcp.start_turn_and_wait_for_completion(TurnStartParams {
thread_id: thread_id.clone(),
input: vec![V2UserInput::Text {
text: "hello".to_string(),
text_elements: Vec::new(),
}],
..Default::default()
}),
)
.await??;
let unsubscribe: ThreadUnsubscribeResponse = mcp
.request(|request_id| ClientRequest::ThreadUnsubscribe {
request_id,
@@ -81,14 +99,105 @@ async fn thread_unsubscribe_keeps_thread_loaded_until_idle_timeout() -> Result<(
params: ThreadLoadedListParams::default(),
})
.await?;
assert_eq!(data, vec![thread_id]);
assert_eq!(data, vec![thread_id.clone()]);
assert_eq!(next_cursor, None);
let resume: ThreadResumeResponse = mcp
.request(|request_id| ClientRequest::ThreadResume {
request_id,
params: ThreadResumeParams {
thread_id: thread_id.clone(),
..Default::default()
},
})
.await?;
assert_eq!(resume.thread.id, thread_id);
// Resubscribing cancels the pending unload, even after the original deadline.
assert!(
timeout(
std::time::Duration::from_millis(2200),
mcp.read_stream_until_notification_message("thread/closed"),
)
.await
.is_err()
);
let _: ThreadUnsubscribeResponse = mcp
.request(|request_id| ClientRequest::ThreadUnsubscribe {
request_id,
params: ThreadUnsubscribeParams {
thread_id: thread_id.clone(),
},
})
.await?;
// Losing the last subscriber starts a fresh countdown.
assert!(
timeout(
std::time::Duration::from_millis(250),
mcp.read_stream_until_notification_message("thread/closed"),
)
.await
.is_err()
);
let closed: ThreadClosedNotification =
timeout(DEFAULT_READ_TIMEOUT, mcp.read_notification("thread/closed")).await??;
assert_eq!(
closed,
ThreadClosedNotification {
thread_id: thread_id.clone()
}
);
let status = timeout(DEFAULT_READ_TIMEOUT, async {
loop {
let status: ThreadStatusChangedNotification =
mcp.read_notification("thread/status/changed").await?;
if status.status == ThreadStatus::NotLoaded {
return anyhow::Ok(status);
}
}
})
.await??;
assert_eq!(
status,
ThreadStatusChangedNotification {
thread_id: thread_id.clone(),
status: ThreadStatus::NotLoaded,
}
);
let loaded: ThreadLoadedListResponse = mcp
.request(|request_id| ClientRequest::ThreadLoadedList {
request_id,
params: ThreadLoadedListParams::default(),
})
.await?;
assert_eq!(
loaded,
ThreadLoadedListResponse {
data: Vec::new(),
next_cursor: None
}
);
let resume: ThreadResumeResponse = mcp
.request(|request_id| ClientRequest::ThreadResume {
request_id,
params: ThreadResumeParams {
thread_id: thread_id.clone(),
..Default::default()
},
})
.await?;
assert_eq!(resume.thread.id, thread_id);
assert_eq!(resume.thread.status, ThreadStatus::Idle);
Ok(())
}
#[test_case(0; "zero_delay")]
#[test_case(1; "one_second_delay")]
#[tokio::test]
async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
async fn thread_unsubscribe_during_turn_keeps_turn_running(delay_secs: u64) -> Result<()> {
let call_id = "deterministic-wait-call";
let tool_name = "deterministic_wait";
let tool_args = json!({});
@@ -121,6 +230,7 @@ async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
let final_response_completed = completions.remove(0);
MockResponsesConfig::new(server.uri())
.with_sandbox_mode("danger-full-access")
.with_root_config(&format!("thread_unload_delay_secs = {delay_secs}"))
.write(&codex_home)?;
let mut mcp = TestAppServer::builder()
@@ -146,6 +256,16 @@ async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
.await?;
let thread_id = thread.id;
// A subscribed, idle thread stays loaded even with no unload delay.
assert!(
timeout(
std::time::Duration::from_millis(250),
mcp.read_stream_until_notification_message("thread/closed"),
)
.await
.is_err()
);
let _: TurnStartResponse = mcp
.request(|request_id| ClientRequest::TurnStart {
request_id,
@@ -207,7 +327,7 @@ async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
assert_eq!(unsubscribe.status, ThreadUnsubscribeStatus::Unsubscribed);
let closed_while_tool_call_blocked = timeout(
std::time::Duration::from_millis(250),
std::time::Duration::from_millis(1200),
mcp.read_stream_until_notification_message("thread/closed"),
);
let closed_while_tool_call_blocked = closed_while_tool_call_blocked.await;
@@ -228,6 +348,20 @@ async fn thread_unsubscribe_during_turn_keeps_turn_running() -> Result<()> {
)
.await?;
timeout(DEFAULT_READ_TIMEOUT, final_response_completed).await??;
if delay_secs > 0 {
// Once the turn finishes, inactivity starts a fresh countdown.
assert!(
timeout(
std::time::Duration::from_millis(250),
mcp.read_stream_until_notification_message("thread/closed"),
)
.await
.is_err()
);
}
let closed: ThreadClosedNotification =
timeout(DEFAULT_READ_TIMEOUT, mcp.read_notification("thread/closed")).await??;
assert_eq!(closed, ThreadClosedNotification { thread_id });
server.shutdown().await;
Ok(())
+14
View File
@@ -313,6 +313,10 @@ pub struct ConfigToml {
/// Default: `300000` (5 minutes).
pub background_terminal_max_timeout: Option<u64>,
/// Seconds a thread must have no subscribers and no activity before app-server
/// unloads it. Defaults to 60; zero unloads immediately. Changes require a server restart.
pub thread_unload_delay_secs: Option<u64>,
/// Deprecated: ignored.
#[schemars(skip)]
pub js_repl_node_path: Option<AbsolutePathBuf>,
@@ -978,6 +982,16 @@ mod tests {
const WORKSPACE_ID_A: &str = "123e4567-e89b-42d3-a456-426614174000";
const WORKSPACE_ID_B: &str = "123e4567-e89b-42d3-a456-426614174001";
#[test]
fn thread_unload_delay_requires_nonnegative_seconds() {
for value in ["-1", "1.5", "\"60\""] {
let error =
toml::from_str::<ConfigToml>(&format!("thread_unload_delay_secs = {value}"))
.expect_err("idle timeout must be a nonnegative integer");
assert!(error.to_string().contains("thread_unload_delay_secs"));
}
}
#[test]
fn forced_chatgpt_workspace_id_accepts_single_string() {
let config: ConfigToml = toml::from_str(&format!(
+6
View File
@@ -6771,6 +6771,12 @@
"description": "Suppress warnings about unstable (under development) features.",
"type": "boolean"
},
"thread_unload_delay_secs": {
"description": "Seconds a thread must have no subscribers and no activity before app-server unloads it. Defaults to 60; zero unloads immediately. Changes require a server restart.",
"format": "uint64",
"minimum": 0.0,
"type": "integer"
},
"tool_output_token_limit": {
"description": "Token budget applied when storing tool/function outputs in the context manager.",
"format": "uint",
+37
View File
@@ -247,6 +247,43 @@ async fn load_config_applies_optional_mcp_startup_grace() -> std::io::Result<()>
Ok(())
}
#[tokio::test]
async fn load_config_resolves_thread_unload_delay() -> anyhow::Result<()> {
let codex_home = tempdir()?;
for (toml, seconds) in [
("", 60),
("thread_unload_delay_secs = 0", 0),
("thread_unload_delay_secs = 1800", 1800),
] {
let config = Config::load_from_base_config_with_overrides(
toml::from_str(toml)?,
ConfigOverrides::default(),
codex_home.abs(),
)
.await?;
assert_eq!(config.thread_unload_delay, Duration::from_secs(seconds));
}
Ok(())
}
#[tokio::test]
async fn load_config_rejects_thread_unload_delay_overflow() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let error = Config::load_from_base_config_with_overrides(
ConfigToml {
thread_unload_delay_secs: Some(u64::MAX),
..Default::default()
},
ConfigOverrides::default(),
codex_home.abs(),
)
.await
.expect_err("idle timeout must fit in a monotonic deadline");
assert_eq!(error.kind(), std::io::ErrorKind::InvalidInput);
assert_eq!(error.to_string(), "thread_unload_delay_secs is too large");
Ok(())
}
#[tokio::test]
async fn test_toml_parsing() {
let history_with_persistence = r#"
+15
View File
@@ -1037,6 +1037,9 @@ pub struct Config {
/// Default: `300000` (5 minutes).
pub background_terminal_max_timeout: u64,
/// Idle timeout for unsubscribed app-server threads, resolved at server startup.
pub thread_unload_delay: Duration,
/// Compatibility-only settings retained for legacy `ghost_snapshot`
/// config loading.
pub ghost_snapshot: GhostSnapshotConfig,
@@ -3813,6 +3816,17 @@ impl Config {
.background_terminal_max_timeout
.unwrap_or(DEFAULT_MAX_BACKGROUND_TERMINAL_TIMEOUT_MS)
.max(MIN_EMPTY_YIELD_TIME_MS);
let thread_unload_delay =
Duration::from_secs(cfg.thread_unload_delay_secs.unwrap_or(/*default*/ 60));
if std::time::Instant::now()
.checked_add(thread_unload_delay)
.is_none()
{
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"thread_unload_delay_secs is too large",
));
}
let ghost_snapshot = {
let mut config = GhostSnapshotConfig::default();
@@ -4298,6 +4312,7 @@ impl Config {
tool_registry,
code_mode,
background_terminal_max_timeout,
thread_unload_delay,
ghost_snapshot,
multi_agent_v2,
token_budget,
@@ -299,6 +299,7 @@ fn new_config(model: Option<String>, arg0_paths: Arg0DispatchPaths) -> anyhow::R
tool_registry: Default::default(),
code_mode: Default::default(),
background_terminal_max_timeout: 300_000,
thread_unload_delay: std::time::Duration::from_secs(60),
ghost_snapshot: GhostSnapshotConfig::default(),
multi_agent_v2: MultiAgentV2Config::default(),
max_goal_token_budget: None,