Coalesce replayed stderr chunks so stored sessions load in linear time (#3456)

Opening a workspace whose agent wrote a lot to stderr could hang the
page. On one Codex workspace, three of seven stored transcripts never
finished streaming and one delivered nothing at all.

The cost is in PlainTextLogProcessor::process, which ends by emitting a
patch that replaces the whole current entry, rebuilt from the entire line
buffer, once per chunk it is handed. That is what makes a live stream
appear to type itself out, and the buffer is only flushed by a time gap,
so it stays small while an agent is running and output trickles in.

Replaying a stored session has no gaps. history_plus_stream hands the
consumer every chunk at once, the buffer never flushes, it grows to the
length of the whole log, and the rebuild runs once per chunk:
O(chunks * total bytes). Codex logs heavily to stderr, which is where it
showed, but every executor that uses stderr_chunked_stream feeds the same
processor.

It also corrupted the view rather than just slowing it. The redundant
giant patches flooded MsgStore's 100MB history ring and evicted the real
conversation: the same log yielded 29 patches before this change and 746
after.

So coalesce chunks that are already available into one before they reach
the consumer. A live stream yields a chunk at a time, so batches are size
1 and streaming behaviour is unchanged. A replay collapses into a single
pass.

Measured on a 4.0MB Codex transcript: 40.3s to 0.105s, and linear from
there. The seven transcripts of the workspace that would not open now
normalize in 0.03s to 0.26s each.

Three tests: the coalescing itself, which fails against the previous
implementation with four chunks where one is expected; ordering with
interleaved stdout; and that the stream still stops at Finished.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Borna Samadi
2026-09-17 12:10:16 +02:00
committed by GitHub
co-authored by Claude Opus 5
parent 735654971b
commit e5327c6af6
+62 -1
View File
@@ -147,10 +147,19 @@ impl MsgStore {
.take_while(|res| future::ready(!matches!(res, Ok(LogMsg::Finished))))
.filter_map(|res| async move {
match res {
Ok(LogMsg::Stderr(s)) => Some(Ok(s)),
Ok(LogMsg::Stderr(s)) => Some(s),
_ => None,
}
})
// Coalesce chunks that are already available into a single chunk.
// Every consumer feeds these to a PlainTextLogProcessor, which
// rebuilds and re-emits the whole buffered entry on each chunk it is
// given. Replaying a stored session makes the entire history ready
// at once, so without this that rebuild runs once per chunk and
// replay costs O(chunks * total bytes). A live stream yields one
// chunk at a time, so batches are size 1 and behaviour is unchanged.
.ready_chunks(1024)
.map(|chunks| Ok(chunks.concat()))
.boxed()
}
@@ -172,3 +181,55 @@ impl MsgStore {
})
}
}
#[cfg(test)]
mod tests {
use super::*;
async fn collect(
stream: futures::stream::BoxStream<'static, Result<String, std::io::Error>>,
) -> Vec<String> {
stream.map(|res| res.expect("stream item")).collect().await
}
#[tokio::test]
async fn stderr_chunked_stream_coalesces_replayed_history() {
// A stored session is replayed by pushing its whole history before
// anyone reads it, so every chunk is ready at once.
let store = Arc::new(MsgStore::new());
for chunk in ["one\n", "two\n", "three\n", "four\n"] {
store.push(LogMsg::Stderr(chunk.to_string()));
}
store.push_finished();
let chunks = collect(store.stderr_chunked_stream()).await;
assert_eq!(chunks, vec!["one\ntwo\nthree\nfour\n".to_string()]);
}
#[tokio::test]
async fn stderr_chunked_stream_keeps_order_and_ignores_stdout() {
let store = Arc::new(MsgStore::new());
store.push(LogMsg::Stderr("a".to_string()));
store.push_stdout("not stderr");
store.push(LogMsg::Stderr("b".to_string()));
store.push(LogMsg::Stderr("c".to_string()));
store.push_finished();
let chunks = collect(store.stderr_chunked_stream()).await;
assert_eq!(chunks.concat(), "abc");
}
#[tokio::test]
async fn stderr_chunked_stream_stops_at_finished() {
let store = Arc::new(MsgStore::new());
store.push(LogMsg::Stderr("before".to_string()));
store.push_finished();
store.push(LogMsg::Stderr("after".to_string()));
let chunks = collect(store.stderr_chunked_stream()).await;
assert_eq!(chunks.concat(), "before");
}
}