mirror of
https://github.com/openai/codex.git
synced 2026-11-19 22:30:42 +00:00
## Why Repeated tool output, such as the same image returned in several results, can consume the trace bundle budget. When output bytes are unavailable, the surrounding result can still carry terminal metadata and conversation links. ## What changed - Store JSON string values of at least 64 KiB, measured after encoding, separately from their tool result or runtime event. Identical encoded values share one record per bundle, while each occurrence retains its position and complete captures reconstruct the original JSON bytes. - Keep separate compression shards open for content and surrounding records. Budget for the serialized event and fall back to a single capped or omitted payload when the split layout or its references exceed the applicable limit. - Track missing values separately from literal `null` so reduction can preserve surviving terminal metadata and keep tool output IDs attached to their original items. ## Testing Add regression tests for shared image and text payloads, byte-for-byte reconstruction, recording completion under tight bundle limits, and reduction with missing output. GitOrigin-RevId: a003197ea998639b6c2ae27f408990c0186406bd
37 lines
1.3 KiB
Rust
37 lines
1.3 KiB
Rust
//! Locate large string values in a serialized tool observation without decoding or copying them.
|
|
|
|
use crate::RawPayloadKind;
|
|
use std::ops::Range;
|
|
|
|
pub(crate) fn large_string_values(body: &[u8]) -> Vec<(Range<usize>, RawPayloadKind)> {
|
|
const MIN_BYTES: usize = 64 << 10;
|
|
let mut parts = Vec::new();
|
|
if body.len() < MIN_BYTES {
|
|
return parts;
|
|
}
|
|
|
|
// The writer supplies valid JSON. A quoted token followed by a colon is an object key.
|
|
// Take slices of the original encoding so escapes, duplicate keys and order survive.
|
|
let mut bytes = body.iter().copied().enumerate();
|
|
while let Some((start, _)) = bytes.find(|(_, byte)| *byte == b'"') {
|
|
while let Some((index, byte)) = bytes.next() {
|
|
match byte {
|
|
b'\\' => {
|
|
bytes.next();
|
|
}
|
|
b'"' => {
|
|
let end = index + 1;
|
|
if end - start >= MIN_BYTES
|
|
&& body[end..].iter().find(|b| !b.is_ascii_whitespace()) != Some(&b':')
|
|
{
|
|
parts.push((start..end, RawPayloadKind::ContentValue));
|
|
}
|
|
break;
|
|
}
|
|
_ => {}
|
|
}
|
|
}
|
|
}
|
|
parts
|
|
}
|