Files
codex/codex-rs
Won Park b04a2c2645 Estimate history tokens from content instead of serialized envelopes (#45094)
## Why

Serialized response items include message IDs, metadata, and JSON escaping that inflate token estimates without adding model-visible content.

## What changed

- Estimate each response item from its content, retaining JSON syntax for structured tool payloads.
- Apply image estimates to all image inputs, including non-base64 URLs, and count audio and encrypted content through their modality-specific estimates.
- Exclude plaintext reasoning and bookkeeping-only items from replay accounting.

## Testing

Update unit expectations for text, images, audio, and encrypted content. Add a remote compaction regression test showing that equal-length text produces identical token usage estimates despite different message IDs, metadata, and JSON escaping, while preserving the submitted messages.

GitOrigin-RevId: 1c43d5abbcc1668b4ab413901a2b4b6511ca5892
2026-09-12 18:41:47 +00:00
..