[codex] exec-server: stream files in chunks (#28354)

## Why

`fs/readFile` buffers the entire file in one response, which makes large
remote reads expensive and prevents callers from applying backpressure.
We need an opt-in streaming path with bounded block sizes while
preserving the existing single-call API for small and sandboxed reads.

## What changed

- Add `ExecServerClient::stream`, returning a named `FileReadStream`
that implements `futures::Stream` and yields immutable 1 MiB byte
blocks.
- Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs.
`fs/readBlock` accepts an explicit offset and length.
- Keep unsandboxed files open between block reads, cap open handles per
connection, and clean them up on EOF, error, stream drop, explicit
close, or connection shutdown.
- Reject platform-sandboxed streaming opens instead of turning the
one-shot sandbox helper into a persistent server. Existing `fs/readFile`
behavior is unchanged.

## Testing

- `just test -p codex-exec-server`
- Integration coverage for 1 MiB chunking, exact block-boundary EOF,
sandbox rejection, and continued reads from the opened file after path
replacement.
- Handle-manager coverage for non-sequential offsets, variable block
lengths, the 128-handle limit, and capacity release after close.
This commit is contained in:
pakrym-oai
2026-06-16 09:50:55 -07:00
committed by GitHub
parent 76135cbe7e
commit a4711b88dd
25 changed files with 1229 additions and 36 deletions
@@ -19,12 +19,18 @@ use crate::protocol::ExecParams;
use crate::protocol::ExecResponse;
use crate::protocol::FsCanonicalizeParams;
use crate::protocol::FsCanonicalizeResponse;
use crate::protocol::FsCloseParams;
use crate::protocol::FsCloseResponse;
use crate::protocol::FsCopyParams;
use crate::protocol::FsCopyResponse;
use crate::protocol::FsCreateDirectoryParams;
use crate::protocol::FsCreateDirectoryResponse;
use crate::protocol::FsGetMetadataParams;
use crate::protocol::FsGetMetadataResponse;
use crate::protocol::FsOpenParams;
use crate::protocol::FsOpenResponse;
use crate::protocol::FsReadBlockParams;
use crate::protocol::FsReadBlockResponse;
use crate::protocol::FsReadDirectoryParams;
use crate::protocol::FsReadDirectoryResponse;
use crate::protocol::FsReadFileParams;
@@ -87,6 +93,7 @@ impl ExecServerHandler {
self.background_task_shutdown.cancel();
self.background_tasks.close();
self.background_tasks.wait().await;
self.file_system.shutdown().await;
if let Some(session) = self.session() {
session.detach().await;
}
@@ -234,6 +241,30 @@ impl ExecServerHandler {
self.file_system.read_file(params).await
}
pub(crate) async fn fs_open(
&self,
params: FsOpenParams,
) -> Result<FsOpenResponse, JSONRPCErrorError> {
self.require_initialized_for("filesystem")?;
self.file_system.open(params).await
}
pub(crate) async fn fs_read_block(
&self,
params: FsReadBlockParams,
) -> Result<FsReadBlockResponse, JSONRPCErrorError> {
self.require_initialized_for("filesystem")?;
self.file_system.read_block(params).await
}
pub(crate) async fn fs_close(
&self,
params: FsCloseParams,
) -> Result<FsCloseResponse, JSONRPCErrorError> {
self.require_initialized_for("filesystem")?;
self.file_system.close(params).await
}
pub(crate) async fn fs_write_file(
&self,
params: FsWriteFileParams,