Gate new rollout work on shared budget exhaustion (#61582)

## Why

A budget can be exhausted while another thread waits to start work. New requests, tools, and automatic turns must observe that stop without discarding an already-consumed response whose cost exhausted the budget.

## What changed

- Share a work gate across the rollout tree while preserving native and host accounting ownership.
- Recheck the budget before HTTP and WebSocket requests, hook execution, tool dispatch, and compaction. Prevent automatic wakeups and late input from restarting stopped work, leaving pending input queued.
- Record response costs without rejecting the consumed response solely because its bill exhausts the budget, and stop new work when enforced accounting overflows.
- Expose a thread-registration receipt that remains committed after unload.

## Testing

Extend rollout-budget coverage to verify that accepted response text remains in history after exhaustion and that waiting tools observe the shared stop while completed tools retain their results.

GitOrigin-RevId: e583d657c6f29e374d374cdb014bf23fd6c28614
This commit is contained in:
tongzhou wang
2026-11-15 00:03:23 +00:00
committed by copyberry
parent 2e61e8ec4e
commit ce82e38b16
26 changed files with 333 additions and 108 deletions
+3 -4
View File
@@ -54,10 +54,6 @@ impl SamplingRequest {
owner_cancellation: &CancellationToken,
send: impl Future<Output = Result<OpenedModelResponse>>,
) -> Result<SamplingStream> {
let send = async {
session.check_rollout_budget()?;
send.await
};
let sampling_turn = match self.kind {
SamplingKind::Sampling => Some(Arc::clone(&step.turn)),
SamplingKind::ForegroundCompaction | SamplingKind::BackgroundCompaction => None,
@@ -83,6 +79,9 @@ impl SamplingRequest {
agent_id: subagent.as_ref().map(|context| context.agent_id.clone()),
agent_type: subagent.map(|context| context.agent_type),
};
// Hook input preparation may wait. Do not start a new hook pair after a stop;
// the HTTP/WS send independently rechecks after a pre-hook has finished.
session.check_rollout_budget()?;
let mut attempt = post_mode.map(|mode| SamplingAttempt {
owner_cancellation: owner_cancellation.clone(),
post: Some(PostHooks {