Support typed content in contextual prompt fragments (#54776)

## Why

Contextual fragments and extension-owned world-state sections currently return strings, limiting them to text content.

## What changed

- Return `ContentItem` from fragment bodies and rendering, preserving classification and attribution. Apply text markers only to `InputText`; pass other content through unchanged.
- Allow extension prompt and world-state contributions to carry typed content, and preserve media in context-window hints.
- Hash non-text world-state fragments by their serialized content and role while retaining the existing text hashing behavior.
- Add shared content-to-tool-output conversion that preserves image references, image detail, and audio.

## Testing

Add coverage for typed world-state hashing and preservation of image content and attribution. Expand code-mode output tests to cover input text, output text, file-backed and inline images, and audio.

GitOrigin-RevId: 2ed00d466a3f169b151202f3eef71e667ea64be1
This commit is contained in:
pakrym-oai
2026-10-17 04:02:21 +00:00
committed by copyberry
parent 3b8f43a06b
commit ce0323f2f2
162 changed files with 1109 additions and 570 deletions
+10 -2
View File
@@ -33,7 +33,12 @@ pub(crate) fn format_inter_agent_completion_message(
AgentStatus::NotFound => "Agent was not found.".to_string(),
AgentStatus::PendingInit | AgentStatus::Running | AgentStatus::Interrupted => return None,
};
Some(InterAgentCompletionMessage::new(task_name, sender, payload).render())
Some(
InterAgentCompletionMessage::new(task_name, sender, payload)
.render()
.into_text()
.unwrap_or_default(),
)
}
pub(crate) fn format_guardian_interruption_message(
@@ -43,7 +48,10 @@ pub(crate) fn format_guardian_interruption_message(
) -> String {
let error = truncate_text(error, TruncationPolicy::Tokens(ERROR_MAX_TOKENS));
let payload = format!("Agent interrupted by Guardian: {error}\n\n{GUARDIAN_NEXT_ACTION}");
InterAgentCompletionMessage::new(task_name, sender, payload).render()
InterAgentCompletionMessage::new(task_name, sender, payload)
.render()
.into_text()
.unwrap_or_default()
}
#[cfg(test)]