Files
codex/codex-rs/tui/src/chatwidget.rs
T
Michael Bolin ce2ecbe72f feat: record messages from user in ~/.codex/history.jsonl (#939)
This is a large change to support a "history" feature like you would
expect in a shell like Bash.

History events are recorded in `$CODEX_HOME/history.jsonl`. Because it
is a JSONL file, it is straightforward to append new entries (as opposed
to the TypeScript file that uses `$CODEX_HOME/history.json`, so to be
valid JSON, each new entry entails rewriting the entire file). Because
it is possible for there to be multiple instances of Codex CLI writing
to `history.jsonl` at once, we use advisory file locking when working
with `history.jsonl` in `codex-rs/core/src/message_history.rs`.

Because we believe history is a sufficiently useful feature, we enable
it by default. Though to provide some safety, we set the file
permissions of `history.jsonl` to be `o600` so that other users on the
system cannot read the user's history. We do not yet support a default
list of `SENSITIVE_PATTERNS` as the TypeScript CLI does:


https://github.com/openai/codex/blob/3fdf9df1335ac9501e3fb0e61715359145711e8b/codex-cli/src/utils/storage/command-history.ts#L10-L17

We are going to take a more conservative approach to this list in the
Rust CLI. For example, while `/\b[A-Za-z0-9-_]{20,}\b/` might exclude
sensitive information like API tokens, it would also exclude valuable
information such as references to Git commits.

As noted in the updated documentation, users can opt-out of history by
adding the following to `config.toml`:

```toml
[history]
persistence = "none" 
```

Because `history.jsonl` could, in theory, be quite large, we take a[n
arguably overly pedantic] approach in reading history entries into
memory. Specifically, we start by telling the client the current number
of entries in the history file (`history_entry_count`) as well as the
inode (`history_log_id`) of `history.jsonl` (see the new fields on
`SessionConfiguredEvent`).

The client is responsible for keeping new entries in memory to create a
"local history," but if the user hits up enough times to go "past" the
end of local history, then the client should use the new
`GetHistoryEntryRequest` in the protocol to fetch older entries.
Specifically, it should pass the `history_log_id` it was given
originally and work backwards from `history_entry_count`. (It should
really fetch history in batches rather than one-at-a-time, but that is
something we can improve upon in subsequent PRs.)

The motivation behind this crazy scheme is that it is designed to defend
against:

* The `history.jsonl` being truncated during the session such that the
index into the history is no longer consistent with what had been read
up to that point. We do not yet have logic to enforce a `max_bytes` for
`history.jsonl`, but once we do, we will aspire to implement it in a way
that should result in a new inode for the file on most systems.
* New items from concurrent Codex CLI sessions amending to the history.
Because, in absence of truncation, `history.jsonl` is an append-only
log, so long as the client reads backwards from `history_entry_count`,
it should always get a consistent view of history. (That said, it will
not be able to read _new_ commands from concurrent sessions, but perhaps
we will introduce a `/` command to reload latest history or something
down the road.)

Admittedly, my testing of this feature thus far has been fairly light. I
expect we will find bugs and introduce enhancements/fixes going forward.
2025-05-15 16:26:23 -07:00

389 lines
14 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
use std::path::PathBuf;
use std::sync::Arc;
use codex_core::codex_wrapper::init_codex;
use codex_core::config::Config;
use codex_core::protocol::AgentMessageEvent;
use codex_core::protocol::AgentReasoningEvent;
use codex_core::protocol::ApplyPatchApprovalRequestEvent;
use codex_core::protocol::ErrorEvent;
use codex_core::protocol::Event;
use codex_core::protocol::EventMsg;
use codex_core::protocol::ExecApprovalRequestEvent;
use codex_core::protocol::ExecCommandBeginEvent;
use codex_core::protocol::ExecCommandEndEvent;
use codex_core::protocol::InputItem;
use codex_core::protocol::McpToolCallBeginEvent;
use codex_core::protocol::McpToolCallEndEvent;
use codex_core::protocol::Op;
use codex_core::protocol::PatchApplyBeginEvent;
use crossterm::event::KeyEvent;
use ratatui::buffer::Buffer;
use ratatui::layout::Constraint;
use ratatui::layout::Direction;
use ratatui::layout::Layout;
use ratatui::layout::Rect;
use ratatui::widgets::Widget;
use ratatui::widgets::WidgetRef;
use tokio::sync::mpsc::UnboundedSender;
use tokio::sync::mpsc::unbounded_channel;
use crate::app_event::AppEvent;
use crate::app_event_sender::AppEventSender;
use crate::bottom_pane::BottomPane;
use crate::bottom_pane::BottomPaneParams;
use crate::bottom_pane::InputResult;
use crate::conversation_history_widget::ConversationHistoryWidget;
use crate::history_cell::PatchEventType;
use crate::user_approval_widget::ApprovalRequest;
pub(crate) struct ChatWidget<'a> {
app_event_tx: AppEventSender,
codex_op_tx: UnboundedSender<Op>,
conversation_history: ConversationHistoryWidget,
bottom_pane: BottomPane<'a>,
input_focus: InputFocus,
config: Config,
}
#[derive(Clone, Copy, Eq, PartialEq)]
enum InputFocus {
HistoryPane,
BottomPane,
}
impl ChatWidget<'_> {
pub(crate) fn new(
config: Config,
app_event_tx: AppEventSender,
initial_prompt: Option<String>,
initial_images: Vec<PathBuf>,
) -> Self {
let (codex_op_tx, mut codex_op_rx) = unbounded_channel::<Op>();
let app_event_tx_clone = app_event_tx.clone();
// Create the Codex asynchronously so the UI loads as quickly as possible.
let config_for_agent_loop = config.clone();
tokio::spawn(async move {
let (codex, session_event, _ctrl_c) = match init_codex(config_for_agent_loop).await {
Ok(vals) => vals,
Err(e) => {
// TODO: surface this error to the user.
tracing::error!("failed to initialize codex: {e}");
return;
}
};
// Forward the captured `SessionInitialized` event that was consumed
// inside `init_codex()` so it can be rendered in the UI.
app_event_tx_clone.send(AppEvent::CodexEvent(session_event.clone()));
let codex = Arc::new(codex);
let codex_clone = codex.clone();
tokio::spawn(async move {
while let Some(op) = codex_op_rx.recv().await {
let id = codex_clone.submit(op).await;
if let Err(e) = id {
tracing::error!("failed to submit op: {e}");
}
}
});
while let Ok(event) = codex.next_event().await {
app_event_tx_clone.send(AppEvent::CodexEvent(event));
}
});
let mut chat_widget = Self {
app_event_tx: app_event_tx.clone(),
codex_op_tx,
conversation_history: ConversationHistoryWidget::new(),
bottom_pane: BottomPane::new(BottomPaneParams {
app_event_tx,
has_input_focus: true,
}),
input_focus: InputFocus::BottomPane,
config,
};
if initial_prompt.is_some() || !initial_images.is_empty() {
let text = initial_prompt.unwrap_or_default();
chat_widget.submit_user_message_with_images(text, initial_images);
}
chat_widget
}
pub(crate) fn handle_key_event(&mut self, key_event: KeyEvent) {
// Special-case <Tab>: normally toggles focus between history and bottom panes.
// However, when the slash-command popup is visible we forward the key
// to the bottom pane so it can handle auto-completion.
if matches!(key_event.code, crossterm::event::KeyCode::Tab)
&& !self.bottom_pane.is_command_popup_visible()
{
self.input_focus = match self.input_focus {
InputFocus::HistoryPane => InputFocus::BottomPane,
InputFocus::BottomPane => InputFocus::HistoryPane,
};
self.conversation_history
.set_input_focus(self.input_focus == InputFocus::HistoryPane);
self.bottom_pane
.set_input_focus(self.input_focus == InputFocus::BottomPane);
self.request_redraw();
}
match self.input_focus {
InputFocus::HistoryPane => {
let needs_redraw = self.conversation_history.handle_key_event(key_event);
if needs_redraw {
self.request_redraw();
}
}
InputFocus::BottomPane => match self.bottom_pane.handle_key_event(key_event) {
InputResult::Submitted(text) => {
self.submit_user_message(text);
}
InputResult::None => {}
},
}
}
fn submit_user_message(&mut self, text: String) {
// Forward to codex and update conversation history.
self.submit_user_message_with_images(text, vec![]);
}
fn submit_user_message_with_images(&mut self, text: String, image_paths: Vec<PathBuf>) {
let mut items: Vec<InputItem> = Vec::new();
if !text.is_empty() {
items.push(InputItem::Text { text: text.clone() });
}
for path in image_paths {
items.push(InputItem::LocalImage { path });
}
if items.is_empty() {
return;
}
self.codex_op_tx
.send(Op::UserInput { items })
.unwrap_or_else(|e| {
tracing::error!("failed to send message: {e}");
});
// Persist the text to cross-session message history.
if !text.is_empty() {
self.codex_op_tx
.send(Op::AddToHistory { text: text.clone() })
.unwrap_or_else(|e| {
tracing::error!("failed to send AddHistory op: {e}");
});
}
// Only show text portion in conversation history for now.
if !text.is_empty() {
self.conversation_history.add_user_message(text);
}
self.conversation_history.scroll_to_bottom();
}
pub(crate) fn clear_conversation_history(&mut self) {
self.conversation_history.clear();
self.request_redraw();
}
pub(crate) fn handle_codex_event(&mut self, event: Event) {
let Event { id, msg } = event;
match msg {
EventMsg::SessionConfigured(event) => {
// Record session information at the top of the conversation.
self.conversation_history
.add_session_info(&self.config, event.clone());
// Forward history metadata to the bottom pane so the chat
// composer can navigate through past messages.
self.bottom_pane
.set_history_metadata(event.history_log_id, event.history_entry_count);
self.request_redraw();
}
EventMsg::AgentMessage(AgentMessageEvent { message }) => {
self.conversation_history.add_agent_message(message);
self.request_redraw();
}
EventMsg::AgentReasoning(AgentReasoningEvent { text }) => {
self.conversation_history.add_agent_reasoning(text);
self.request_redraw();
}
EventMsg::TaskStarted => {
self.bottom_pane.set_task_running(true);
self.request_redraw();
}
EventMsg::TaskComplete => {
self.bottom_pane.set_task_running(false);
self.request_redraw();
}
EventMsg::Error(ErrorEvent { message }) => {
self.conversation_history.add_error(message);
self.bottom_pane.set_task_running(false);
}
EventMsg::ExecApprovalRequest(ExecApprovalRequestEvent {
command,
cwd,
reason,
}) => {
let request = ApprovalRequest::Exec {
id,
command,
cwd,
reason,
};
self.bottom_pane.push_approval_request(request);
}
EventMsg::ApplyPatchApprovalRequest(ApplyPatchApprovalRequestEvent {
changes,
reason,
grant_root,
}) => {
// ------------------------------------------------------------------
// Before we even prompt the user for approval we surface the patch
// summary in the main conversation so that the dialog appears in a
// sensible chronological order:
// (1) codex → proposes patch (HistoryCell::PendingPatch)
// (2) UI → asks for approval (BottomPane)
// This mirrors how command execution is shown (command begins →
// approval dialog) and avoids surprising the user with a modal
// prompt before they have seen *what* is being requested.
// ------------------------------------------------------------------
self.conversation_history
.add_patch_event(PatchEventType::ApprovalRequest, changes);
self.conversation_history.scroll_to_bottom();
// Now surface the approval request in the BottomPane as before.
let request = ApprovalRequest::ApplyPatch {
id,
reason,
grant_root,
};
self.bottom_pane.push_approval_request(request);
self.request_redraw();
}
EventMsg::ExecCommandBegin(ExecCommandBeginEvent {
call_id,
command,
cwd: _,
}) => {
self.conversation_history
.add_active_exec_command(call_id, command);
self.request_redraw();
}
EventMsg::PatchApplyBegin(PatchApplyBeginEvent {
call_id: _,
auto_approved,
changes,
}) => {
// Even when a patch is auto‑approved we still display the
// summary so the user can follow along.
self.conversation_history
.add_patch_event(PatchEventType::ApplyBegin { auto_approved }, changes);
if !auto_approved {
self.conversation_history.scroll_to_bottom();
}
self.request_redraw();
}
EventMsg::ExecCommandEnd(ExecCommandEndEvent {
call_id,
exit_code,
stdout,
stderr,
}) => {
self.conversation_history
.record_completed_exec_command(call_id, stdout, stderr, exit_code);
self.request_redraw();
}
EventMsg::McpToolCallBegin(McpToolCallBeginEvent {
call_id,
server,
tool,
arguments,
}) => {
self.conversation_history
.add_active_mcp_tool_call(call_id, server, tool, arguments);
self.request_redraw();
}
EventMsg::McpToolCallEnd(McpToolCallEndEvent {
call_id,
success,
result,
}) => {
self.conversation_history
.record_completed_mcp_tool_call(call_id, success, result);
self.request_redraw();
}
EventMsg::GetHistoryEntryResponse(event) => {
let codex_core::protocol::GetHistoryEntryResponseEvent {
offset,
log_id,
entry,
} = event;
// Inform bottom pane / composer.
self.bottom_pane
.on_history_entry_response(log_id, offset, entry.map(|e| e.text));
}
event => {
self.conversation_history
.add_background_event(format!("{event:?}"));
self.request_redraw();
}
}
}
/// Update the live log preview while a task is running.
pub(crate) fn update_latest_log(&mut self, line: String) {
// Forward only if we are currently showing the status indicator.
self.bottom_pane.update_status_text(line);
}
fn request_redraw(&mut self) {
self.app_event_tx.send(AppEvent::Redraw);
}
pub(crate) fn handle_scroll_delta(&mut self, scroll_delta: i32) {
// If the user is trying to scroll exactly one line, we let them, but
// otherwise we assume they are trying to scroll in larger increments.
let magnified_scroll_delta = if scroll_delta == 1 {
1
} else {
// Play with this: perhaps it should be non-linear?
scroll_delta * 2
};
self.conversation_history.scroll(magnified_scroll_delta);
self.request_redraw();
}
/// Forward an `Op` directly to codex.
pub(crate) fn submit_op(&self, op: Op) {
if let Err(e) = self.codex_op_tx.send(op) {
tracing::error!("failed to submit op: {e}");
}
}
}
impl WidgetRef for &ChatWidget<'_> {
fn render_ref(&self, area: Rect, buf: &mut Buffer) {
let bottom_height = self.bottom_pane.calculate_required_height(&area);
let chunks = Layout::default()
.direction(Direction::Vertical)
.constraints([Constraint::Min(0), Constraint::Length(bottom_height)])
.split(area);
self.conversation_history.render(chunks[0], buf);
(&self.bottom_pane).render(chunks[1], buf);
}
}