mirror of
https://github.com/openai/codex.git
synced 2026-10-30 19:18:49 +00:00
## Summary Adds syntax highlighting to the TUI for fenced code blocks in markdown responses and file diffs, plus a `/theme` command with live preview and persistent theme selection. Uses syntect (~250 grammars, 32 bundled themes, ~1 MB binary cost) — the same engine behind `bat`, `delta`, and `xi-editor`. Includes guardrails for large inputs, graceful fallback to plain text, and SSH-aware clipboard integration for the `/copy` command. <img width="1554" height="1014" alt="image" src="https://github.com/user-attachments/assets/38737a79-8717-4715-b857-94cf1ba59b85" /> <img width="2354" height="1374" alt="image" src="https://github.com/user-attachments/assets/25d30a00-c487-4af8-9cb6-63b0695a4be7" /> ## Problem Code blocks in the TUI (markdown responses and file diffs) render without syntax highlighting, making it hard to scan code at a glance. Users also have no way to pick a color theme that matches their terminal aesthetic. ## Mental model The highlighting system has three layers: 1. **Syntax engine** (`render::highlight`) -- a thin wrapper around syntect + two-face. It owns a process-global `SyntaxSet` (~250 grammars) and a `RwLock<Theme>` that can be swapped at runtime. All public entry points accept `(code, lang)` and return ratatui `Span`/`Line` vectors or `None` when the language is unrecognized or the input exceeds safety guardrails. 2. **Rendering consumers** -- `markdown_render` feeds fenced code blocks through the engine; `diff_render` highlights Add/Delete content as a whole file and Update hunks per-hunk (preserving parser state across hunk lines). Both callers fall back to plain unstyled text when the engine returns `None`. 3. **Theme lifecycle** -- at startup the config's `tui.theme` is resolved to a syntect `Theme` via `set_theme_override`. At runtime the `/theme` picker calls `set_syntax_theme` to swap themes live; on cancel it restores the snapshot taken at open. On confirm it persists `[tui] theme = "..."` to config.toml. ## Non-goals - Inline diff highlighting (word-level change detection within a line). - Semantic / LSP-backed highlighting. - Theme authoring tooling; users supply standard `.tmTheme` files. ## Tradeoffs | Decision | Upside | Downside | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | syntect over tree-sitter / arborium | ~1 MB binary increase for ~250 grammars + 32 themes; battle-tested crate powering widely-used tools (`bat`, `delta`, `xi-editor`). tree-sitter would add ~12 MB for 20-30 languages or ~35 MB for full coverage. | Regex-based; less structurally accurate than tree-sitter for some languages (e.g. language injections like JS-in-HTML). | | Global `RwLock<Theme>` | Enables live `/theme` preview without threading Theme through every call site | Lock contention risk (mitigated: reads vastly outnumber writes, single UI thread) | | Skip background / italic / underline from themes | Terminal BG preserved, avoids ugly rendering on some themes | Themes that rely on these properties lose fidelity | | Guardrails: 512 KB / 10k lines | Prevents pathological stalls on huge diffs or pastes | Very large files render without color | ## Architecture ``` config.toml ─[tui.theme]─> set_theme_override() ─> THEME (RwLock) │ ┌───────────────────────────────────────────┘ │ markdown_render ─── highlight_code_to_lines(code, lang) ─> Vec<Line> diff_render ─── highlight_code_to_styled_spans(code, lang) ─> Option<Vec<Vec<Span>>> │ │ (None ⇒ plain text fallback) │ /theme picker ─── set_syntax_theme(theme) // live preview swap ─── current_syntax_theme() // snapshot for cancel ─── resolve_theme_by_name(name) // lookup by kebab-case ``` Key files: - `tui/src/render/highlight.rs` -- engine, theme management, guardrails - `tui/src/diff_render.rs` -- syntax-aware diff line wrapping - `tui/src/theme_picker.rs` -- `/theme` command builder - `tui/src/bottom_pane/list_selection_view.rs` -- side content panel, callbacks - `core/src/config/types.rs` -- `Tui::theme` field - `core/src/config/edit.rs` -- `syntax_theme_edit()` helper ## Observability - `tracing::warn` when a configured theme name cannot be resolved. - `Config::startup_warnings` surfaces the same message as a TUI banner. - `tracing::error` when persisting theme selection fails. ## Tests - Unit tests in `highlight.rs`: language coverage, fallback behavior, CRLF stripping, style conversion, guardrail enforcement, theme name mapping exhaustiveness. - Unit tests in `diff_render.rs`: snapshot gallery at multiple terminal sizes (80x24, 94x35, 120x40), syntax-highlighted wrapping, large-diff guardrail, rename-to-different-extension highlighting, parser state preservation across hunk lines. - Unit tests in `theme_picker.rs`: preview rendering (wide + narrow), dim overlay on deletions, subtitle truncation, cancel-restore, fallback for unavailable configured theme. - Unit tests in `list_selection_view.rs`: side layout geometry, stacked fallback, buffer clearing, cancel/selection-changed callbacks. - Integration test in `lib.rs`: theme warning uses the final (post-resume) config. ## Cargo Deny: Unmaintained Dependency Exceptions This PR adds two `cargo deny` advisory exceptions for transitive dependencies pulled in by `syntect v5.3.0`: | Advisory | Crate | Status | |----------|-------|--------| | RUSTSEC-2024-0320 | `yaml-rust` | Unmaintained (maintainer unreachable) | | RUSTSEC-2025-0141 | `bincode` | Unmaintained (development ceased; v1.3.3 considered complete) | **Why this is safe in our usage:** - Neither advisory describes a known security vulnerability. Both are "unmaintained" notices only. - `bincode` is used by syntect to deserialize pre-compiled syntax sets. Again, these are **static vendored artifacts** baked into the binary at build time. No user-supplied bincode data is ever deserialized. - Attack surface is zero for both crates; exploitation would require a supply-chain compromise of our own build artifacts. - These exceptions can be removed when syntect migrates to `yaml-rust2` and drops `bincode`, or when alternative crates are available upstream.
1111 lines
31 KiB
Rust
1111 lines
31 KiB
Rust
use pretty_assertions::assert_eq;
|
|
use ratatui::style::Stylize;
|
|
use ratatui::text::Line;
|
|
use ratatui::text::Span;
|
|
use ratatui::text::Text;
|
|
|
|
use crate::markdown_render::render_markdown_text;
|
|
use insta::assert_snapshot;
|
|
|
|
#[test]
|
|
fn empty() {
|
|
assert_eq!(render_markdown_text(""), Text::default());
|
|
}
|
|
|
|
#[test]
|
|
fn paragraph_single() {
|
|
assert_eq!(
|
|
render_markdown_text("Hello, world!"),
|
|
Text::from("Hello, world!")
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn paragraph_soft_break() {
|
|
assert_eq!(
|
|
render_markdown_text("Hello\nWorld"),
|
|
Text::from_iter(["Hello", "World"])
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn paragraph_multiple() {
|
|
assert_eq!(
|
|
render_markdown_text("Paragraph 1\n\nParagraph 2"),
|
|
Text::from_iter(["Paragraph 1", "", "Paragraph 2"])
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn headings() {
|
|
let md = "# Heading 1\n## Heading 2\n### Heading 3\n#### Heading 4\n##### Heading 5\n###### Heading 6\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["# ".bold().underlined(), "Heading 1".bold().underlined()]),
|
|
Line::default(),
|
|
Line::from_iter(["## ".bold(), "Heading 2".bold()]),
|
|
Line::default(),
|
|
Line::from_iter(["### ".bold().italic(), "Heading 3".bold().italic()]),
|
|
Line::default(),
|
|
Line::from_iter(["#### ".italic(), "Heading 4".italic()]),
|
|
Line::default(),
|
|
Line::from_iter(["##### ".italic(), "Heading 5".italic()]),
|
|
Line::default(),
|
|
Line::from_iter(["###### ".italic(), "Heading 6".italic()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_single() {
|
|
let text = render_markdown_text("> Blockquote");
|
|
let expected = Text::from(Line::from_iter(["> ", "Blockquote"]).green());
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_soft_break() {
|
|
// Soft break via lazy continuation should render as a new line in blockquotes.
|
|
let text = render_markdown_text("> This is a blockquote\nwith a soft break\n");
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"> This is a blockquote".to_string(),
|
|
"> with a soft break".to_string()
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_multiple_with_break() {
|
|
let text = render_markdown_text("> Blockquote 1\n\n> Blockquote 2\n");
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["> ", "Blockquote 1"]).green(),
|
|
Line::default(),
|
|
Line::from_iter(["> ", "Blockquote 2"]).green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_three_paragraphs_short_lines() {
|
|
let md = "> one\n>\n> two\n>\n> three\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["> ", "one"]).green(),
|
|
Line::from_iter(["> "]).green(),
|
|
Line::from_iter(["> ", "two"]).green(),
|
|
Line::from_iter(["> "]).green(),
|
|
Line::from_iter(["> ", "three"]).green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_nested_two_levels() {
|
|
let md = "> Level 1\n>> Level 2\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["> ", "Level 1"]).green(),
|
|
Line::from_iter(["> "]).green(),
|
|
Line::from_iter(["> ", "> ", "Level 2"]).green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_with_list_items() {
|
|
let md = "> - item 1\n> - item 2\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["> ", "- ", "item 1"]).green(),
|
|
Line::from_iter(["> ", "- ", "item 2"]).green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_with_ordered_list() {
|
|
let md = "> 1. first\n> 2. second\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(vec![
|
|
Span::from("> "),
|
|
"1. ".light_blue(),
|
|
Span::from("first"),
|
|
])
|
|
.green(),
|
|
Line::from_iter(vec![
|
|
Span::from("> "),
|
|
"2. ".light_blue(),
|
|
Span::from("second"),
|
|
])
|
|
.green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_list_then_nested_blockquote() {
|
|
let md = "> - parent\n> > child\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["> ", "- ", "parent"]).green(),
|
|
Line::from_iter(["> ", " ", "> ", "child"]).green(),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn list_item_with_inline_blockquote_on_same_line() {
|
|
let md = "1. > quoted\n";
|
|
let text = render_markdown_text(md);
|
|
let mut lines = text.lines.iter();
|
|
let first = lines.next().expect("one line");
|
|
// Expect content to include the ordered marker, a space, "> ", and the text
|
|
let s: String = first.spans.iter().map(|sp| sp.content.clone()).collect();
|
|
assert_eq!(s, "1. > quoted");
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_surrounded_by_blank_lines() {
|
|
let md = "foo\n\n> bar\n\nbaz\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"foo".to_string(),
|
|
"".to_string(),
|
|
"> bar".to_string(),
|
|
"".to_string(),
|
|
"baz".to_string(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_in_ordered_list_on_next_line() {
|
|
// Blockquote begins on a new line within an ordered list item; it should
|
|
// render inline on the same marker line.
|
|
let md = "1.\n > quoted\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["1. > quoted".to_string()]);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_in_unordered_list_on_next_line() {
|
|
// Blockquote begins on a new line within an unordered list item; it should
|
|
// render inline on the same marker line.
|
|
let md = "-\n > quoted\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["- > quoted".to_string()]);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_two_paragraphs_inside_ordered_list_has_blank_line() {
|
|
// Two blockquote paragraphs inside a list item should be separated by a blank line.
|
|
let md = "1.\n > para 1\n >\n > para 2\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"1. > para 1".to_string(),
|
|
" > ".to_string(),
|
|
" > para 2".to_string(),
|
|
],
|
|
"expected blockquote content to stay aligned after list marker"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_inside_nested_list() {
|
|
let md = "1. A\n - B\n > inner\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["1. A", " - B", " > inner"]);
|
|
}
|
|
|
|
#[test]
|
|
fn list_item_text_then_blockquote() {
|
|
let md = "1. before\n > quoted\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["1. before", " > quoted"]);
|
|
}
|
|
|
|
#[test]
|
|
fn list_item_blockquote_then_text() {
|
|
let md = "1.\n > quoted\n after\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["1. > quoted", " > after"]);
|
|
}
|
|
|
|
#[test]
|
|
fn list_item_text_blockquote_text() {
|
|
let md = "1. before\n > quoted\n after\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["1. before", " > quoted", " > after"]);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_with_heading_and_paragraph() {
|
|
let md = "> # Heading\n> paragraph text\n";
|
|
let text = render_markdown_text(md);
|
|
// Validate on content shape; styling is handled elsewhere
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"> # Heading".to_string(),
|
|
"> ".to_string(),
|
|
"> paragraph text".to_string(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_heading_inherits_heading_style() {
|
|
let text = render_markdown_text("> # test header\n> in blockquote\n");
|
|
assert_eq!(
|
|
text.lines,
|
|
[
|
|
Line::from_iter([
|
|
"> ".into(),
|
|
"# ".bold().underlined(),
|
|
"test header".bold().underlined(),
|
|
])
|
|
.green(),
|
|
Line::from_iter(["> "]).green(),
|
|
Line::from_iter(["> ", "in blockquote"]).green(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_with_code_block() {
|
|
let md = "> ```\n> code\n> ```\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["> code".to_string()]);
|
|
}
|
|
|
|
#[test]
|
|
fn blockquote_with_multiline_code_block() {
|
|
let md = "> ```\n> first\n> second\n> ```\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["> first", "> second"]);
|
|
}
|
|
|
|
#[test]
|
|
fn nested_blockquote_with_inline_and_fenced_code() {
|
|
/*
|
|
let md = \"> Nested quote with code:\n\
|
|
> > Inner quote and `inline code`\n\
|
|
> >\n\
|
|
> > ```\n\
|
|
> > # fenced code inside a quote\n\
|
|
> > echo \"hello from a quote\"\n\
|
|
> > ```\n";
|
|
*/
|
|
let md = r#"> Nested quote with code:
|
|
> > Inner quote and `inline code`
|
|
> >
|
|
> > ```
|
|
> > # fenced code inside a quote
|
|
> > echo "hello from a quote"
|
|
> > ```
|
|
"#;
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"> Nested quote with code:".to_string(),
|
|
"> ".to_string(),
|
|
"> > Inner quote and inline code".to_string(),
|
|
"> > ".to_string(),
|
|
"> > # fenced code inside a quote".to_string(),
|
|
"> > echo \"hello from a quote\"".to_string(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn list_unordered_single() {
|
|
let text = render_markdown_text("- List item 1\n");
|
|
let expected = Text::from_iter([Line::from_iter(["- ", "List item 1"])]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn list_unordered_multiple() {
|
|
let text = render_markdown_text("- List item 1\n- List item 2\n");
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["- ", "List item 1"]),
|
|
Line::from_iter(["- ", "List item 2"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn list_ordered() {
|
|
let text = render_markdown_text("1. List item 1\n2. List item 2\n");
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "List item 1".into()]),
|
|
Line::from_iter(["2. ".light_blue(), "List item 2".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn list_nested() {
|
|
let text = render_markdown_text("- List item 1\n - Nested list item 1\n");
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["- ", "List item 1"]),
|
|
Line::from_iter([" - ", "Nested list item 1"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn list_ordered_custom_start() {
|
|
let text = render_markdown_text("3. First\n4. Second\n");
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["3. ".light_blue(), "First".into()]),
|
|
Line::from_iter(["4. ".light_blue(), "Second".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn nested_unordered_in_ordered() {
|
|
let md = "1. Outer\n - Inner A\n - Inner B\n2. Next\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "Outer".into()]),
|
|
Line::from_iter([" - ", "Inner A"]),
|
|
Line::from_iter([" - ", "Inner B"]),
|
|
Line::from_iter(["2. ".light_blue(), "Next".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn nested_ordered_in_unordered() {
|
|
let md = "- Outer\n 1. One\n 2. Two\n- Last\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["- ", "Outer"]),
|
|
Line::from_iter([" 1. ".light_blue(), "One".into()]),
|
|
Line::from_iter([" 2. ".light_blue(), "Two".into()]),
|
|
Line::from_iter(["- ", "Last"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn loose_list_item_multiple_paragraphs() {
|
|
let md = "1. First paragraph\n\n Second paragraph of same item\n\n2. Next item\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "First paragraph".into()]),
|
|
Line::default(),
|
|
Line::from_iter([" ", "Second paragraph of same item"]),
|
|
Line::from_iter(["2. ".light_blue(), "Next item".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn tight_item_with_soft_break() {
|
|
let md = "- item line1\n item line2\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["- ", "item line1"]),
|
|
Line::from_iter([" ", "item line2"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn deeply_nested_mixed_three_levels() {
|
|
let md = "1. A\n - B\n 1. C\n2. D\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "A".into()]),
|
|
Line::from_iter([" - ", "B"]),
|
|
Line::from_iter([" 1. ".light_blue(), "C".into()]),
|
|
Line::from_iter(["2. ".light_blue(), "D".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn loose_items_due_to_blank_line_between_items() {
|
|
let md = "1. First\n\n2. Second\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "First".into()]),
|
|
Line::from_iter(["2. ".light_blue(), "Second".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn mixed_tight_then_loose_in_one_list() {
|
|
let md = "1. Tight\n\n2.\n Loose\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "Tight".into()]),
|
|
Line::from_iter(["2. ".light_blue(), "Loose".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn ordered_item_with_indented_continuation_is_tight() {
|
|
let md = "1. Foo\n Bar\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "Foo".into()]),
|
|
Line::from_iter([" ", "Bar"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn inline_code() {
|
|
let text = render_markdown_text("Example of `Inline code`");
|
|
let expected = Line::from_iter(["Example of ".into(), "Inline code".cyan()]).into();
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn strong() {
|
|
assert_eq!(
|
|
render_markdown_text("**Strong**"),
|
|
Text::from(Line::from("Strong".bold()))
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn emphasis() {
|
|
assert_eq!(
|
|
render_markdown_text("*Emphasis*"),
|
|
Text::from(Line::from("Emphasis".italic()))
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn strikethrough() {
|
|
assert_eq!(
|
|
render_markdown_text("~~Strikethrough~~"),
|
|
Text::from(Line::from("Strikethrough".crossed_out()))
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn strong_emphasis() {
|
|
let text = render_markdown_text("**Strong *emphasis***");
|
|
let expected = Text::from(Line::from_iter([
|
|
"Strong ".bold(),
|
|
"emphasis".bold().italic(),
|
|
]));
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn link() {
|
|
let text = render_markdown_text("[Link](https://example.com)");
|
|
let expected = Text::from(Line::from_iter([
|
|
"Link".into(),
|
|
" (".into(),
|
|
"https://example.com".cyan().underlined(),
|
|
")".into(),
|
|
]));
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_known_lang_has_syntax_colors() {
|
|
let text = render_markdown_text("```rust\nfn main() {}\n```\n");
|
|
let content: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
// Content should be preserved; ignore trailing empty line from highlighting.
|
|
let content: Vec<&str> = content
|
|
.iter()
|
|
.map(std::string::String::as_str)
|
|
.filter(|s| !s.is_empty())
|
|
.collect();
|
|
assert_eq!(content, vec!["fn main() {}"]);
|
|
|
|
// At least one span should have non-default style (syntax highlighting).
|
|
let has_colored_span = text
|
|
.lines
|
|
.iter()
|
|
.flat_map(|l| l.spans.iter())
|
|
.any(|sp| sp.style.fg.is_some());
|
|
assert!(has_colored_span, "expected syntax-highlighted spans with color");
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_unknown_lang_plain() {
|
|
let text = render_markdown_text("```xyzlang\nhello world\n```\n");
|
|
let content: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
let content: Vec<&str> = content
|
|
.iter()
|
|
.map(std::string::String::as_str)
|
|
.filter(|s| !s.is_empty())
|
|
.collect();
|
|
assert_eq!(content, vec!["hello world"]);
|
|
|
|
// No syntax coloring for unknown language — all spans have default style.
|
|
let has_colored_span = text
|
|
.lines
|
|
.iter()
|
|
.flat_map(|l| l.spans.iter())
|
|
.any(|sp| sp.style.fg.is_some());
|
|
assert!(!has_colored_span, "expected no syntax coloring for unknown lang");
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_no_lang_plain() {
|
|
let text = render_markdown_text("```\nno lang specified\n```\n");
|
|
let content: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
let content: Vec<&str> = content
|
|
.iter()
|
|
.map(std::string::String::as_str)
|
|
.filter(|s| !s.is_empty())
|
|
.collect();
|
|
assert_eq!(content, vec!["no lang specified"]);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_multiple_lines_root() {
|
|
let md = "```\nfirst\nsecond\n```\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["", "first"]),
|
|
Line::from_iter(["", "second"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_indented() {
|
|
let md = " function greet() {\n console.log(\"Hi\");\n }\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter([" ", "function greet() {"]),
|
|
Line::from_iter([" ", " console.log(\"Hi\");"]),
|
|
Line::from_iter([" ", "}"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn horizontal_rule_renders_em_dashes() {
|
|
let md = "Before\n\n---\n\nAfter\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["Before", "", "———", "", "After"]);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_with_inner_triple_backticks_outer_four() {
|
|
let md = r#"````text
|
|
Here is a code block that shows another fenced block:
|
|
|
|
```md
|
|
# Inside fence
|
|
- bullet
|
|
- `inline code`
|
|
```
|
|
````
|
|
"#;
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
// Filter empty trailing lines for stability; the code block may or may
|
|
// not emit a trailing blank depending on the highlighting path.
|
|
let trimmed: Vec<&str> = {
|
|
let mut v: Vec<&str> = lines.iter().map(std::string::String::as_str).collect();
|
|
while v.last() == Some(&"") {
|
|
v.pop();
|
|
}
|
|
v
|
|
};
|
|
assert_eq!(
|
|
trimmed,
|
|
vec![
|
|
"Here is a code block that shows another fenced block:",
|
|
"",
|
|
"```md",
|
|
"# Inside fence",
|
|
"- bullet",
|
|
"- `inline code`",
|
|
"```",
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_inside_unordered_list_item_is_indented() {
|
|
let md = "- Item\n\n ```\n code line\n ```\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["- Item", "", " code line"]);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_multiple_lines_inside_unordered_list() {
|
|
let md = "- Item\n\n ```\n first\n second\n ```\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["- Item", "", " first", " second"]);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_inside_unordered_list_item_multiple_lines() {
|
|
let md = "- Item\n\n ```\n first\n second\n ```\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(lines, vec!["- Item", "", " first", " second"]);
|
|
}
|
|
|
|
#[test]
|
|
fn markdown_render_complex_snapshot() {
|
|
let md = r#"# H1: Markdown Streaming Test
|
|
Intro paragraph with bold **text**, italic *text*, and inline code `x=1`.
|
|
Combined bold-italic ***both*** and escaped asterisks \*literal\*.
|
|
Auto-link: <https://example.com> and reference link [ref][r1].
|
|
Link with title: [hover me](https://example.com "Example") and mailto <mailto:test@example.com>.
|
|
Image: 
|
|
> Blockquote level 1
|
|
>> Blockquote level 2 with `inline code`
|
|
- Unordered list item 1
|
|
- Nested bullet with italics _inner_
|
|
- Unordered list item 2 with ~~strikethrough~~
|
|
1. Ordered item one
|
|
2. Ordered item two with sublist:
|
|
1) Alt-numbered subitem
|
|
- [ ] Task: unchecked
|
|
- [x] Task: checked with link [home](https://example.org)
|
|
---
|
|
Table below (alignment test):
|
|
| Left | Center | Right |
|
|
|:-----|:------:|------:|
|
|
| a | b | c |
|
|
Inline HTML: <sup>sup</sup> and <sub>sub</sub>.
|
|
HTML block:
|
|
<div style="border:1px solid #ccc;padding:2px">inline block</div>
|
|
Escapes: \_underscores\_, backslash \\, ticks ``code with `backtick` inside``.
|
|
Emoji shortcodes: :sparkles: :tada: (if supported).
|
|
Hard break test (line ends with two spaces)
|
|
Next line should be close to previous.
|
|
Footnote reference here[^1] and another[^longnote].
|
|
Horizontal rule with asterisks:
|
|
***
|
|
Fenced code block (JSON):
|
|
```json
|
|
{ "a": 1, "b": [true, false] }
|
|
```
|
|
Fenced code with tildes and triple backticks inside:
|
|
~~~markdown
|
|
To close ``` you need tildes.
|
|
~~~
|
|
Indented code block:
|
|
for i in range(3): print(i)
|
|
Definition-like list:
|
|
Term
|
|
: Definition with `code`.
|
|
Character entities: & < > " '
|
|
[^1]: This is the first footnote.
|
|
[^longnote]: A longer footnote with a link to [Rust](https://www.rust-lang.org/).
|
|
Escaped pipe in text: a \| b \| c.
|
|
URL with parentheses: [link](https://example.com/path_(with)_parens).
|
|
[r1]: https://example.com/ref "Reference link title"
|
|
"#;
|
|
|
|
let text = render_markdown_text(md);
|
|
// Convert to plain text lines for snapshot (ignore styles)
|
|
let rendered = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect::<Vec<_>>()
|
|
.join("\n");
|
|
|
|
assert_snapshot!(rendered);
|
|
}
|
|
|
|
#[test]
|
|
fn ordered_item_with_code_block_and_nested_bullet() {
|
|
let md = "1. **item 1**\n\n2. **item 2**\n ```\n code\n ```\n - `PROCESS_START` (a `OnceLock<Instant>`) keeps the start time for the entire process.\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|line| {
|
|
line.spans
|
|
.iter()
|
|
.map(|span| span.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"1. item 1".to_string(),
|
|
"2. item 2".to_string(),
|
|
String::new(),
|
|
" code".to_string(),
|
|
" - PROCESS_START (a OnceLock<Instant>) keeps the start time for the entire process.".to_string(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn nested_five_levels_mixed_lists() {
|
|
let md = "1. First\n - Second level\n 1. Third level (ordered)\n - Fourth level (bullet)\n - Fifth level to test indent consistency\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "First".into()]),
|
|
Line::from_iter([" - ", "Second level"]),
|
|
Line::from_iter([" 1. ".light_blue(), "Third level (ordered)".into()]),
|
|
Line::from_iter([" - ", "Fourth level (bullet)"]),
|
|
Line::from_iter([
|
|
" - ",
|
|
"Fifth level to test indent consistency",
|
|
]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn html_inline_is_verbatim() {
|
|
let md = "Hello <span>world</span>!";
|
|
let text = render_markdown_text(md);
|
|
let expected: Text = Line::from_iter(["Hello ", "<span>", "world", "</span>", "!"]).into();
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn html_block_is_verbatim_multiline() {
|
|
let md = "<div>\n <span>hi</span>\n</div>\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["<div>"]),
|
|
Line::from_iter([" <span>hi</span>"]),
|
|
Line::from_iter(["</div>"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn html_in_tight_ordered_item_soft_breaks_with_space() {
|
|
let md = "1. Foo\n <i>Bar</i>\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "Foo".into()]),
|
|
Line::from_iter([" ", "<i>", "Bar", "</i>"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn html_continuation_paragraph_in_unordered_item_indented() {
|
|
let md = "- Item\n\n <em>continued</em>\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["- ", "Item"]),
|
|
Line::default(),
|
|
Line::from_iter([" ", "<em>", "continued", "</em>"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn unordered_item_continuation_paragraph_is_indented() {
|
|
let md = "- Intro\n\n Continuation paragraph line 1\n Continuation paragraph line 2\n";
|
|
let text = render_markdown_text(md);
|
|
let lines: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|line| {
|
|
line.spans
|
|
.iter()
|
|
.map(|span| span.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
assert_eq!(
|
|
lines,
|
|
vec![
|
|
"- Intro".to_string(),
|
|
String::new(),
|
|
" Continuation paragraph line 1".to_string(),
|
|
" Continuation paragraph line 2".to_string(),
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn ordered_item_continuation_paragraph_is_indented() {
|
|
let md = "1. Intro\n\n More details about intro\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "Intro".into()]),
|
|
Line::default(),
|
|
Line::from_iter([" ", "More details about intro"]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn nested_item_continuation_paragraph_is_indented() {
|
|
let md = "1. A\n - B\n\n Continuation for B\n2. C\n";
|
|
let text = render_markdown_text(md);
|
|
let expected = Text::from_iter([
|
|
Line::from_iter(["1. ".light_blue(), "A".into()]),
|
|
Line::from_iter([" - ", "B"]),
|
|
Line::default(),
|
|
Line::from_iter([" ", "Continuation for B"]),
|
|
Line::from_iter(["2. ".light_blue(), "C".into()]),
|
|
]);
|
|
assert_eq!(text, expected);
|
|
}
|
|
|
|
#[test]
|
|
fn code_block_preserves_trailing_blank_lines() {
|
|
// A fenced code block with an intentional trailing blank line must keep it.
|
|
let md = "```rust\nfn main() {}\n\n```\n";
|
|
let text = render_markdown_text(md);
|
|
let content: Vec<String> = text
|
|
.lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<String>()
|
|
})
|
|
.collect();
|
|
// Should have: "fn main() {}" then "" (the blank line).
|
|
// Filter only to content lines (skip leading/trailing empty from rendering).
|
|
assert!(
|
|
content.iter().any(|c| c == "fn main() {}"),
|
|
"expected code line, got {content:?}"
|
|
);
|
|
// The trailing blank line inside the fence should be preserved.
|
|
let code_start = content.iter().position(|c| c == "fn main() {}").unwrap();
|
|
assert!(
|
|
content.len() > code_start + 1,
|
|
"expected a line after 'fn main() {{}}' but content ends: {content:?}"
|
|
);
|
|
assert_eq!(
|
|
content[code_start + 1], "",
|
|
"trailing blank line inside code fence was lost: {content:?}"
|
|
);
|
|
}
|