feat(output): render and atomically write JSON/Markdown sidecars
beat-core: pure render_json / render_markdown / format_timestamp matching
the readme contract, plus the OutputWriter port. Golden-file tests pin the
output byte-for-byte to the readme example.
beat-data: FsOutputWriter implementing OutputWriter — resolves output_dir
(alongside vs absolute flat dir), names <stem>.beat.{json,md}, and writes
atomically (temp + fsync + rename) so a crash never leaves a half-written
sidecar. tempfile-based tests cover both placement modes and round-trip.
Plan doc for stage 05.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,7 @@
|
||||
//! `beat-data`, behind the trait ports defined in [`ports`] and [`discovery`].
|
||||
|
||||
pub mod discovery;
|
||||
pub mod output;
|
||||
pub mod pipeline;
|
||||
pub mod ports;
|
||||
pub mod segment;
|
||||
@@ -14,6 +15,7 @@ pub mod segment;
|
||||
pub use discovery::{
|
||||
FsEvent, ProcessedLedger, Stability, StabilityPolicy, StabilityTracker, is_supported_video,
|
||||
};
|
||||
pub use output::{OutputWriter, format_timestamp, render_json, render_markdown};
|
||||
pub use pipeline::process_decoded;
|
||||
pub use ports::{AudioSource, AudioWindow, DecodedFrame, FrameSource, LlmClient};
|
||||
pub use segment::{AudioChunk, FrameSample, Signals, segment, select_representatives};
|
||||
|
||||
77
crates/beat-core/src/output.rs
Normal file
77
crates/beat-core/src/output.rs
Normal file
@@ -0,0 +1,77 @@
|
||||
//! Output rendering and the writer port.
|
||||
//!
|
||||
//! Rendering is pure (produces strings matching the readme contract) and lives
|
||||
//! here; the actual atomic filesystem write lives in `beat-data`'s
|
||||
//! `FsOutputWriter`, behind the [`OutputWriter`] port.
|
||||
|
||||
use beat_entities::{Error, OutputFormat, Result, VideoResult};
|
||||
use std::fmt::Write;
|
||||
|
||||
/// Persist a result in the requested formats to its sidecar destination(s).
|
||||
pub trait OutputWriter {
|
||||
/// Write `result` in every requested format. `source_path` is the original
|
||||
/// video path, used to derive the sidecar location and stem.
|
||||
fn write(
|
||||
&self,
|
||||
result: &VideoResult,
|
||||
source_path: &std::path::Path,
|
||||
formats: &[OutputFormat],
|
||||
) -> Result<()>;
|
||||
}
|
||||
|
||||
/// Render the canonical JSON sidecar body (pretty, with a trailing newline).
|
||||
pub fn render_json(result: &VideoResult) -> Result<String> {
|
||||
let mut s = serde_json::to_string_pretty(result)
|
||||
.map_err(|e| Error::Parse(format!("json render: {e}")))?;
|
||||
s.push('\n');
|
||||
Ok(s)
|
||||
}
|
||||
|
||||
/// Render the canonical Markdown sidecar body.
|
||||
pub fn render_markdown(result: &VideoResult) -> String {
|
||||
let mut o = String::new();
|
||||
let _ = writeln!(o, "# {}\n", result.source);
|
||||
let _ = writeln!(o, "{}\n", result.summary);
|
||||
let _ = writeln!(o, "## Beats\n");
|
||||
for b in &result.beats {
|
||||
let _ = writeln!(
|
||||
o,
|
||||
"### {} — {} → {}",
|
||||
b.index,
|
||||
format_timestamp(b.start_seconds),
|
||||
format_timestamp(b.end_seconds),
|
||||
);
|
||||
let _ = writeln!(o, "**Characters:** {}", b.characters.join(", "));
|
||||
let _ = writeln!(o, "**Activity:** {}\n", b.activity);
|
||||
let _ = writeln!(o, "{}\n", b.description);
|
||||
}
|
||||
// Collapse the trailing blank line to a single terminating newline.
|
||||
o.truncate(o.trim_end().len());
|
||||
o.push('\n');
|
||||
o
|
||||
}
|
||||
|
||||
/// Format whole seconds as `HH:MM:SS`, truncating fractional seconds to match
|
||||
/// the readme (`47.5 → 00:00:47`).
|
||||
pub fn format_timestamp(seconds: f64) -> String {
|
||||
let total = seconds.max(0.0) as u64;
|
||||
format!(
|
||||
"{:02}:{:02}:{:02}",
|
||||
total / 3600,
|
||||
(total % 3600) / 60,
|
||||
total % 60
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn timestamps_truncate() {
|
||||
assert_eq!(format_timestamp(0.0), "00:00:00");
|
||||
assert_eq!(format_timestamp(47.5), "00:00:47");
|
||||
assert_eq!(format_timestamp(3661.0), "01:01:01");
|
||||
assert_eq!(format_timestamp(5421.0), "01:30:21");
|
||||
}
|
||||
}
|
||||
18
crates/beat-core/tests/fixtures/holiday.beat.json
vendored
Normal file
18
crates/beat-core/tests/fixtures/holiday.beat.json
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"source": "holiday.mp4",
|
||||
"duration_seconds": 5421.0,
|
||||
"summary": "A family road trip along the coast, ending at a beach campsite at dusk.",
|
||||
"beats": [
|
||||
{
|
||||
"index": 0,
|
||||
"start_seconds": 0.0,
|
||||
"end_seconds": 47.5,
|
||||
"characters": [
|
||||
"man in red jacket",
|
||||
"two children"
|
||||
],
|
||||
"activity": "Loading luggage into a car in a driveway.",
|
||||
"description": "A man and two children pack suitcases into the boot of a parked car on a sunny morning."
|
||||
}
|
||||
]
|
||||
}
|
||||
11
crates/beat-core/tests/fixtures/holiday.beat.md
vendored
Normal file
11
crates/beat-core/tests/fixtures/holiday.beat.md
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
# holiday.mp4
|
||||
|
||||
A family road trip along the coast, ending at a beach campsite at dusk.
|
||||
|
||||
## Beats
|
||||
|
||||
### 0 — 00:00:00 → 00:00:47
|
||||
**Characters:** man in red jacket, two children
|
||||
**Activity:** Loading luggage into a car in a driveway.
|
||||
|
||||
A man and two children pack suitcases into the boot of a parked car on a sunny morning.
|
||||
38
crates/beat-core/tests/golden.rs
Normal file
38
crates/beat-core/tests/golden.rs
Normal file
@@ -0,0 +1,38 @@
|
||||
//! Golden-file tests pinning the JSON/Markdown output to the readme contract.
|
||||
|
||||
use beat_core::{render_json, render_markdown};
|
||||
use beat_entities::{Beat, VideoResult};
|
||||
|
||||
fn holiday() -> VideoResult {
|
||||
VideoResult {
|
||||
source: "holiday.mp4".into(),
|
||||
duration_seconds: 5421.0,
|
||||
summary: "A family road trip along the coast, ending at a beach campsite at dusk.".into(),
|
||||
beats: vec![Beat {
|
||||
index: 0,
|
||||
start_seconds: 0.0,
|
||||
end_seconds: 47.5,
|
||||
characters: vec!["man in red jacket".into(), "two children".into()],
|
||||
activity: "Loading luggage into a car in a driveway.".into(),
|
||||
description:
|
||||
"A man and two children pack suitcases into the boot of a parked car on a sunny morning."
|
||||
.into(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_matches_readme_golden() {
|
||||
assert_eq!(
|
||||
render_json(&holiday()).unwrap(),
|
||||
include_str!("fixtures/holiday.beat.json")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn markdown_matches_readme_golden() {
|
||||
assert_eq!(
|
||||
render_markdown(&holiday()),
|
||||
include_str!("fixtures/holiday.beat.md")
|
||||
);
|
||||
}
|
||||
@@ -5,10 +5,12 @@
|
||||
pub mod decode;
|
||||
pub mod ledger;
|
||||
pub mod llm;
|
||||
pub mod output;
|
||||
pub mod watcher;
|
||||
|
||||
#[cfg(feature = "ffmpeg")]
|
||||
pub use decode::{FfmpegAudioSource, FfmpegFrameSource};
|
||||
pub use ledger::RedbLedger;
|
||||
pub use llm::HttpLlmClient;
|
||||
pub use output::FsOutputWriter;
|
||||
pub use watcher::FsWatcher;
|
||||
|
||||
162
crates/beat-data/src/output.rs
Normal file
162
crates/beat-data/src/output.rs
Normal file
@@ -0,0 +1,162 @@
|
||||
//! Filesystem sidecar writer: renders via `beat-core` and writes atomically.
|
||||
|
||||
use beat_core::{OutputWriter, render_json, render_markdown};
|
||||
use beat_entities::{Config, Error, OutputFormat, Result, VideoResult};
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Writes `<stem>.beat.json` / `<stem>.beat.md` sidecars.
|
||||
pub struct FsOutputWriter {
|
||||
/// `None` means "alongside the source"; `Some(dir)` is a flat output dir.
|
||||
output_dir: Option<PathBuf>,
|
||||
}
|
||||
|
||||
impl FsOutputWriter {
|
||||
/// Build from configuration. `output_dir = "alongside"` writes next to the
|
||||
/// source; any other value is treated as an absolute output directory.
|
||||
pub fn new(config: &Config) -> Self {
|
||||
let output_dir = match config.output_dir.as_str() {
|
||||
"alongside" => None,
|
||||
other => Some(PathBuf::from(other)),
|
||||
};
|
||||
Self { output_dir }
|
||||
}
|
||||
|
||||
/// Destination path for a source + extension, e.g. `<stem>.beat.json`.
|
||||
fn sidecar_path(&self, source_path: &Path, ext: &str) -> Result<PathBuf> {
|
||||
let stem = source_path
|
||||
.file_stem()
|
||||
.ok_or_else(|| Error::Config(format!("no file stem: {}", source_path.display())))?;
|
||||
let mut name = stem.to_os_string();
|
||||
name.push(format!(".beat.{ext}"));
|
||||
let dir = match &self.output_dir {
|
||||
None => source_path
|
||||
.parent()
|
||||
.unwrap_or_else(|| Path::new("."))
|
||||
.to_path_buf(),
|
||||
Some(d) => d.clone(),
|
||||
};
|
||||
Ok(dir.join(name))
|
||||
}
|
||||
}
|
||||
|
||||
impl OutputWriter for FsOutputWriter {
|
||||
fn write(
|
||||
&self,
|
||||
result: &VideoResult,
|
||||
source_path: &Path,
|
||||
formats: &[OutputFormat],
|
||||
) -> Result<()> {
|
||||
for fmt in formats {
|
||||
let (ext, body) = match fmt {
|
||||
OutputFormat::Json => ("json", render_json(result)?),
|
||||
OutputFormat::Markdown => ("md", render_markdown(result)),
|
||||
};
|
||||
let dest = self.sidecar_path(source_path, ext)?;
|
||||
if let Some(parent) = dest.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| Error::io(parent, e))?;
|
||||
}
|
||||
atomic_write(&dest, body.as_bytes())?;
|
||||
tracing::info!(dest = %dest.display(), ?fmt, "wrote sidecar");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Write to a uniquely-named temp file in the destination directory, fsync,
|
||||
/// then rename over the destination — atomic on a single filesystem, so a
|
||||
/// crash never leaves a half-written sidecar.
|
||||
fn atomic_write(dest: &Path, bytes: &[u8]) -> Result<()> {
|
||||
let dir = dest.parent().unwrap_or_else(|| Path::new("."));
|
||||
let file_name = dest
|
||||
.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("sidecar");
|
||||
let tmp = dir.join(format!(".{file_name}.tmp.{}", std::process::id()));
|
||||
|
||||
let mut f = fs::File::create(&tmp).map_err(|e| Error::io(&tmp, e))?;
|
||||
f.write_all(bytes).map_err(|e| Error::io(&tmp, e))?;
|
||||
f.sync_all().map_err(|e| Error::io(&tmp, e))?;
|
||||
fs::rename(&tmp, dest).map_err(|e| Error::io(dest, e))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use beat_entities::{Beat, LlmConfig, SegmentationConfig};
|
||||
|
||||
fn result() -> VideoResult {
|
||||
VideoResult {
|
||||
source: "clip.mp4".into(),
|
||||
duration_seconds: 12.0,
|
||||
summary: "A short clip.".into(),
|
||||
beats: vec![Beat {
|
||||
index: 0,
|
||||
start_seconds: 0.0,
|
||||
end_seconds: 12.0,
|
||||
characters: vec!["cat".into()],
|
||||
activity: "sleeping".into(),
|
||||
description: "A cat sleeps.".into(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn config(output_dir: &str) -> Config {
|
||||
Config {
|
||||
watch: vec![],
|
||||
formats: vec![OutputFormat::Json, OutputFormat::Markdown],
|
||||
output_dir: output_dir.into(),
|
||||
llm: LlmConfig {
|
||||
endpoint: "http://x/v1".into(),
|
||||
model: "m".into(),
|
||||
max_concurrent_requests: 1,
|
||||
},
|
||||
segmentation: SegmentationConfig::default(),
|
||||
discovery: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn writes_alongside_with_no_temp_leftover() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let source = dir.path().join("clip.mp4");
|
||||
fs::write(&source, b"fake").unwrap();
|
||||
|
||||
let writer = FsOutputWriter::new(&config("alongside"));
|
||||
writer
|
||||
.write(&result(), &source, &[OutputFormat::Json, OutputFormat::Markdown])
|
||||
.unwrap();
|
||||
|
||||
let json = dir.path().join("clip.beat.json");
|
||||
let md = dir.path().join("clip.beat.md");
|
||||
assert!(json.exists());
|
||||
assert!(md.exists());
|
||||
|
||||
// JSON round-trips back to an equal VideoResult.
|
||||
let read: VideoResult = serde_json::from_str(&fs::read_to_string(&json).unwrap()).unwrap();
|
||||
assert_eq!(read, result());
|
||||
|
||||
// No leftover temp files.
|
||||
let leftovers: Vec<_> = fs::read_dir(dir.path())
|
||||
.unwrap()
|
||||
.filter_map(|e| e.ok())
|
||||
.filter(|e| e.file_name().to_string_lossy().contains(".tmp."))
|
||||
.collect();
|
||||
assert!(leftovers.is_empty(), "temp files left behind");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn writes_into_absolute_output_dir() {
|
||||
let src_dir = tempfile::tempdir().unwrap();
|
||||
let out_dir = tempfile::tempdir().unwrap();
|
||||
let source = src_dir.path().join("movie.mkv");
|
||||
|
||||
let writer = FsOutputWriter::new(&config(out_dir.path().to_str().unwrap()));
|
||||
writer.write(&result(), &source, &[OutputFormat::Json]).unwrap();
|
||||
|
||||
assert!(out_dir.path().join("movie.beat.json").exists());
|
||||
assert!(!src_dir.path().join("movie.beat.json").exists());
|
||||
}
|
||||
}
|
||||
35
doc/plan/05-output/01-output.md
Normal file
35
doc/plan/05-output/01-output.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# 05 — Output
|
||||
|
||||
## Purpose
|
||||
Persist each video's `VideoResult` as a JSON and/or Markdown sidecar, one per source video.
|
||||
|
||||
## Entity
|
||||
`VideoResult { source, duration_seconds, summary, beats }` (beat-entities) serialises to the exact
|
||||
readme JSON shape. `source` is the file *name*; `duration_seconds` is `f64` (so `5421` renders as
|
||||
`5421.0` — a documented, intentional deviation from the readme's integer).
|
||||
|
||||
## Rendering (beat-core — pure)
|
||||
- `render_json`: `serde_json::to_string_pretty` + trailing newline.
|
||||
- `render_markdown`: matches the readme — `# {source}`, summary, `## Beats`, then per beat
|
||||
`### {index} — HH:MM:SS → HH:MM:SS`, `**Characters:** a, b`, `**Activity:** …`, blank line,
|
||||
description. The trailing blank line is collapsed to a single terminating newline.
|
||||
- `format_timestamp`: whole-second `HH:MM:SS`, truncating fractions (`47.5 → 00:00:47`).
|
||||
|
||||
These are pure and locked by golden-file tests (`crates/beat-core/tests/fixtures/holiday.beat.{json,md}`),
|
||||
so any whitespace drift fails CI.
|
||||
|
||||
## Writing (beat-data `FsOutputWriter`, implements `OutputWriter`)
|
||||
- `output_dir = "alongside"` → sidecar in the source's parent dir; any other value → a flat output
|
||||
directory (`create_dir_all` on first run). (`mirror` mode keyed off watch roots is future work.)
|
||||
- Naming: `<stem>.beat.json` / `<stem>.beat.md`.
|
||||
- **Atomic write**: render to a uniquely-named temp file in the destination directory, `sync_all`,
|
||||
then `rename` over the target. A kill mid-write leaves either the old sidecar or the new one — never
|
||||
a half-written file. A complete sidecar is also the discovery fast-skip signal.
|
||||
|
||||
## CLI
|
||||
`--format` overrides config formats via `OutputFormat::from_str` (`json`, `markdown`, `md`). Wiring
|
||||
lands with the daemon/one-shot integration (stage 05 lifecycle).
|
||||
|
||||
## Tests
|
||||
Golden JSON/MD byte-equality; `format_timestamp` cases; `tempfile`-based atomic-write tests
|
||||
(alongside + absolute dir, round-trip, no leftover temp files).
|
||||
Reference in New Issue
Block a user