feat(blog): prune posts removed or renamed upstream
the blog repo is the source of truth for the full set of posts, but upserts alone never delete: removing a file or changing a slug or filename left the old row serving forever. each poll now reconciles — after upserting the current tree, events under source='blog' whose id is not in the parsed set are deleted via a new EventWriter::prune_events port. nothing is lost: git still has every post, and restoring or fixing a file re-ingests it on the next tip change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -73,7 +73,7 @@ PostgreSQL with three migrations in `crates/moments-data/migrations/`. Two roles
|
||||
|
||||
All under `/v1/`: `healthz`, `events`, `sources`, `projects`, `blog`, `blog/{slug}`, `activity/daily`, `forge/{source}/*`, `og/contributions.png`.
|
||||
|
||||
Blog posts are markdown files with YAML frontmatter (`title`, `slug`, `date`; optional `draft`/`public`) in the `grenade/blog` Gitea repo. The worker's `BlogSource` polls the repo (branch-tip sha as change detection) and upserts posts into `events` with `source='blog'` and `occurred_at` from the frontmatter date, so imported posts keep their original publish dates. Publishing or editing a post = pushing to that repo.
|
||||
Blog posts are markdown files with YAML frontmatter (`title`, `slug`, `date`; optional `draft`/`public`) in the `grenade/blog` Gitea repo. The worker's `BlogSource` polls the repo (branch-tip sha as change detection) and upserts posts into `events` with `source='blog'` and `occurred_at` from the frontmatter date, so imported posts keep their original publish dates. The repo is the source of truth for the full set of posts: publishing, editing, renaming, and deleting are all just pushes — each poll upserts the current tree and prunes `source='blog'` rows that are no longer in it.
|
||||
|
||||
## Deployment
|
||||
|
||||
|
||||
@@ -31,4 +31,8 @@ pub trait EventReader: Send + Sync {
|
||||
pub trait EventWriter: Send + Sync {
|
||||
async fn upsert_events(&self, events: &[Event]) -> Result<usize, StoreError>;
|
||||
async fn upsert_repo_languages(&self, languages: &[RepoLanguage]) -> Result<usize, StoreError>;
|
||||
/// Delete events of `source` whose id is not in `keep_ids`. For sources
|
||||
/// whose upstream is authoritative for the full set (e.g. the blog repo),
|
||||
/// this reconciles deletes and renames that upserts alone never would.
|
||||
async fn prune_events(&self, source: moments_entities::Source, keep_ids: &[String]) -> Result<usize, StoreError>;
|
||||
}
|
||||
|
||||
@@ -171,6 +171,18 @@ impl EventSource for BlogSource {
|
||||
}
|
||||
|
||||
let total = self.writer.upsert_events(&events).await?;
|
||||
|
||||
// The repo is the source of truth for the full set of posts: anything
|
||||
// stored under source='blog' that no longer parses out of the current
|
||||
// tree (deleted file, renamed slug/filename, broken frontmatter) is
|
||||
// pruned. Nothing is lost — git still has it, and fixing or restoring
|
||||
// the file re-ingests it on the next tip change.
|
||||
let keep: Vec<String> = events.iter().map(|e| e.id.clone()).collect();
|
||||
let pruned = self.writer.prune_events(Source::Blog, &keep).await?;
|
||||
if pruned > 0 {
|
||||
tracing::info!(pruned, "blog posts removed upstream; pruned from store");
|
||||
}
|
||||
|
||||
self.state.save(SOURCE_NAME, Some(&tip), None).await?;
|
||||
debug!(ingested = total, tip = %tip, "blog poll complete");
|
||||
Ok(total)
|
||||
|
||||
@@ -479,6 +479,22 @@ impl EventWriter for PgStore {
|
||||
Ok(inserted)
|
||||
}
|
||||
|
||||
async fn prune_events(&self, source: Source, keep_ids: &[String]) -> Result<usize, StoreError> {
|
||||
let n = sqlx::query(
|
||||
r#"
|
||||
DELETE FROM events
|
||||
WHERE source = $1 AND NOT (id = ANY($2))
|
||||
"#,
|
||||
)
|
||||
.bind(source.as_str())
|
||||
.bind(keep_ids)
|
||||
.execute(&self.pool)
|
||||
.await
|
||||
.map_err(map_err)?
|
||||
.rows_affected();
|
||||
Ok(n as usize)
|
||||
}
|
||||
|
||||
async fn upsert_repo_languages(&self, languages: &[RepoLanguage]) -> Result<usize, StoreError> {
|
||||
if languages.is_empty() {
|
||||
return Ok(0);
|
||||
|
||||
Reference in New Issue
Block a user