feat(sources): feed discovery + OPML import for the RSS rail

Make adding RSS sources painless. Instead of requiring the exact feed URL, a user can paste any page — a blog homepage, a YouTube channel, a subreddit, a Mastodon profile — and the server resolves the concrete feed it advertises. Plus OPML bulk-import to migrate a reader export or a YouTube subscriptions list in one shot. No new SourceKind, no migration; this enriches the existing worker-polled RSS rail. New crate `newsfeed-fetch` — the one place that does outbound feed HTTP: - probe(url): normalise (+ Reddit /.rss rewrite) -> fetch -> if it parses as a feed, done; else scan the HTML <head> for <link rel=alternate type=rss/atom>, resolve the relative href, and validate. Covers YouTube/Mastodon/Substack/ WordPress/blogs, which all advertise their feed this way. - parse_opml(): recurses nested outline folders. - fetch_entries()/entry mapping moved here from the worker so both the API (discovery) and worker (polling) share a single HTTP+feed-rs path. - core: add the FeedProbe port (kept I/O-free; adapter lives in newsfeed-fetch). - api: AppState carries Arc<dyn FeedProbe>; POST /v1/sources resolves a pasted homepage to its feed; add POST /v1/sources/discover (preview) and /v1/sources/import (OPML, deduped by URL, per-feed failures collected). - worker: delegate to newsfeed_fetch::fetch_entries; drop the sourcing/ module. - web: Sources page gains paste-URL + "Find feed" preview, OPML file import with a summary, and a "polled Nm ago" hint per source; new client methods and regenerated ts-rs bindings. Verified end-to-end locally: homepage URL -> discovered feed.xml + title; create stores the resolved URL; OPML import added 1/skipped 1 dup; worker polled and both items landed in the feed. fmt/clippy/test + web build/lint green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
2026-07-08 14:09:21 +03:00
parent ee63939151
commit 91fd03a102
21 changed files with 849 additions and 145 deletions
--- a/crates/newsfeed-entities/src/source.rs
+++ b/crates/newsfeed-entities/src/source.rs
@@ -73,6 +73,45 @@ pub struct NewSource {
    pub weight: Option<f64>,
 }

+/// Request to resolve an arbitrary page URL to a concrete feed (`POST /v1/sources/discover`).
+#[derive(Debug, Clone, Deserialize, TS)]
+#[ts(export)]
+pub struct DiscoverFeedRequest {
+    /// Any page URL: a blog homepage, a YouTube channel/video, a subreddit, a Mastodon
+    /// profile, or an already-direct feed URL.
+    pub url: String,
+}
+
+/// A feed resolved from a page URL: the concrete feed URL to poll, plus its title.
+#[derive(Debug, Clone, Serialize, Deserialize, TS)]
+#[ts(export)]
+pub struct DiscoveredFeed {
+    /// The concrete RSS/Atom URL the worker should poll.
+    pub feed_url: String,
+    /// The feed's own title, when the parsed feed advertises one.
+    pub title: Option<String>,
+}
+
+/// Request to bulk-import RSS sources from an OPML document (`POST /v1/sources/import`).
+#[derive(Debug, Clone, Deserialize, TS)]
+#[ts(export)]
+pub struct OpmlImport {
+    /// The raw OPML XML (e.g. exported from another reader or a subscriptions manager).
+    pub opml: String,
+}
+
+/// Outcome of an OPML import.
+#[derive(Debug, Clone, Default, Serialize, Deserialize, TS)]
+#[ts(export)]
+pub struct OpmlImportResult {
+    /// Feeds newly added as sources.
+    pub added: u32,
+    /// Feeds skipped because a source with that URL already existed (or repeated in the file).
+    pub skipped: u32,
+    /// Feeds that failed to import, each as `"<url>: <reason>"`.
+    pub failed: Vec<String>,
+}
+
 impl NewSource {
    /// Validate a create-source request.
    pub fn validate(&self) -> Result<()> {