Two fixes surfaced by the live feed.
modelwatch: some repos report a bogus tiny `safetensors.total` (e.g.
deepreinforce-ai/Ornith-1.0-35B came through as "1M params"), which both
misrendered and — worse — let an oversized model slip past the size cap. Add
`params_from_name` (parse the first <num>B|M token from the repo id, which by
convention is the total size) and `effective_params` (trust safetensors.total
only when it's >= 100M, else fall back to the name). Use it for the size filter
and the summary. New unit tests cover the name parser, the fallback, and that a
70B reporting a 1M total is still rejected as too large. (Forward-looking: the
idempotent ingest won't re-render items already in the feed.)
fetch: feed requests now send browser-shaped `Accept` / `Accept-Language`
headers. reqwest sends no Accept by default, which some anti-bot filters 403.
NB this is hygiene, not a Reddit fix — Reddit fingerprints the TLS/HTTP client
(rustls) and hard-403s it where curl only gets rate-limited (429); headers don't
change the fingerprint. The control path (github releases.atom, etc.) is
unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Make adding RSS sources painless. Instead of requiring the exact feed URL,
a user can paste any page — a blog homepage, a YouTube channel, a subreddit,
a Mastodon profile — and the server resolves the concrete feed it advertises.
Plus OPML bulk-import to migrate a reader export or a YouTube subscriptions
list in one shot. No new SourceKind, no migration; this enriches the existing
worker-polled RSS rail.
New crate `newsfeed-fetch` — the one place that does outbound feed HTTP:
- probe(url): normalise (+ Reddit /.rss rewrite) -> fetch -> if it parses as a
feed, done; else scan the HTML <head> for <link rel=alternate type=rss/atom>,
resolve the relative href, and validate. Covers YouTube/Mastodon/Substack/
WordPress/blogs, which all advertise their feed this way.
- parse_opml(): recurses nested outline folders.
- fetch_entries()/entry mapping moved here from the worker so both the API
(discovery) and worker (polling) share a single HTTP+feed-rs path.
- core: add the FeedProbe port (kept I/O-free; adapter lives in newsfeed-fetch).
- api: AppState carries Arc<dyn FeedProbe>; POST /v1/sources resolves a pasted
homepage to its feed; add POST /v1/sources/discover (preview) and
/v1/sources/import (OPML, deduped by URL, per-feed failures collected).
- worker: delegate to newsfeed_fetch::fetch_entries; drop the sourcing/ module.
- web: Sources page gains paste-URL + "Find feed" preview, OPML file import
with a summary, and a "polled Nm ago" hint per source; new client methods
and regenerated ts-rs bindings.
Verified end-to-end locally: homepage URL -> discovered feed.xml + title;
create stores the resolved URL; OPML import added 1/skipped 1 dup; worker
polled and both items landed in the feed. fmt/clippy/test + web build/lint green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
