Two fixes surfaced by the live feed.
modelwatch: some repos report a bogus tiny `safetensors.total` (e.g.
deepreinforce-ai/Ornith-1.0-35B came through as "1M params"), which both
misrendered and — worse — let an oversized model slip past the size cap. Add
`params_from_name` (parse the first <num>B|M token from the repo id, which by
convention is the total size) and `effective_params` (trust safetensors.total
only when it's >= 100M, else fall back to the name). Use it for the size filter
and the summary. New unit tests cover the name parser, the fallback, and that a
70B reporting a 1M total is still rejected as too large. (Forward-looking: the
idempotent ingest won't re-render items already in the feed.)
fetch: feed requests now send browser-shaped `Accept` / `Accept-Language`
headers. reqwest sends no Accept by default, which some anti-bot filters 403.
NB this is hygiene, not a Reddit fix — Reddit fingerprints the TLS/HTTP client
(rustls) and hard-403s it where curl only gets rate-limited (429); headers don't
change the fingerprint. The control path (github releases.atom, etc.) is
unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
The scoped /etc/sudoers.d/newsfeed_gitea_ci allowlist only permitted rsync of
the api/worker binaries + their configs and a fixed restorecon command. The
modelwatch deploy adds a newsfeed-modelwatch binary, a modelwatch.toml, and an
extra restorecon path, so `sudo rsync … /usr/local/bin/newsfeed-modelwatch`
was refused ("a password is required") and deploy-api failed (rsync code 12).
Add the two rsync destinations and extend the restorecon rule to match deploy.yml.
As with any new deploy touchpoint, infra-setup.sh must be re-run on the API host
(to push the updated sudoers) before the next deploy can succeed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Add newsfeed-modelwatch: a one-shot, systemd-timer-driven producer that watches
the Hugging Face Hub for open-weight releases and pushes filtered candidates to
the ingest endpoint. Answers "how do I automate the HF firehose into my feed" —
the RSS half already works on the pull rail; this is the JSON/API half.
How it works: each run polls the HF models API (a watchlist of orgs +
trendingScore), applies an admission predicate (license allowlist, total-params
cap, safetensors present, not gated, pipeline_tag), and POSTs the survivors to
POST /v1/ingest/candidates under a bearer token, tagged source="huggingface".
Two properties of the existing ingest rail shape the design:
- Ingest is idempotent on (user, external_id). Using the repo id as external_id
makes the producer STATELESS — no dedup table; re-runs/overlapping timers just
re-submit and the server drops repeats.
- HF `tags` are copied onto the candidate, so the user's per-interest weights do
the ranking. The predicate is only an ADMISSION filter (what's worth surfacing
at all) and stays subordinate to explicit weights, per the house rule.
Layout: newsfeed-modelwatch is a client of the API, not an internal component —
it holds no core/data deps. Pure logic (HF types, predicate, mapping to
CandidateSubmission, param formatting) lives in the lib with unit tests; the bin
does the HTTP polling/posting. CandidateSubmission gains Serialize so the
in-workspace producer can build and post one.
Ops:
- asset/systemd/newsfeed-modelwatch.{service,timer}: oneshot + 3-hourly timer.
The ingest token is a secret, kept in /etc/newsfeed/modelwatch.env
(NEWSFEED_TOKEN, mapped to `token` by figment) so a redeploy of the config
never clobbers it.
- asset/config/modelwatch.toml.tmpl: watchlist + predicate, no secret.
- infra-setup.sh installs the units, creates the env placeholder (never
overwritten), enables the timer, and prints the token step.
- deploy.yml builds + ships the binary and the (secret-free) config each deploy.
Verified: unit tests (gated union, license precedence, admit accept/reject,
mapping); dry-run against live HF; full loop against a local API — minted token,
submitted a real release, confirmed it landed in the feed with the huggingface
source linked and HF tags carried through. fmt/clippy/test all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Make adding RSS sources painless. Instead of requiring the exact feed URL,
a user can paste any page — a blog homepage, a YouTube channel, a subreddit,
a Mastodon profile — and the server resolves the concrete feed it advertises.
Plus OPML bulk-import to migrate a reader export or a YouTube subscriptions
list in one shot. No new SourceKind, no migration; this enriches the existing
worker-polled RSS rail.
New crate `newsfeed-fetch` — the one place that does outbound feed HTTP:
- probe(url): normalise (+ Reddit /.rss rewrite) -> fetch -> if it parses as a
feed, done; else scan the HTML <head> for <link rel=alternate type=rss/atom>,
resolve the relative href, and validate. Covers YouTube/Mastodon/Substack/
WordPress/blogs, which all advertise their feed this way.
- parse_opml(): recurses nested outline folders.
- fetch_entries()/entry mapping moved here from the worker so both the API
(discovery) and worker (polling) share a single HTTP+feed-rs path.
- core: add the FeedProbe port (kept I/O-free; adapter lives in newsfeed-fetch).
- api: AppState carries Arc<dyn FeedProbe>; POST /v1/sources resolves a pasted
homepage to its feed; add POST /v1/sources/discover (preview) and
/v1/sources/import (OPML, deduped by URL, per-feed failures collected).
- worker: delegate to newsfeed_fetch::fetch_entries; drop the sourcing/ module.
- web: Sources page gains paste-URL + "Find feed" preview, OPML file import
with a summary, and a "polled Nm ago" hint per source; new client methods
and regenerated ts-rs bindings.
Verified end-to-end locally: homepage URL -> discovered feed.xml + title;
create stores the resolved URL; OPML import added 1/skipped 1 dup; worker
polled and both items landed in the feed. fmt/clippy/test + web build/lint green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Deployment needs two DNS records this script doesn't manage: a split-horizon
newsfeed.internal -> oolon (the mesh access path), and NO internal override for
the public rob.fyi (an internal record pointing it elsewhere makes mesh clients
land on the wrong vhost/cert — which is exactly what happened: rob.fyi resolved
internally to the office proxy 10.6.0.46 and served admin.zap.pics).
Surface both at the end of every run so the gap is visible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
8081 is the alt-HTTP port — one of the crowded defaults every proxy/dev server
grabs — so it's collision-prone on a shared host. Move to 22672, a registered-
range port in the sparse 20000s band, derived deterministically from the
service name and recorded in architecture port-allocations.md.
Updated everywhere the number appears: api bind config + default, firewalld
service XML, nginx upstream, vite dev proxy, deploy workflow API_PORT,
infra-setup (SELinux semanage label), and docs.
Operator note: re-run script/infra-setup.sh on slartibartfast to relabel the
SELinux port (22672) and ship the updated firewalld XML before the next deploy
restarts the api on the new port.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
infra-setup only handled the internal (newsfeed.internal) cert and vhost and
just printed a generic "set it up yourself" note for the WAN name. Now it also,
on the web host:
- issues the Let's Encrypt cert for rob.fyi idempotently (certbot, Cloudflare
DNS-01, ECDSA, --keep-until-expiring), skipping cleanly if the CF token or
certbot is absent;
- installs the certbot renew deploy-hook (reload nginx);
- installs the public vhost, gated on the cert existing so a missing cert can't
break `nginx -t` for all of nginx (external-tls.md §4).
PUBLIC_FQDN/CERT_EMAIL/CF_CREDS are infra-truth vars at the top; set PUBLIC_FQDN
empty to skip. Cloudflare A record + OPNsense :443 forward remain manual and are
called out in the closing output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Reverts the move to `infra`. Verified on a fedora-43 runner that the ssh
reachability check ran fine (it added the host key and connected) — the
earlier deploy failure was the unprovisioned remote sudo step, not a missing
ssh client. Every runner has ssh: `git` (installed for checkout) pulls
`git-core`, which hard-requires `openssh-clients`. So fedora-43 (ssh + rsync +
curl) is the right, lighter runner for these deploy jobs.
See architecture gitea-runners.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
Run #2 exposed two runner-mismatch bugs (now documented in architecture
gitea-runners.md):
- build-web failed with `corepack: command not found` — the fedora runners
ship pnpm preinstalled via npm, not corepack. Call pnpm directly.
- deploy jobs need an ssh client, which the base fedora images lack (they have
rsync but not openssh-clients). Move deploy-api/deploy-web to `runs-on: infra`.
build-api already passes on `runs-on: rust`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
The first run failed with `cargo: command not found` — `runs-on: fedora-43`
is the generic (node) runner, not the Rust toolchain. Split the build:
Rust on `runs-on: rust`, the SPA on `fedora-43` (matching the fleet's
existing conventions, cf. cichlid/helexa workflows).
Also drop the musl static-cross build: the entire fleet is Fedora 43, so
native binaries built on the `rust` runner run on the targets without glibc
skew (deployment-gitea-actions.md §6 permits building on a runner no newer
than the oldest target). Removes the unverifiable musl-toolchain dependency.
Pin pnpm via packageManager so corepack resolves it deterministically.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
