main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  grenadeandClaude Opus 4.8
|
3db1f586a8
|
feat(modelwatch): push producer for open-weight model releases
Add newsfeed-modelwatch: a one-shot, systemd-timer-driven producer that watches
the Hugging Face Hub for open-weight releases and pushes filtered candidates to
the ingest endpoint. Answers "how do I automate the HF firehose into my feed" —
the RSS half already works on the pull rail; this is the JSON/API half.
How it works: each run polls the HF models API (a watchlist of orgs +
trendingScore), applies an admission predicate (license allowlist, total-params
cap, safetensors present, not gated, pipeline_tag), and POSTs the survivors to
POST /v1/ingest/candidates under a bearer token, tagged source="huggingface".
Two properties of the existing ingest rail shape the design:
- Ingest is idempotent on (user, external_id). Using the repo id as external_id
makes the producer STATELESS — no dedup table; re-runs/overlapping timers just
re-submit and the server drops repeats.
- HF `tags` are copied onto the candidate, so the user's per-interest weights do
the ranking. The predicate is only an ADMISSION filter (what's worth surfacing
at all) and stays subordinate to explicit weights, per the house rule.
Layout: newsfeed-modelwatch is a client of the API, not an internal component —
it holds no core/data deps. Pure logic (HF types, predicate, mapping to
CandidateSubmission, param formatting) lives in the lib with unit tests; the bin
does the HTTP polling/posting. CandidateSubmission gains Serialize so the
in-workspace producer can build and post one.
Ops:
- asset/systemd/newsfeed-modelwatch.{service,timer}: oneshot + 3-hourly timer.
The ingest token is a secret, kept in /etc/newsfeed/modelwatch.env
(NEWSFEED_TOKEN, mapped to `token` by figment) so a redeploy of the config
never clobbers it.
- asset/config/modelwatch.toml.tmpl: watchlist + predicate, no secret.
- infra-setup.sh installs the units, creates the env placeholder (never
overwritten), enables the timer, and prints the token step.
- deploy.yml builds + ships the binary and the (secret-free) config each deploy.
Verified: unit tests (gated union, license precedence, admit accept/reject,
mapping); dry-run against live HF; full loop against a local API — minted token,
submitted a real release, confirmed it landed in the feed with the huggingface
source linked and HF tags carried through. fmt/clippy/test all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
|
||
|
grenadeandClaude Opus 4.8
|
91fd03a102
|
feat(sources): feed discovery + OPML import for the RSS rail
Make adding RSS sources painless. Instead of requiring the exact feed URL, a user can paste any page — a blog homepage, a YouTube channel, a subreddit, a Mastodon profile — and the server resolves the concrete feed it advertises. Plus OPML bulk-import to migrate a reader export or a YouTube subscriptions list in one shot. No new SourceKind, no migration; this enriches the existing worker-polled RSS rail. New crate `newsfeed-fetch` — the one place that does outbound feed HTTP: - probe(url): normalise (+ Reddit /.rss rewrite) -> fetch -> if it parses as a feed, done; else scan the HTML <head> for <link rel=alternate type=rss/atom>, resolve the relative href, and validate. Covers YouTube/Mastodon/Substack/ WordPress/blogs, which all advertise their feed this way. - parse_opml(): recurses nested outline folders. - fetch_entries()/entry mapping moved here from the worker so both the API (discovery) and worker (polling) share a single HTTP+feed-rs path. - core: add the FeedProbe port (kept I/O-free; adapter lives in newsfeed-fetch). - api: AppState carries Arc<dyn FeedProbe>; POST /v1/sources resolves a pasted homepage to its feed; add POST /v1/sources/discover (preview) and /v1/sources/import (OPML, deduped by URL, per-feed failures collected). - worker: delegate to newsfeed_fetch::fetch_entries; drop the sourcing/ module. - web: Sources page gains paste-URL + "Find feed" preview, OPML file import with a summary, and a "polled Nm ago" hint per source; new client methods and regenerated ts-rs bindings. Verified end-to-end locally: homepage URL -> discovered feed.xml + title; create stores the resolved URL; OPML import added 1/skipped 1 dup; worker polled and both items landed in the feed. fmt/clippy/test + web build/lint green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP |
||
|
grenadeandClaude Opus 4.8
|
81fe5f7d4d
|
fix: move newsfeed-api off 8081 to 22672 (conflict-unlikely port)
8081 is the alt-HTTP port — one of the crowded defaults every proxy/dev server grabs — so it's collision-prone on a shared host. Move to 22672, a registered- range port in the sparse 20000s band, derived deterministically from the service name and recorded in architecture port-allocations.md. Updated everywhere the number appears: api bind config + default, firewalld service XML, nginx upstream, vite dev proxy, deploy workflow API_PORT, infra-setup (SELinux semanage label), and docs. Operator note: re-run script/infra-setup.sh on slartibartfast to relabel the SELinux port (22672) and ship the updated firewalld XML before the next deploy restarts the api on the new port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP |
||
|
grenadeandClaude Opus 4.8
|
a8c6f57ad0
|
ci(deploy): use correct runner labels and build native binaries
The first run failed with `cargo: command not found` — `runs-on: fedora-43` is the generic (node) runner, not the Rust toolchain. Split the build: Rust on `runs-on: rust`, the SPA on `fedora-43` (matching the fleet's existing conventions, cf. cichlid/helexa workflows). Also drop the musl static-cross build: the entire fleet is Fedora 43, so native binaries built on the `rust` runner run on the targets without glibc skew (deployment-gitea-actions.md §6 permits building on a runner no newer than the oldest target). Removes the unverifiable musl-toolchain dependency. Pin pnpm via packageManager so corepack resolves it deterministically. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP |
||
|
grenadeandClaude Opus 4.8
|
5ce52fff4d
|
feat: scaffold newsfeed — user-controlled news feed
A self-hosted feed where the user owns the ranking: per-source and signed per-interest weights, decayed by recency, via a transparent deterministic scorer. Content is sourced algorithmically (worker RSS/Atom polling) and agentically (per-user API tokens POSTing candidates to the ingest endpoint). Single-user today, multi-user by construction (every row keyed on user_id). Rust cargo workspace + Vite/React/SWC/TS SPA: - newsfeed-entities: DTOs (ts-rs bindings -> web/src/api/bindings) - newsfeed-core: ranking, auth primitives, ingest, data-access ports - newsfeed-data: SQLite adapters (sqlx, runtime queries) - newsfeed-api: axum REST/JSON daemon - newsfeed-worker: RSS polling + rescoring loop - web: responsive, mobile-first SPA (React Query, generated types) Deploy (Gitea Actions, build static musl + SPA, rsync as gitea_ci): api+worker -> slartibartfast, SPA -> oolon (nginx serves + proxies /v1). Deliberate deviations from house conventions (documented in CLAUDE.md/readme): - SQLite instead of Postgres; api+worker co-locate sharing one DB file. - Runtime sqlx queries instead of query! macros (SQLite dynamic typing; keeps CI database-free). Verified end-to-end: auth, token ingest, interest-weighted ranking, signals, pagination (curl + browser); cargo fmt/clippy -D/test and pnpm build/lint pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP |