Full-stack feature showing programming languages by commit activity
as a stream graph on the dashboard.
Backend:
- migration: repo_languages table (source, repo, language, bytes, color)
- worker: fetch language breakdowns via GitHub GraphQL (batched,
20 repos/request) and Gitea REST API during poll cycles
- API: GET /v1/languages/daily (daily commit counts per language),
GET /v1/languages/repos (all stored repo language data)
- fix timezone bug in daily_counts and language_daily_counts: the
PostgreSQL server timezone (Europe/Sofia, UTC+3) shifted day
boundaries, miscounting events near midnight. Now uses explicit
UTC boundaries in generate_series JOINs.
- use per-source CASE for repo name extraction in language query
to match gitea payload structure (repo.full_name vs repo.name)
- Gitea languages use GitHub colors via COALESCE fallback
Frontend:
- LanguageStreamGraph component: pure SVG stream graph, weekly
buckets, centered baseline, top 8 languages + Other, GitHub
canonical language colors, legend with color dots
- DashPage/ProjectPage: fetch repo languages once via new endpoint
instead of per-repo forge proxy calls (eliminates 200+ GitHub
API calls and 403 rate limit errors)
- removed fetchLanguages forge proxy wrapper (dead code)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The REST /user/repos endpoint only returns repos where the user is
owner, collaborator, or org member. Repos contributed to via PRs
(e.g. polkadot-js/api, zed-industries/zed) were never discovered
and their commits were missing from moments.
Now supplements /user/repos with a GraphQL
repositoriesContributedTo query, which returns all repos the user
has committed to, opened issues/PRs on, or reviewed — with cursor-
based pagination and no result cap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After initial backfill, scan_repo was fetching only page 1 (100 most
recent commits) per repo. If more than 100 commits landed between
7-day polls, older ones in that window were permanently missed.
Now stores the newest commit date in poller_state.last_modified and
passes it as &since= on subsequent polls, with full pagination, so
only genuinely new commits are fetched but none are skipped.
On first poll after deploy, last_modified is NULL so no since filter
is applied — triggering a full re-backfill that catches any
previously missed commits.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cover all data sources (github events/search/repo, gitea with org
discovery, hg revset queries, bugzilla), frontend routes (dash with
contribution graphs, activity timeline with timespan filtering,
project detail with readme/languages, cv), api endpoints including
forge proxy and og image, environment variables, and deployment.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Automated/bulk-commit repos score -1 so they sort last regardless
of recency or volume.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The commit presentation layer only checked repository.full_name,
missing commits ingested by github_repo which store the repo name
in _repo instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The events query's COALESCE for github source was missing _repo,
so per-repo commit events from github_repo had no repo match and
project pages showed 0 activities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add SVG icons for each forge before the repo name on dashboard cards.
Icons sourced from user-provided SVGs in ui/public/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /repos/{owner}/{repo}/commits endpoint doesn't include repo info
in its response. Without _repo in the payload, these commits were
invisible to the projects query. Add _repo to parse_commit and include
it in the COALESCE chain for github source repo extraction.
After deploy, reset github-repo poller state to re-ingest with _repo:
DELETE FROM poller_state WHERE source LIKE 'github-repo%';
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add "in N repositories" to both the year and all-time graph summary
lines. Year graph counts repos with overlapping activity; all-time
graph uses total project count. OG image includes repo count too.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The static asset regex matched .png in /api/v1/og/contributions.png
before the /api/ proxy block, returning 404. Add negative lookahead
to skip /api/ paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add /v1/og/contributions.png endpoint that builds an SVG of the
all-time weekly contribution graph (one row per year) from daily
counts, then rasterizes to PNG via resvg. Served with 1h cache.
Add og:image and twitter:card meta tags to index.html pointing at
the endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restructure the all-time contribution graph from a single row of ~700
circles (sub-pixel when scaled) to one row per year with ~52 weekly
columns, matching the width of the daily graph above. Year labels on
the left.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add AllTimeGraph component showing one circle per week across the full
history (earliest event to today). Uses the /sources endpoint to find
the earliest date, then fetches daily counts and aggregates to weekly.
Clicking a week navigates to /activity/YYYY-MM-DD..YYYY-MM-DD.
Update parseTimespan to handle both date-only (YYYY-MM-DD) and full
ISO datetime strings in range expressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use viewBox + width=100% instead of fixed pixel dimensions so the
SVG scales to match the project card grid below.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add /v1/activity/daily endpoint returning per-day event counts via
generate_series + LEFT JOIN. Frontend renders an SVG contribution
graph with circles colored by quantile-based thresholds. Clicking a
day navigates to /activity/YYYY-MM-DD showing that day's events.
New /activity/:timespan route parses single dates (YYYY-MM-DD) and
ranges (YYYY-MM-DD..YYYY-MM-DD) from the URL to initialize the
activity timeline filter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
atob() produces Latin-1 strings, mangling multi-byte UTF-8 characters
like box-drawing glyphs. Use TextDecoder for correct UTF-8 handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add /v1/forge/{source}/* proxy endpoint to the API server with an
allowlisted set of hosts. Frontend readme and language requests now
go through the proxy instead of hitting forge APIs directly (Gitea
has no CORS headers). Gitea readme fetch tries README.md, readme.md,
and Readme.md casings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use CASE/source instead of COALESCE for repo name extraction — Gitea's
repo.name is the short name while full_name includes the owner prefix.
Fix Gitea README fetch to use /contents/README.md with base64 decoding
instead of the nonexistent /readme endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ProjectPage fetches README (raw markdown) and language breakdown from
GitHub/Gitea REST APIs, rendering the readme as markdown and languages
as a colored proportional bar with labels.
Dashboard cards lazily fetch top 3 languages per repo and display them
inline. Language color map covers common languages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add repo filter param to /v1/events (SQL COALESCE across payload
shapes per source). New /project/:source/* route renders a filtered
activity timeline for a single repo. Dashboard cards link to the
drill-down page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restructure routes: / and /dash show a project overview dashboard,
/activity hosts the existing timeline, /cv remains. Shared Layout
component provides consistent nav header and footer across all routes.
New /v1/projects endpoint aggregates per-repo activity stats (commits,
issues, PRs, date range) from existing event data via SQL. Dashboard
ranks projects by weighted recency + volume score and renders a card
grid.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a new github-repo EventSource that enumerates all repos via
/user/repos and walks each repo's /commits?author= endpoint, which
has no 1000-result cap unlike the Search API. Events use the same
github-commit:{sha} ID scheme as github_search for dedup. Per-repo
poller state enables full backfill on first run, page-1-only on
subsequent polls. Weekly poll interval by default.
Closes #1
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove /dev/null redirects in hg-ingest.sh so errors are visible.
cd to work dir before loop to prevent getcwd failures after rm.
Use $HOME instead of ~ for proper expansion in default values.
Reduce timeline entry title, subtitle, and body font sizes for a
more compact activity feed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrites the hg worker to use json-log?rev=author() which matches the
changeset author (not the pusher), capturing commits landed by sheriffs.
Repos are discovered within configured groups plus individually listed
repos. The worker skips entirely after the first successful backfill.
Adds script/hg-ingest.sh for offline ingestion via local hg clones —
clones one repo at a time, caches extracted changesets to .tsv, inserts
via psql, and sets poller_state when done.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The user activity feed only returns events from the user's own namespace.
This adds org discovery via /api/v1/user/orgs and polls each org's
activity feed, filtering for events by the configured user. Per-org
poller state keys enable independent backfill. Org feed errors are
non-fatal to avoid disrupting the user feed poll.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reproduces the legacy cv (previously at grenade.github.io/cv) as a
react-router /cv route, fetched at runtime from the same gist. moves
the lowercase aesthetic from per-element overrides to a single body-
level rule so a future toggle can flip it from one place. adds a small
site-wide footer noting why no cookie consent banner is shown.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
deploy.sh:
- never rsync into /; stage to /tmp on the remote and install at final
paths via sudo bash heredoc, closing the parent-dir attribute leak
that broke three hosts in the earlier rsync incident
- shell-quote heredoc args via ${var@Q}
- drop -A -X on the remaining (web) rsyncs
- generic worker.secrets loop reads (env-var → pass path) from manifest;
GITEA_TOKEN now flows through automatically
- in-memory bash substitution for templates (secrets never on argv)
- simplify semanage port labelling: --add 2>/dev/null || --modify (the
old grep pre-check matched only the first listed port)
- restorecon back to short flags (Fedora policycoreutils has no long
forms; --recursive errored at deploy time)
- quieter health probe loop: curl diagnostics only on final failure
manifest as source of truth:
- api.config.bind drives BIND_ADDR, firewalld port, semanage label,
health-probe URL
- web.config.{server_name,root,api_upstream} drives nginx render,
rsync targets, restorecon scope
- nginx config renamed to site.conf.tmpl; firewalld svc to
moments-api.xml.tmpl; both rendered at deploy time
- topology flip: api → nikola, worker → frootmig (anjie freed)
new scripts:
- script/teardown.sh: idempotent component teardown, never rsyncs,
shared-state cleanup gated on absence of remaining env files,
--remove-docroot guard against shallow / system paths
- script/db-perms.sh: rewritten — fixes grep/append role mismatch that
appended duplicates on re-run, adds postgres reload, hits primary +
standby in a single invocation
readme: genericized; deployment topology no longer carries real host
or site names.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
nikola and frootmig are flagging power events and drive warnings on
the iLO interface and need drive replacement. Move both moments
components onto anjie.kosherinata.internal until those hosts are
back in service. Update the nginx upstream and the readme topology
table to match; the postgres pg_ident.conf on magrathea now needs
to map anjie's cert CN to both moments_ro and moments_rw (two lines
for the same cert_cn).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These ship in a public repo; topology narration in nginx, systemd,
firewalld, and env templates is gratuitous. Keep the config terse —
directives speak for themselves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The per-site nginx ingress for rob.tn lives on oolon (the host the
external router forwards 443 traffic to), not on nikola. Adjust the
topology so:
- web (static ui + nginx) → oolon.hanzalova.internal
- api binds 0.0.0.0:42424 on nikola.kosherinata.internal so oolon
can reverse-proxy across the WG mesh
- new firewalld service moments-api opens 42424 in the default zone
on nikola
- oolon labels port 42424 http_port_t so httpd_t may name_connect
outbound to it (httpd_can_network_connect was already set)
- nginx ssl_certificate switched to oolon's host cert; upstream
rewritten to nikola.kosherinata.internal:42424
Plaintext between oolon and nikola for now — the WG mesh provides
the encryption layer and the data is already public. Documented
the deferral so a future move to per-hop mTLS is obvious.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires up the prod deployment per architecture-doc conventions:
- api → nikola.kosherinata.internal, loopback bind 127.0.0.1:42424
(less-common port, registered with SELinux as http_port_t).
- worker → frootmig.kosherinata.internal, no listening port.
- web (static ui/dist + nginx server_name rob.tn) → nikola, with
/api/* reverse-proxied to the loopback API.
- db → existing magrathea cluster via mTLS, hostname-baked DATABASE_URL
rendered into /etc/moments/{api,worker}.env at deploy time.
Cert rotation: step-ca renews host certs every 24h; .path units watch
/etc/pki/tls/misc/<host>.pem and trigger systemctl restart of the
relevant service. Both binaries hold cert state in rustls and read
once at startup, so restart is the right reload semantics.
deploy.sh contract matches the architecture doc: positional env arg,
component list (or `all` / `default`), --dry-run support. Renders
config templates from `pass`, rsyncs over ssh+sudo, runs sysusers /
restorecon / semanage / systemctl / nginx -t idempotently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires two historical sources for completeness with the 2019 timeline:
- hg-edge.mozilla.org: scans json-pushes for a configured set of
build/* repos and matches changeset author client-side, since the
pushlog `user=` filter targets the pusher (sheriffs/reviewers in
this case) rather than the author. Daily poll cadence — mozilla
retired hg, no new events expected.
- bugzilla.mozilla.org: queries /rest/bug?creator=<email>. Without
an api key the unauthenticated endpoint only returns public bugs,
which is what the public timeline wants anyway.
Reshape renders "<author> committed <short_node> in <repo>" for hg
and "filed bug #<id> in <product>" for bugzilla, both linking back
to the canonical upstream URL via a stamped `_host` payload field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hits /api/v1/users/{user}/activities/feeds?only-performed-by=true
on the configured gitea host (default git.lair.cafe). Page-1 polling
on a 10-min cadence; first run paginates back through up to 20
pages (1000 items) to seed history.
Gitea has no ETag support on this endpoint, so each tick is a fresh
fetch — relying on idempotent upsert by `gitea:<id>` for dedup.
Reshape covers the gitea op_type set:
commit_repo → "pushed N commits to repo:branch" + commits body,
parsing the JSON-encoded `content` field
push_tag → "tagged X in repo"
create_repo → "created repo"
rename/transfer/delete_branch/delete_tag/star/fork — straightforward
create/close/reopen_issue → "{verb} issue #N in repo: title"
create/close/reopen_pull_request → "{verb} pull request #N"
merge_pull_request → GitMerge icon
comment_issue, comment_pull → markdown body from comment.body
approve/reject_pull_request, publish_release
fallback for anything else (mirror_sync_*, future op_types)
Issue / PR / release events use gitea's pipe-separated
`<index>|<title>` content field; pushes have JSON-encoded content.
Host stamping: parse_gitea_event injects `_host` into each row's
payload so the reshape layer can construct web URLs without a
config dependency. Multi-host gitea would still work as long as
each source instance has its own host configured.
Worker config:
GITEA_HOST default git.lair.cafe
GITEA_USER default grenade
GITEA_TOKEN optional (raises rate limit; required
for private repo activity to surface)
GITEA_POLL_INTERVAL_SECS default 600
Tests: +2 in moments-data (commit_repo parses, private flag
captured), +4 in moments-core (commit_repo with body, create_issue
pipe-content, merge icon swap, fallback) — 27 total green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PushEvent payloads carry `created`, `forced`, `distinct_size`, and
`ref` flags that I wasn't consulting — the result on the timeline
was "pushed 0 commits" for what were actually branch creations
(distinct_size 0 because the commits already existed elsewhere)
and force-pushes that didn't change the resulting tree.
* created=true → "created branch X in repo" + GitBranchCreate icon
* forced + size>0 → "force-pushed N commits to repo:branch"
* forced + size==0 → "force-pushed repo:branch"
* normal + size>0 → "pushed N commits to repo:branch" (unchanged)
* normal + size==0 → "pushed to repo:branch" (no awkward "0 commits")
Also: drop the instagram, facebook, and steel-horse-adventures
links from the UI header — those represent personae the user no
longer wants to surface from rob.tn.
Tests: +3 in presentation/github.rs covering the new push
branches — 21 total green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tower-http's TraceLayer logged the failure status code but not the
underlying error, leaving 500s opaque without curling the response
body. Log the error from the internal() helper so server logs carry
the actual cause (permission denied, query error, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The api connects as the read-only role and was failing on startup
with `permission denied for schema public` because moments_ro lacks
CREATE rights — moments_rw owns the database and runs migrations.
Migrations are now owned exclusively by moments-worker. In deploy
(step 7) systemd ordering ensures the worker runs at least once
before the api unit starts, so the schema is in place by the time
the api accepts traffic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the CRA + React 16 + class-component frontend with the
shape from architecture/generic.md §4: vite + react + swc + ts,
served as static from nginx in prod, vite dev server in dev with
/api proxied to localhost:8080.
Layout:
ui/
package.json, vite.config.ts, tsconfig.{json,app,node}.json
index.html
src/
main.tsx — react root + react-query provider
App.tsx — header, filters, vertical timeline
App.css — dark backdrop, hot-pink links
api/client.ts — TS types mirroring moments-entities;
fetchEvents, fetchSources via /api/v1
components/
Filters.tsx — source toggles, count slider, date range
TimelineEntry.tsx — renders one TimelineItem with body
support for markdown, commits, links
lib/icon.tsx — TimelineIcon → react-bootstrap-icons map
+ colour per icon
Stack: react 19, @tanstack/react-query 5, react-bootstrap 2 (on
bootstrap 5), react-vertical-timeline-component 3, rc-slider 11
(<Slider range /> replaces the removed v8 Range), react-markdown 9.
Dev proxy: /api/* → http://localhost:8080/* (rewrite strips /api).
Backend stays location-agnostic at /v1; ingress prefix is added
by nginx (and the dev proxy) so the same fetch shape works in
both environments.
Verified: tsc -b clean, vite build clean (417 KB js / 245 KB css
gzip 128 / 33), vite dev server serves the index. NOT verified
visually in a browser — that's a `pnpm run dev` away on roosta
once the api is up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
