grenade/moments
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: grenade:f30f949895bebebab553052a4c51a0847c7274d0
Branches Tags
grenade:main
...
compare: grenade:main
Branches Tags
grenade:main

63 Commits
f30f949895 ... main

Author SHA1 Message Date
rob thijssen
2821548e6e feat(ui): add avg-by-hour panel to dashboard stats 
Complements the existing avg-by-weekday chart with its orthogonal
partner: which hour of the day the user typically commits. The api
buckets events by EXTRACT(hour FROM occurred_at AT TIME ZONE $tz) so
the chart matches the clock the user sees rather than UTC; the UI
passes the browser's resolved IANA timezone. Renders as 24 mini-bars
below the weekday chart with labels every 4 hours and per-bar
tooltips showing the average events/day at that hour.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 16:34:17 +03:00
rob thijssen
72eeb547af chore(deploy): self-heal /tmp perms before staging 
frootmig periodically has its /tmp reset from the standard sticky-
world-writable 1777 to root-owned 0755 (cause not yet pinned down),
which breaks the unprivileged rsync of the deploy stage dir and
surfaces as a cryptic "Permission denied" plus a follow-on install
failure. Stat /tmp before each rsync and, if the mode is off, sudo
chmod it back to 1777 — visible in the deploy log so it's obvious
which host keeps drifting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 07:59:06 +03:00
rob thijssen
86411bb88e fix(worker): dedup gitea events from overlapping user and org feeds 
Gitea writes one Action row per interested user-context. A push to an
org repo by user U produces two rows — one with user_id=U, one with
user_id=org — differing only in `id` and `user_id`. Polling both the
user feed and org feeds (which we do, and need to, since neither alone
catches every cross-namespace event) surfaced both rows; the
`gitea:{action_row_id}` id gave them distinct ids, so the upsert dedup
never fired and ~38% of events on org-repo project pages rendered
twice. Switch to a content-derived id keyed on (op_type, act_user_id,
repo_id, ref_name, comment_id, created) so the two rows collide on
upsert, and add a migration that re-keys existing rows to the same
formula while collapsing the duplicates already in the table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 07:53:43 +03:00
rob thijssen
acb061baca chore(deploy): build rust binaries in a podman container 
Workstation runs Fedora 44 (glibc 2.43); servers are still on F42 and
F43. A native release build produces ELFs the older glibc can't load
(GLIBC_2.43 not found), and the api/worker units fail-loop on every
deploy. Build inside docker.io/library/rust:1-bookworm (glibc 2.36)
so the artifacts are forward-compatible with every Fedora target.
Output goes to target/deploy/ to keep separate from native dev
builds, and the cargo registry/git index are cached in named podman
volumes so subsequent builds are incremental. podman is a hard
requirement; no docker fallback.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 16:05:13 +03:00
rob thijssen
8a7177a54a feat(ui): render GFM and embedded HTML in project READMEs 
ReactMarkdown was running with no plugins, so README headers full of
raw <div align=center>, tables, <details>/<summary>, and other GFM
markup rendered as escaped text. Wire in remark-gfm for tables and
GFM features, rehype-raw for embedded HTML, and rehype-sanitize with
an extended schema that permits README-typical tags and attributes
(align, target, width/height, picture/source, etc.) while still
blocking script/iframe/object — READMEs come from external repos so
they need adversarial-input handling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 16:05:05 +03:00
rob thijssen
818a535903 feat(worker): capture commits on non-default branches and forks 
The ingestion paths each had a gap that let non-default-branch work
slip through: /search/commits silently excludes forks, the per-repo
REST commit scan only walked the default branch, and the user events
feed ages out after 90 days. Catch them by enumerating branches per
repo and scanning each (with per-branch state cursors so a brand-new
branch isn't cut off by the default branch's cursor), pre-filtering
branches via a GraphQL HEAD-author check so big upstream forks like
azure-docs don't trigger hundreds of wasted REST calls, treating
GitHub's HTTP 500 on author-filtered empty branches as "no commits"
rather than a server error, and adding fork:true to the search query.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 16:04:58 +03:00
rob thijssen
9a8c0955b5 chore: phrasing 2026-05-12 13:20:11 +03:00
rob thijssen
25eab2d795 feat: add robots.txt allowing all crawlers including social bots 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:49:55 +03:00
rob thijssen
2130032d46 chore: update Cargo.lock for fontdb dependency 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:35:08 +03:00
rob thijssen
92a66422ab feat(ui): add meta description, og:locale, and og:site_name 
Adds the standard HTML meta description (for SEO), og:locale, and
og:site_name tags flagged by Open Graph validators.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:32:33 +03:00
rob thijssen
94b6fbe42d feat(ui): add og:logo meta tag pointing to 512px icon 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:29:40 +03:00
rob thijssen
048646a7c1 feat(ui): add og:url meta tag for canonical URL 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:27:47 +03:00
rob thijssen
1f2fea3427 fix: load system fonts for OG image text rendering 
usvg's default Options creates an empty fontdb, so no fonts are found
for text rendering regardless of what's installed. Load system fonts
into a fontdb::Database and set the default font family to Noto Sans.

Also picks up a formatting change to index.html from a linter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:17:17 +03:00
rob thijssen
d539892b70 fix: scale OG contribution graph to fill 1200x630 canvas 
Compute cell size from available width so the graph fills the canvas
instead of rendering at a fixed small size. Scale year labels
proportionally. Position headline and subtitle at the top with the
graph centered in the remaining space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:11:05 +03:00
rob thijssen
a57682e610 feat: improve OG image and meta tags for social sharing 
- Resize OG image from ~676x216 to 1200x630 (recommended size)
- Add "rob thijssen" headline text overlay to the OG image
- Center the contribution graph within the canvas
- Expand og:title to 55 chars and og:description to 148 chars
  to meet social platform optimal lengths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:08:29 +03:00
rob thijssen
22c80fd7af feat(ui): transpose weekday averages to vertical bar chart 
Show days on the X axis and volume on the Y axis, replacing the
horizontal bar layout with vertical bars for a more natural
time-series reading direction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 16:02:30 +03:00
rob thijssen
8b5656ef26 fix: specify sans-serif font in OG image SVG text elements 
The SVG text elements had no font-family, causing usvg to default to
Times New Roman which isn't installed on the server. Specifying
sans-serif uses the system default and silences the warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:56:24 +03:00
rob thijssen
dd1de38b2f feat(ui): show more languages in top languages chart 
Increase from 10 to 14 rows so languages visible in the contribution
graph (e.g. Svelte, C++) also appear in the legend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:52:33 +03:00
rob thijssen
283b2126c0 feat(ui): color contribution graph circles by dominant language 
Replace fixed green palette with per-period dominant language colors.
Each circle's hue reflects the language with the most commits for that
day (last-year graph) or month (all-time graph), with opacity scaled
by volume quartile. Language data comes from the existing language
daily counts endpoint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:50:19 +03:00
rob thijssen
e8dcb5fcaf feat(ui): show private activity count on timeline when no public events 
When viewing a date range with zero public activities, the status line
now shows the count of private contributions (derived from daily counts
which include private repos). Helps explain gaps in the public timeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:41:22 +03:00
rob thijssen
b41e8c330a feat: include private repo contributions in graph metrics 
Aggregate graph endpoints (daily counts, language daily counts, source
summaries, OG image) now include private repository activity. These
endpoints only expose numeric counts — no commit messages, repo names,
or other metadata — so private details remain hidden. The activity
timeline continues to serve only public events.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:35:22 +03:00
rob thijssen
f386e0b574 feat(ui): reshape all-time graph and add dashboard stats panels 
Transpose AllTimeGraph to show years on X axis and months on Y axis
instead of year-per-row with weekly columns. Add TopLanguages bar chart
(all-time code volume by language) and ContributionStats panel (current
and longest streaks, busiest day, active days, weekday averages) in a
three-column row matching the project card grid.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-11 15:27:39 +03:00
rob thijssen
111a2af573 feat(ui): language distribution bar on project cards 
Extract LanguageBar into a shared component used by both DashPage
(compact, bar only) and ProjectPage (full, with percentage labels).
Remove redundant forge source text from project cards since the
forge icon already indicates it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 07:13:41 +03:00
rob thijssen
6f30a61184 feat(ui): smooth language stream graph with Catmull-Rom splines 
Replace straight line segments with cubic bezier curves using
Catmull-Rom to bezier conversion (tension 0.5) for a smoother
stream graph visualization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 07:02:50 +03:00
rob thijssen
14643273c0 fix: weight language graph by repo language proportions 
Each commit was counted once per language in the repo regardless of
that language's share, so Shell (present in many repos as small
deploy scripts) appeared larger than Rust. Now weights each commit
by the language's byte proportion in the repo (e.g. a commit to a
95% Rust / 5% Shell repo contributes 0.95 to Rust, 0.05 to Shell).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 06:59:47 +03:00
rob thijssen
ee93429317 feat: language stream graph on dashboard 
Full-stack feature showing programming languages by commit activity
as a stream graph on the dashboard.

Backend:
- migration: repo_languages table (source, repo, language, bytes, color)
- worker: fetch language breakdowns via GitHub GraphQL (batched,
  20 repos/request) and Gitea REST API during poll cycles
- API: GET /v1/languages/daily (daily commit counts per language),
  GET /v1/languages/repos (all stored repo language data)
- fix timezone bug in daily_counts and language_daily_counts: the
  PostgreSQL server timezone (Europe/Sofia, UTC+3) shifted day
  boundaries, miscounting events near midnight. Now uses explicit
  UTC boundaries in generate_series JOINs.
- use per-source CASE for repo name extraction in language query
  to match gitea payload structure (repo.full_name vs repo.name)
- Gitea languages use GitHub colors via COALESCE fallback

Frontend:
- LanguageStreamGraph component: pure SVG stream graph, weekly
  buckets, centered baseline, top 8 languages + Other, GitHub
  canonical language colors, legend with color dots
- DashPage/ProjectPage: fetch repo languages once via new endpoint
  instead of per-repo forge proxy calls (eliminates 200+ GitHub
  API calls and 403 rate limit errors)
- removed fetchLanguages forge proxy wrapper (dead code)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 06:27:59 +03:00
rob thijssen
c66aaeb268 feat: discover contributed repos via GitHub GraphQL API 
The REST /user/repos endpoint only returns repos where the user is
owner, collaborator, or org member. Repos contributed to via PRs
(e.g. polkadot-js/api, zed-industries/zed) were never discovered
and their commits were missing from moments.

Now supplements /user/repos with a GraphQL
repositoriesContributedTo query, which returns all repos the user
has committed to, opened issues/PRs on, or reviewed — with cursor-
based pagination and no result cap.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 05:38:57 +03:00
rob thijssen
2a20b47a29 fix: resolve clippy redundant_closure warning in moments-api 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 05:04:18 +03:00
rob thijssen
f77a8ab48f fix: use since cursor in github-repo polls to prevent missed commits 
After initial backfill, scan_repo was fetching only page 1 (100 most
recent commits) per repo. If more than 100 commits landed between
7-day polls, older ones in that window were permanently missed.

Now stores the newest commit date in poller_state.last_modified and
passes it as &since= on subsequent polls, with full pagination, so
only genuinely new commits are fetched but none are skipped.

On first poll after deploy, last_modified is NULL so no since filter
is applied — triggering a full re-backfill that catches any
previously missed commits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 05:03:41 +03:00
rob thijssen
1679153c43 docs: add CLAUDE.md and ignore .zed/ 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-06 04:43:00 +03:00
rob thijssen
0aa53d30db docs: rewrite readme to reflect current architecture 
Cover all data sources (github events/search/repo, gitea with org
discovery, hg revset queries, bugzilla), frontend routes (dash with
contribution graphs, activity timeline with timespan filtering,
project detail with readme/languages, cv), api endpoints including
forge proxy and og image, environment variables, and deployment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:52:25 +03:00
rob thijssen
cd833b18f1 fix(ui): demote repos with >= 10k commits to end of dashboard 
Automated/bulk-commit repos score -1 so they sort last regardless
of recency or volume.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:48:52 +03:00
rob thijssen
293d112c18 fix: fall back to _repo in commit reshape for github-repo events 
The commit presentation layer only checked repository.full_name,
missing commits ingested by github_repo which store the repo name
in _repo instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:44:31 +03:00
rob thijssen
ef1e84a41b feat(ui): link forge icon to repo on project page 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:42:04 +03:00
rob thijssen
f8c13b5e21 fix: icon colors for dark backgrounds 2026-05-05 18:40:29 +03:00
rob thijssen
abc90c8da0 feat(ui): forge icon on project page header 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:30:13 +03:00
rob thijssen
d46a0e3777 fix: add _repo fallback to events repo filter for github-repo commits 
The events query's COALESCE for github source was missing _repo,
so per-repo commit events from github_repo had no repo match and
project pages showed 0 activities.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:28:32 +03:00
rob thijssen
642209068a feat(ui): forge icons on repo cards (github, gitea, mozilla) 
Add SVG icons for each forge before the repo name on dashboard cards.
Icons sourced from user-provided SVGs in ui/public/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:24:47 +03:00
rob thijssen
c1e964de06 feat(ui): show all repos on dashboard instead of top 24 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 18:09:34 +03:00
rob thijssen
45fd45f5da fix: stamp _repo into github-repo commit payloads for project attribution 
The /repos/{owner}/{repo}/commits endpoint doesn't include repo info
in its response. Without _repo in the payload, these commits were
invisible to the projects query. Add _repo to parse_commit and include
it in the COALESCE chain for github source repo extraction.

After deploy, reset github-repo poller state to re-ingest with _repo:
  DELETE FROM poller_state WHERE source LIKE 'github-repo%';

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:59:31 +03:00
rob thijssen
03c816d2d3 feat(ui): show repo count in contribution graph summaries 
Add "in N repositories" to both the year and all-time graph summary
lines. Year graph counts repos with overlapping activity; all-time
graph uses total project count. OG image includes repo count too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:50:15 +03:00
rob thijssen
13db392273 fix(nginx): exclude /api/ paths from static asset location block 
The static asset regex matched .png in /api/v1/og/contributions.png
before the /api/ proxy block, returning 404. Add negative lookahead
to skip /api/ paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:40:31 +03:00
rob thijssen
e63583877c feat(api): server-rendered OG image of all-time contribution graph 
Add /v1/og/contributions.png endpoint that builds an SVG of the
all-time weekly contribution graph (one row per year) from daily
counts, then rasterizes to PNG via resvg. Served with 1h cache.

Add og:image and twitter:card meta tags to index.html pointing at
the endpoint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:37:19 +03:00
rob thijssen
2284a886d0 fix(ui): all-time graph as year rows with 52 weekly columns each 
Restructure the all-time contribution graph from a single row of ~700
circles (sub-pixel when scaled) to one row per year with ~52 weekly
columns, matching the width of the daily graph above. Year labels on
the left.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:15:49 +03:00
rob thijssen
1ca85fe632 feat(ui): all-time weekly contribution graph + date range timespan support 
Add AllTimeGraph component showing one circle per week across the full
history (earliest event to today). Uses the /sources endpoint to find
the earliest date, then fetches daily counts and aggregates to weekly.
Clicking a week navigates to /activity/YYYY-MM-DD..YYYY-MM-DD.

Update parseTimespan to handle both date-only (YYYY-MM-DD) and full
ISO datetime strings in range expressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:13:49 +03:00
rob thijssen
822def3227 fix(ui): scale contribution graph to full container width 
Use viewBox + width=100% instead of fixed pixel dimensions so the
SVG scales to match the project card grid below.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:08:17 +03:00
rob thijssen
27ce16e630 feat(ui): contribution graph with daily activity heatmap 
Add /v1/activity/daily endpoint returning per-day event counts via
generate_series + LEFT JOIN. Frontend renders an SVG contribution
graph with circles colored by quantile-based thresholds. Clicking a
day navigates to /activity/YYYY-MM-DD showing that day's events.

New /activity/:timespan route parses single dates (YYYY-MM-DD) and
ranges (YYYY-MM-DD..YYYY-MM-DD) from the URL to initialize the
activity timeline filter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 17:05:28 +03:00
rob thijssen
7de23303bd chore(ui): add favicon set to index.html 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:50:19 +03:00
rob thijssen
0d350ce584 fix: decode base64 readme content as utf-8 instead of latin-1 
atob() produces Latin-1 strings, mangling multi-byte UTF-8 characters
like box-drawing glyphs. Use TextDecoder for correct UTF-8 handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:28:40 +03:00
rob thijssen
1275a7785f chore: update Cargo.lock for reqwest in moments-api 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:25:19 +03:00
rob thijssen
6b9ce99a06 fix: proxy forge API requests to avoid CORS, case-insensitive readme 
Add /v1/forge/{source}/* proxy endpoint to the API server with an
allowlisted set of hosts. Frontend readme and language requests now
go through the proxy instead of hitting forge APIs directly (Gitea
has no CORS headers). Gitea readme fetch tries README.md, readme.md,
and Readme.md casings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:24:32 +03:00
rob thijssen
f676ecdc19 fix: try multiple readme filename casings for Gitea repos 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:19:34 +03:00
rob thijssen
46ef63a68e fix: source-aware repo extraction, Gitea readme/languages endpoints 
Use CASE/source instead of COALESCE for repo name extraction — Gitea's
repo.name is the short name while full_name includes the owner prefix.
Fix Gitea README fetch to use /contents/README.md with base64 decoding
instead of the nonexistent /readme endpoint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 16:18:40 +03:00
rob thijssen
ba216580ea feat(ui): project readme, language bars, and per-card language summary 
ProjectPage fetches README (raw markdown) and language breakdown from
GitHub/Gitea REST APIs, rendering the readme as markdown and languages
as a colored proportional bar with labels.

Dashboard cards lazily fetch top 3 languages per repo and display them
inline. Language color map covers common languages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 15:28:15 +03:00
rob thijssen
80f3f7c5cb feat(ui): project drill-down route with repo-filtered event timeline 
Add repo filter param to /v1/events (SQL COALESCE across payload
shapes per source). New /project/:source/* route renders a filtered
activity timeline for a single repo. Dashboard cards link to the
drill-down page.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 15:22:11 +03:00
rob thijssen
a70fab4feb feat(ui): add /dash route, shared nav, project dashboard with /v1/projects API 
Restructure routes: / and /dash show a project overview dashboard,
/activity hosts the existing timeline, /cv remains. Shared Layout
component provides consistent nav header and footer across all routes.

New /v1/projects endpoint aggregates per-repo activity stats (commits,
issues, PRs, date range) from existing event data via SQL. Dashboard
ranks projects by weighted recency + volume score and renders a card
grid.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 15:19:49 +03:00
rob thijssen
a71b4e6b84 feat(github): per-repo commit enumeration for full history backfill 
Adds a new github-repo EventSource that enumerates all repos via
/user/repos and walks each repo's /commits?author= endpoint, which
has no 1000-result cap unlike the Search API. Events use the same
github-commit:{sha} ID scheme as github_search for dedup. Per-repo
poller state enables full backfill on first run, page-1-only on
subsequent polls. Weekly poll interval by default.

Closes #1

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 14:59:26 +03:00
rob thijssen
2da9461b44 fix(hg): show clone errors, stable cwd; shrink timeline fonts 
Remove /dev/null redirects in hg-ingest.sh so errors are visible.
cd to work dir before loop to prevent getcwd failures after rm.
Use $HOME instead of ~ for proper expansion in default values.

Reduce timeline entry title, subtitle, and body font sizes for a
more compact activity feed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 14:45:26 +03:00
rob thijssen
3f3a1fb33e fix: connection string 2026-05-05 14:22:42 +03:00
rob thijssen
88fbbba60b feat(hg): revset-based author query, group discovery, one-shot ingest script 
Rewrites the hg worker to use json-log?rev=author() which matches the
changeset author (not the pusher), capturing commits landed by sheriffs.
Repos are discovered within configured groups plus individually listed
repos. The worker skips entirely after the first successful backfill.

Adds script/hg-ingest.sh for offline ingestion via local hg clones —
clones one repo at a time, caches extracted changesets to .tsv, inserts
via psql, and sets poller_state when done.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 13:58:21 +03:00
rob thijssen
1bbe55dc84 feat(gitea): poll org activity feeds to capture cross-namespace events 
The user activity feed only returns events from the user's own namespace.
This adds org discovery via /api/v1/user/orgs and polls each org's
activity feed, filtering for events by the configured user. Per-org
poller state keys enable independent backfill. Org feed errors are
non-fatal to avoid disrupting the user feed poll.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-05 12:23:25 +03:00
rob thijssen
4c8a663288 feat(ui): add /cv route, site-wide lowercase, no-cookies footer 
reproduces the legacy cv (previously at grenade.github.io/cv) as a
react-router /cv route, fetched at runtime from the same gist. moves
the lowercase aesthetic from per-element overrides to a single body-
level rule so a future toggle can flip it from one place. adds a small
site-wide footer noting why no cookie consent banner is shown.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 17:22:44 +03:00
rob thijssen
8867ff5df3 feat(deploy): manifest-driven config, teardown + db-perms, hardening 
deploy.sh:
- never rsync into /; stage to /tmp on the remote and install at final
  paths via sudo bash heredoc, closing the parent-dir attribute leak
  that broke three hosts in the earlier rsync incident
- shell-quote heredoc args via ${var@Q}
- drop -A -X on the remaining (web) rsyncs
- generic worker.secrets loop reads (env-var → pass path) from manifest;
  GITEA_TOKEN now flows through automatically
- in-memory bash substitution for templates (secrets never on argv)
- simplify semanage port labelling: --add 2>/dev/null || --modify (the
  old grep pre-check matched only the first listed port)
- restorecon back to short flags (Fedora policycoreutils has no long
  forms; --recursive errored at deploy time)
- quieter health probe loop: curl diagnostics only on final failure

manifest as source of truth:
- api.config.bind drives BIND_ADDR, firewalld port, semanage label,
  health-probe URL
- web.config.{server_name,root,api_upstream} drives nginx render,
  rsync targets, restorecon scope
- nginx config renamed to site.conf.tmpl; firewalld svc to
  moments-api.xml.tmpl; both rendered at deploy time
- topology flip: api → nikola, worker → frootmig (anjie freed)

new scripts:
- script/teardown.sh: idempotent component teardown, never rsyncs,
  shared-state cleanup gated on absence of remaining env files,
  --remove-docroot guard against shallow / system paths
- script/db-perms.sh: rewritten — fixes grep/append role mismatch that
  appended duplicates on re-run, adds postgres reload, hits primary +
  standby in a single invocation

readme: genericized; deployment topology no longer carries real host
or site names.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 16:39:10 +03:00
64 changed files with 6211 additions and 477 deletions
Expand all files Collapse all files

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@
**/*.rs.bk **/*.rs.bk
.env .env
.env.local .env.local
.zed/
# frontend # frontend
/ui/node_modules /ui/node_modules

78
CLAUDE.md Normal file
View File

@@ -0,0 +1,78 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**moments** is a personal activity timeline and portfolio site. It ingests developer activity from multiple forges (GitHub, Gitea, Mercurial, Bugzilla), stores raw JSON payloads in PostgreSQL, and serves a React frontend showing contribution graphs, a ranked project dashboard, and a filterable activity timeline.
## Architecture
Hexagonal (ports & adapters) Rust backend with a React/TypeScript frontend.
### Crate Dependency Graph
```
moments-entities — pure types/DTOs, no DB or HTTP deps
^
moments-core — port traits (EventReader, EventWriter, EventSource, PollerStateStore)
+ presentation reshape + poller loop
^
moments-data — sole adapter: PgStore implements all core traits
+ EventSource impls (github, gitea, hg, bugzilla)
+ SQL migrations
^
moments-api — axum HTTP API binary (read-only, connects as moments_ro)
moments-worker — ingestion daemon binary (runs migrations, connects as moments_rw)
```
### Key Design Decisions
- **Raw payload storage**: upstream JSON is stored verbatim in `events.payload` (JSONB). The `reshape()` function in `moments-core/src/presentation.rs` transforms payloads into `TimelineItem` at request time — no re-ingestion needed to change presentation.
- **Public/private gate**: `events.public` boolean controls API visibility. Only `public = true` rows are served.
- **Wire types are hand-maintained**: `ui/src/api/client.ts` mirrors Rust entity types manually.
- **Migrations**: run automatically on worker startup via `sqlx::migrate!`. The API binary never runs migrations.
### Frontend
React 19 + Vite 6 (SWC) + TypeScript + Bootstrap 5. State/data via `@tanstack/react-query`. Package manager is **pnpm**.
Routes: `/` (dashboard), `/activity` (timeline), `/project/:source/*` (project detail), `/cv` (resume).
## Build & Dev Commands
### Rust
```sh
cargo build --workspace # build all crates
cargo build --workspace --release # release build
cargo clippy --workspace # lint
cargo fmt --check # format check
cargo test --workspace # run tests
# Run binaries (need DATABASE_URL)
DATABASE_URL=postgres://localhost/moments cargo run -p moments-api
DATABASE_URL=postgres://localhost/moments cargo run -p moments-worker
```
### Frontend
```sh
cd ui
pnpm install # install deps
pnpm dev # dev server on :5173 (proxies /api/* to localhost:8080)
pnpm lint # tsc --noEmit type-check
pnpm build # production build (tsc -b && vite build)
```
## Database
PostgreSQL with three migrations in `crates/moments-data/migrations/`. Two roles: `moments_rw` (worker, full access) and `moments_ro` (API, SELECT-only).
## API Endpoints
All under `/v1/`: `healthz`, `events`, `sources`, `projects`, `activity/daily`, `forge/{source}/*`, `og/contributions.png`.
## Deployment
Production uses `./script/deploy.sh`. Services run under systemd with hardened units. Secrets resolved from `pass` store via template substitution. Nginx reverse-proxies `/api/` to the API host.

391
Cargo.lock generated
View File

@@ -88,6 +88,18 @@ version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "arrayref"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb"
[[package]]
name = "arrayvec"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]] [[package]]
name = "async-compression" name = "async-compression"
version = "0.4.42" version = "0.4.42"
@@ -196,6 +208,12 @@ version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bitflags"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]] [[package]]
name = "bitflags" name = "bitflags"
version = "2.11.1" version = "2.11.1"
@@ -220,12 +238,24 @@ version = "3.20.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb" checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
[[package]]
name = "bytemuck"
version = "1.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8efb64bd706a16a1bdde310ae86b351e4d21550d98d056f22f8a7f7a2183fec"
[[package]] [[package]]
name = "byteorder" name = "byteorder"
version = "1.5.0" version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "byteorder-lite"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f1fe948ff07f4bd06c30984e69f5b4899c516a3ef74f34df92a2df2ab535495"
[[package]] [[package]]
name = "bytes" name = "bytes"
version = "1.11.1" version = "1.11.1"
@@ -306,6 +336,12 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "color_quant"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
[[package]] [[package]]
name = "colorchoice" name = "colorchoice"
version = "1.0.5" version = "1.0.5"
@@ -350,6 +386,15 @@ version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "core_maths"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77745e017f5edba1a9c1d854f6f3a52dac8a12dd5af5d2f54aecf61e43d80d30"
dependencies = [
"libm",
]
[[package]] [[package]]
name = "cpufeatures" name = "cpufeatures"
version = "0.2.17" version = "0.2.17"
@@ -408,6 +453,12 @@ dependencies = [
"typenum", "typenum",
] ]
[[package]]
name = "data-url"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be1e0bca6c3637f992fc1cc7cbc52a78c1ef6db076dbf1059c4323d6a2048376"
[[package]] [[package]]
name = "der" name = "der"
version = "0.7.10" version = "0.7.10"
@@ -484,6 +535,15 @@ dependencies = [
"windows-sys 0.48.0", "windows-sys 0.48.0",
] ]
[[package]]
name = "euclid"
version = "0.22.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1a05365e3b1c6d1650318537c7460c6923f1abdd272ad6842baa2b509957a06"
dependencies = [
"num-traits",
]
[[package]] [[package]]
name = "event-listener" name = "event-listener"
version = "5.4.1" version = "5.4.1"
@@ -495,6 +555,15 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
] ]
[[package]]
name = "fdeflate"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e6853b52649d4ac5c0bd02320cddc5ba956bdb407c4b75a2c6b75bf51500f8c"
dependencies = [
"simd-adler32",
]
[[package]] [[package]]
name = "find-msvc-tools" name = "find-msvc-tools"
version = "0.1.9" version = "0.1.9"
@@ -511,6 +580,12 @@ dependencies = [
"miniz_oxide", "miniz_oxide",
] ]
[[package]]
name = "float-cmp"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98de4bbd547a563b716d8dfa9aad1cb19bfab00f4fa09a6a4ed21dbcf44ce9c4"
[[package]] [[package]]
name = "flume" name = "flume"
version = "0.11.1" version = "0.11.1"
@@ -528,6 +603,29 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "fontconfig-parser"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbc773e24e02d4ddd8395fd30dc147524273a83e54e0f312d986ea30de5f5646"
dependencies = [
"roxmltree",
]
[[package]]
name = "fontdb"
version = "0.23.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "457e789b3d1202543297a350643cf459f836cade38934e7a4cf6a39e7cde2905"
dependencies = [
"fontconfig-parser",
"log",
"memmap2",
"slotmap",
"tinyvec",
"ttf-parser",
]
[[package]] [[package]]
name = "form_urlencoded" name = "form_urlencoded"
version = "1.2.2" version = "1.2.2"
@@ -645,6 +743,16 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "gif"
version = "0.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ae047235e33e2829703574b54fdec96bfbad892062d97fed2f76022287de61b"
dependencies = [
"color_quant",
"weezl",
]
[[package]] [[package]]
name = "hashbrown" name = "hashbrown"
version = "0.15.5" version = "0.15.5"
@@ -941,6 +1049,22 @@ dependencies = [
"icu_properties", "icu_properties",
] ]
[[package]]
name = "image-webp"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "525e9ff3e1a4be2fbea1fdf0e98686a6d98b4d8f937e1bf7402245af1909e8c3"
dependencies = [
"byteorder-lite",
"quick-error",
]
[[package]]
name = "imagesize"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "edcd27d72f2f071c64249075f42e205ff93c9a4c5f6c6da53e79ed9f9832c285"
[[package]] [[package]]
name = "indexmap" name = "indexmap"
version = "2.14.0" version = "2.14.0"
@@ -991,6 +1115,17 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "kurbo"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c62026ae44756f8a599ba21140f350303d4f08dcdcc71b5ad9c9bb8128c13c62"
dependencies = [
"arrayvec",
"euclid",
"smallvec",
]
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
version = "1.5.0" version = "1.5.0"
@@ -1018,7 +1153,7 @@ version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e02f3bb43d335493c96bf3fd3a321600bf6bd07ed34bc64118e9293bdffea46c" checksum = "e02f3bb43d335493c96bf3fd3a321600bf6bd07ed34bc64118e9293bdffea46c"
dependencies = [ dependencies = [
"bitflags", "bitflags 2.11.1",
"libc", "libc",
"plain", "plain",
"redox_syscall 0.7.4", "redox_syscall 0.7.4",
@@ -1092,6 +1227,15 @@ version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "memmap2"
version = "0.9.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714098028fe011992e1c3962653c96b2d578c4b4bce9036e15ff220319b1e0e3"
dependencies = [
"libc",
]
[[package]] [[package]]
name = "mime" name = "mime"
version = "0.3.17" version = "0.3.17"
@@ -1127,9 +1271,12 @@ dependencies = [
"axum", "axum",
"chrono", "chrono",
"clap", "clap",
"fontdb",
"moments-core", "moments-core",
"moments-data", "moments-data",
"moments-entities", "moments-entities",
"reqwest",
"resvg",
"serde", "serde",
"serde_json", "serde_json",
"tokio", "tokio",
@@ -1160,6 +1307,7 @@ dependencies = [
"chrono", "chrono",
"moments-core", "moments-core",
"moments-entities", "moments-entities",
"percent-encoding",
"reqwest", "reqwest",
"serde", "serde",
"serde_json", "serde_json",
@@ -1307,6 +1455,12 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pico-args"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5be167a7af36ee22fe3115051bc51f6e6c7054c9348e28deb4f49bd6f705a315"
[[package]] [[package]]
name = "pin-project-lite" name = "pin-project-lite"
version = "0.2.17" version = "0.2.17"
@@ -1346,6 +1500,19 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6" checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
[[package]]
name = "png"
version = "0.17.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82151a2fc869e011c153adc57cf2789ccb8d9906ce52c0b39a6b5697749d7526"
dependencies = [
"bitflags 1.3.2",
"crc32fast",
"fdeflate",
"flate2",
"miniz_oxide",
]
[[package]] [[package]]
name = "potential_utf" name = "potential_utf"
version = "0.1.5" version = "0.1.5"
@@ -1373,6 +1540,12 @@ dependencies = [
"unicode-ident", "unicode-ident",
] ]
[[package]]
name = "quick-error"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
[[package]] [[package]]
name = "quinn" name = "quinn"
version = "0.11.9" version = "0.11.9"
@@ -1508,7 +1681,7 @@ version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [ dependencies = [
"bitflags", "bitflags 2.11.1",
] ]
[[package]] [[package]]
@@ -1517,7 +1690,7 @@ version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f450ad9c3b1da563fb6948a8e0fb0fb9269711c9c73d9ea1de5058c79c8d643a" checksum = "f450ad9c3b1da563fb6948a8e0fb0fb9269711c9c73d9ea1de5058c79c8d643a"
dependencies = [ dependencies = [
"bitflags", "bitflags 2.11.1",
] ]
[[package]] [[package]]
@@ -1575,6 +1748,32 @@ dependencies = [
"webpki-roots 1.0.7", "webpki-roots 1.0.7",
] ]
[[package]]
name = "resvg"
version = "0.45.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8928798c0a55e03c9ca6c4c6846f76377427d2c1e1f7e6de3c06ae57942df43"
dependencies = [
"gif",
"image-webp",
"log",
"pico-args",
"rgb",
"svgtypes",
"tiny-skia",
"usvg",
"zune-jpeg",
]
[[package]]
name = "rgb"
version = "0.8.53"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47b34b781b31e5d73e9fbc8689c70551fd1ade9a19e3e28cfec8580a79290cc4"
dependencies = [
"bytemuck",
]
[[package]] [[package]]
name = "ring" name = "ring"
version = "0.17.14" version = "0.17.14"
@@ -1589,6 +1788,12 @@ dependencies = [
"windows-sys 0.52.0", "windows-sys 0.52.0",
] ]
[[package]]
name = "roxmltree"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
[[package]] [[package]]
name = "rsa" name = "rsa"
version = "0.9.10" version = "0.9.10"
@@ -1656,6 +1861,24 @@ version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "rustybuzz"
version = "0.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd3c7c96f8a08ee34eff8857b11b49b07d71d1c3f4e88f8a88d4c9e9f90b1702"
dependencies = [
"bitflags 2.11.1",
"bytemuck",
"core_maths",
"log",
"smallvec",
"ttf-parser",
"unicode-bidi-mirroring",
"unicode-ccc",
"unicode-properties",
"unicode-script",
]
[[package]] [[package]]
name = "ryu" name = "ryu"
version = "1.0.23" version = "1.0.23"
@@ -1797,12 +2020,36 @@ version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214" checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "simplecss"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a9c6883ca9c3c7c90e888de77b7a5c849c779d25d74a1269b0218b14e8b136c"
dependencies = [
"log",
]
[[package]]
name = "siphasher"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
[[package]] [[package]]
name = "slab" name = "slab"
version = "0.4.12" version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "slotmap"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bdd58c3c93c3d278ca835519292445cb4b0d4dc59ccfdf7ceadaab3f8aeb4038"
dependencies = [
"version_check",
]
[[package]] [[package]]
name = "smallvec" name = "smallvec"
version = "1.15.1" version = "1.15.1"
@@ -1937,7 +2184,7 @@ checksum = "aa003f0038df784eb8fecbbac13affe3da23b45194bd57dba231c8f48199c526"
dependencies = [ dependencies = [
"atoi", "atoi",
"base64", "base64",
"bitflags", "bitflags 2.11.1",
"byteorder", "byteorder",
"bytes", "bytes",
"chrono", "chrono",
@@ -1980,7 +2227,7 @@ checksum = "db58fcd5a53cf07c184b154801ff91347e4c30d17a3562a635ff028ad5deda46"
dependencies = [ dependencies = [
"atoi", "atoi",
"base64", "base64",
"bitflags", "bitflags 2.11.1",
"byteorder", "byteorder",
"chrono", "chrono",
"crc", "crc",
@@ -2041,6 +2288,15 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "strict-num"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6637bab7722d379c8b41ba849228d680cc12d0a45ba1fa2b48f2a30577a06731"
dependencies = [
"float-cmp",
]
[[package]] [[package]]
name = "stringprep" name = "stringprep"
version = "0.1.5" version = "0.1.5"
@@ -2064,6 +2320,16 @@ version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "svgtypes"
version = "0.15.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68c7541fff44b35860c1a7a47a7cadf3e4a304c457b58f9870d9706ece028afc"
dependencies = [
"kurbo",
"siphasher",
]
[[package]] [[package]]
name = "syn" name = "syn"
version = "2.0.117" version = "2.0.117"
@@ -2124,6 +2390,32 @@ dependencies = [
"cfg-if", "cfg-if",
] ]
[[package]]
name = "tiny-skia"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83d13394d44dae3207b52a326c0c85a8bf87f1541f23b0d143811088497b09ab"
dependencies = [
"arrayref",
"arrayvec",
"bytemuck",
"cfg-if",
"log",
"png",
"tiny-skia-path",
]
[[package]]
name = "tiny-skia-path"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c9e7fc0c2e86a30b117d0462aa261b72b7a99b7ebd7deb3a14ceda95c5bdc93"
dependencies = [
"arrayref",
"bytemuck",
"strict-num",
]
[[package]] [[package]]
name = "tinystr" name = "tinystr"
version = "0.8.3" version = "0.8.3"
@@ -2233,7 +2525,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d4e6559d53cc268e5031cd8429d05415bc4cb4aefc4aa5d6cc35fbf5b924a1f8" checksum = "d4e6559d53cc268e5031cd8429d05415bc4cb4aefc4aa5d6cc35fbf5b924a1f8"
dependencies = [ dependencies = [
"async-compression", "async-compression",
"bitflags", "bitflags 2.11.1",
"bytes", "bytes",
"futures-core", "futures-core",
"futures-util", "futures-util",
@@ -2343,6 +2635,15 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "ttf-parser"
version = "0.25.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2df906b07856748fa3f6e0ad0cbaa047052d4a7dd609e231c4f72cee8c36f31"
dependencies = [
"core_maths",
]
[[package]] [[package]]
name = "typenum" name = "typenum"
version = "1.20.0" version = "1.20.0"
@@ -2355,6 +2656,18 @@ version = "0.3.18"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c1cb5db39152898a79168971543b1cb5020dff7fe43c8dc468b0885f5e29df5" checksum = "5c1cb5db39152898a79168971543b1cb5020dff7fe43c8dc468b0885f5e29df5"
[[package]]
name = "unicode-bidi-mirroring"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5dfa6e8c60bb66d49db113e0125ee8711b7647b5579dc7f5f19c42357ed039fe"
[[package]]
name = "unicode-ccc"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce61d488bcdc9bc8b5d1772c404828b17fc481c0a582b5581e95fb233aef503e"
[[package]] [[package]]
name = "unicode-ident" name = "unicode-ident"
version = "1.0.24" version = "1.0.24"
@@ -2376,6 +2689,18 @@ version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d" checksum = "7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d"
[[package]]
name = "unicode-script"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "383ad40bb927465ec0ce7720e033cb4ca06912855fc35db31b5755d0de75b1ee"
[[package]]
name = "unicode-vo"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1d386ff53b415b7fe27b50bb44679e2cc4660272694b7b6f3326d8480823a94"
[[package]] [[package]]
name = "untrusted" name = "untrusted"
version = "0.9.0" version = "0.9.0"
@@ -2394,6 +2719,33 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "usvg"
version = "0.45.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80be9b06fbae3b8b303400ab20778c80bbaf338f563afe567cf3c9eea17b47ef"
dependencies = [
"base64",
"data-url",
"flate2",
"fontdb",
"imagesize",
"kurbo",
"log",
"pico-args",
"roxmltree",
"rustybuzz",
"simplecss",
"siphasher",
"strict-num",
"svgtypes",
"tiny-skia-path",
"unicode-bidi",
"unicode-script",
"unicode-vo",
"xmlwriter",
]
[[package]] [[package]]
name = "utf8_iter" name = "utf8_iter"
version = "1.0.4" version = "1.0.4"
@@ -2547,6 +2899,12 @@ dependencies = [
"rustls-pki-types", "rustls-pki-types",
] ]
[[package]]
name = "weezl"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88"
[[package]] [[package]]
name = "whoami" name = "whoami"
version = "1.6.1" version = "1.6.1"
@@ -2859,6 +3217,12 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "xmlwriter"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec7a2a501ed189703dba8b08142f057e887dfc4b2cc4db2d343ac6376ba3e0b9"
[[package]] [[package]]
name = "yoke" name = "yoke"
version = "0.8.2" version = "0.8.2"
@@ -2967,3 +3331,18 @@ name = "zmij"
version = "1.0.21" version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
[[package]]
name = "zune-core"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f423a2c17029964870cfaabb1f13dfab7d092a62a29a89264f4d36990ca414a"
[[package]]
name = "zune-jpeg"
version = "0.4.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29ce2c8a9384ad323cf564b67da86e21d3cfdff87908bc1223ed5c99bc792713"
dependencies = [
"zune-core",
]

2
Cargo.toml
View File

@@ -30,6 +30,8 @@ anyhow = "1"
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json", "gzip"] } reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json", "gzip"] }
figment = { version = "0.10", features = ["toml", "env"] } figment = { version = "0.10", features = ["toml", "env"] }
clap = { version = "4", features = ["derive", "env"] } clap = { version = "4", features = ["derive", "env"] }
resvg = "0.45"
fontdb = "0.23"
# internal # internal
moments-entities = { path = "crates/moments-entities", version = "=0.1.0" } moments-entities = { path = "crates/moments-entities", version = "=0.1.0" }

2
asset/config/api.env.tmpl
View File

@@ -1,6 +1,6 @@
JOURNAL_STREAM=1 JOURNAL_STREAM=1
RUST_LOG=info,sqlx=warn,tower_http=info RUST_LOG=info,sqlx=warn,tower_http=info
BIND_ADDR=0.0.0.0:42424 BIND_ADDR={{BIND}}
DATABASE_URL=postgres://moments_ro@magrathea.kosherinata.internal:5432/moments?sslmode=verify-full&sslrootcert=/etc/pki/ca-trust/source/anchors/root-internal.pem&sslcert=/etc/pki/tls/misc/{{HOSTNAME}}.pem&sslkey=/etc/pki/tls/private/{{HOSTNAME}}.pem DATABASE_URL=postgres://moments_ro@magrathea.kosherinata.internal:5432/moments?sslmode=verify-full&sslrootcert=/etc/pki/ca-trust/source/anchors/root-internal.pem&sslcert=/etc/pki/tls/misc/{{HOSTNAME}}.pem&sslkey=/etc/pki/tls/private/{{HOSTNAME}}.pem

7
asset/config/worker.env.tmpl
View File

@@ -7,14 +7,17 @@ GITHUB_USER=grenade
GITHUB_TOKEN={{GITHUB_TOKEN}} GITHUB_TOKEN={{GITHUB_TOKEN}}
POLL_INTERVAL_SECS=600 POLL_INTERVAL_SECS=600
SEARCH_POLL_INTERVAL_SECS=86400 SEARCH_POLL_INTERVAL_SECS=86400
REPO_POLL_INTERVAL_SECS=604800
GITEA_HOST=git.lair.cafe GITEA_HOST=git.lair.cafe
GITEA_USER=grenade GITEA_USER=grenade
GITEA_TOKEN={{GITEA_TOKEN}}
GITEA_POLL_INTERVAL_SECS=600 GITEA_POLL_INTERVAL_SECS=600
HG_HOST=hg-edge.mozilla.org HG_HOST=hg-edge.mozilla.org
HG_REPOS=build/puppet,build/tools,build/buildbot-configs HG_GROUPS=build,integration
HG_AUTHOR_TERMS=thijssen,grenade HG_REPOS=mozilla-central
HG_AUTHOR_TERMS=rthijssen,grenade
HG_POLL_INTERVAL_SECS=86400 HG_POLL_INTERVAL_SECS=86400
BUGZILLA_HOST=bugzilla.mozilla.org BUGZILLA_HOST=bugzilla.mozilla.org

2
asset/firewalld/moments-api.xml → asset/firewalld/moments-api.xml.tmpl
View File

@@ -2,5 +2,5 @@
<service> <service>
<short>moments-api</short> <short>moments-api</short>
<description>moments read-only HTTP API</description> <description>moments read-only HTTP API</description>
<port protocol="tcp" port="42424"/> <port protocol="tcp" port="{{API_PORT}}"/>
</service> </service>

8
asset/manifest.yml
View File

@@ -3,7 +3,7 @@ environments:
prod: prod:
components: components:
api: api:
hosts: [anjie.kosherinata.internal] hosts: [nikola.kosherinata.internal]
config: config:
bind: 0.0.0.0:42424 bind: 0.0.0.0:42424
db_role: moments_ro db_role: moments_ro
@@ -11,7 +11,7 @@ environments:
db_port: 5432 db_port: 5432
db_name: moments db_name: moments
worker: worker:
hosts: [anjie.kosherinata.internal] hosts: [frootmig.kosherinata.internal]
config: config:
db_role: moments_rw db_role: moments_rw
db_host: magrathea.kosherinata.internal db_host: magrathea.kosherinata.internal
@@ -27,10 +27,10 @@ environments:
bugzilla_email: rthijssen@mozilla.com bugzilla_email: rthijssen@mozilla.com
secrets: secrets:
GITHUB_TOKEN: github.com/grenade/admin-token GITHUB_TOKEN: github.com/grenade/admin-token
# GITEA_TOKEN, BUGZILLA_API_KEY: optional, omit unless required. GITEA_TOKEN: git.lair.cafe/grenade/admin-token
web: web:
hosts: [oolon.kosherinata.internal] hosts: [oolon.kosherinata.internal]
config: config:
server_name: rob.tn server_name: rob.tn
root: /var/www/rob.tn root: /var/www/rob.tn
api_upstream: http://anjie.kosherinata.internal:42424 api_upstream: http://nikola.kosherinata.internal:42424

18
asset/nginx/rob.tn.conf → asset/nginx/site.conf.tmpl
View File

@@ -1,18 +1,18 @@
upstream moments_api { upstream moments_api {
server anjie.kosherinata.internal:42424 max_fails=3 fail_timeout=30s; server {{API_UPSTREAM_ADDR}} max_fails=3 fail_timeout=30s;
keepalive 8; keepalive 8;
} }
server { server {
server_name rob.tn; server_name {{SERVER_NAME}};
listen 443 ssl; listen 443 ssl;
http2 on; http2 on;
ssl_certificate /etc/letsencrypt/live/rob.tn/fullchain.pem; ssl_certificate /etc/letsencrypt/live/{{SERVER_NAME}}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/rob.tn/privkey.pem; ssl_certificate_key /etc/letsencrypt/live/{{SERVER_NAME}}/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3; ssl_protocols TLSv1.2 TLSv1.3;
root /var/www/rob.tn; root {{DOCROOT}};
index index.html; index index.html;
location / { location / {
@@ -20,7 +20,7 @@ server {
add_header Cache-Control "no-cache" always; add_header Cache-Control "no-cache" always;
} }
location ~* \.(js|css|woff2?|ttf|eot|svg|png|jpg|jpeg|gif|ico|webp|avif)$ { location ~* ^(?!/api/)\S+\.(js|css|woff2?|ttf|eot|svg|png|jpg|jpeg|gif|ico|webp|avif)$ {
expires 30d; expires 30d;
add_header Cache-Control "public, max-age=2592000, immutable"; add_header Cache-Control "public, max-age=2592000, immutable";
try_files $uri =404; try_files $uri =404;
@@ -28,7 +28,7 @@ server {
location /api/ { location /api/ {
rewrite ^/api/(.*)$ /$1 break; rewrite ^/api/(.*)$ /$1 break;
proxy_pass http://moments_api; proxy_pass {{API_UPSTREAM_SCHEME}}://moments_api;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Connection ""; proxy_set_header Connection "";
proxy_set_header Host $host; proxy_set_header Host $host;
@@ -38,6 +38,6 @@ server {
proxy_connect_timeout 5s; proxy_connect_timeout 5s;
} }
access_log /var/log/nginx/rob.tn.access.log; access_log /var/log/nginx/{{SERVER_NAME}}.access.log;
error_log /var/log/nginx/rob.tn.error.log; error_log /var/log/nginx/{{SERVER_NAME}}.error.log;
} }

3
crates/moments-api/Cargo.toml
View File

@@ -20,3 +20,6 @@ serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
chrono.workspace = true chrono.workspace = true
clap.workspace = true clap.workspace = true
reqwest.workspace = true
resvg.workspace = true
fontdb.workspace = true

355
crates/moments-api/src/main.rs
View File

@@ -1,17 +1,17 @@
use std::{net::SocketAddr, sync::Arc}; use std::{net::SocketAddr, sync::Arc, time::Duration};
use axum::{ use axum::{
Json, Router, Json, Router,
extract::{Query, State}, extract::{Path, Query, State},
http::StatusCode, http::StatusCode,
response::IntoResponse, response::IntoResponse,
routing::get, routing::get,
}; };
use chrono::{DateTime, Utc}; use chrono::{DateTime, Datelike, NaiveDate, Utc};
use clap::Parser; use clap::Parser;
use moments_core::{EventReader, reshape}; use moments_core::{EventReader, reshape};
use moments_data::PgStore; use moments_data::PgStore;
use moments_entities::{EventQuery, Source, SourceSummary, TimelineItem}; use moments_entities::{DailyCount, EventQuery, HourlyAvg, LanguageDailyCount, ProjectSummary, RepoLanguage, Source, SourceSummary, TimelineItem};
use serde::Deserialize; use serde::Deserialize;
use tower_http::{cors::CorsLayer, trace::TraceLayer}; use tower_http::{cors::CorsLayer, trace::TraceLayer};
use tracing::info; use tracing::info;
@@ -29,6 +29,7 @@ struct Args {
#[derive(Clone)] #[derive(Clone)]
struct AppState { struct AppState {
store: Arc<PgStore>, store: Arc<PgStore>,
http: reqwest::Client,
} }
#[tokio::main] #[tokio::main]
@@ -42,14 +43,25 @@ async fn main() -> anyhow::Result<()> {
// public`. The worker must have run at least once before the api accepts // public`. The worker must have run at least once before the api accepts
// traffic; in deploy this is ordered via systemd dependencies (§3). // traffic; in deploy this is ordered via systemd dependencies (§3).
let store = PgStore::connect(&args.database_url).await?; let store = PgStore::connect(&args.database_url).await?;
let http = reqwest::Client::builder()
.timeout(Duration::from_secs(15))
.build()?;
let state = AppState { let state = AppState {
store: Arc::new(store), store: Arc::new(store),
http,
}; };
let app = Router::new() let app = Router::new()
.route("/v1/healthz", get(healthz)) .route("/v1/healthz", get(healthz))
.route("/v1/events", get(list_events)) .route("/v1/events", get(list_events))
.route("/v1/sources", get(list_sources)) .route("/v1/sources", get(list_sources))
.route("/v1/projects", get(list_projects))
.route("/v1/activity/daily", get(daily_counts))
.route("/v1/activity/hourly", get(hourly_avgs))
.route("/v1/languages/daily", get(language_daily_counts))
.route("/v1/languages/repos", get(repo_languages))
.route("/v1/forge/{source}/{*rest}", get(forge_proxy))
.route("/v1/og/contributions.png", get(og_contributions))
.with_state(state) .with_state(state)
.layer(TraceLayer::new_for_http()) .layer(TraceLayer::new_for_http())
.layer(CorsLayer::permissive()); .layer(CorsLayer::permissive());
@@ -81,6 +93,8 @@ struct EventsQueryParams {
to: Option<DateTime<Utc>>, to: Option<DateTime<Utc>>,
/// Comma-separated list, e.g. `source=github,gitea`. /// Comma-separated list, e.g. `source=github,gitea`.
source: Option<String>, source: Option<String>,
/// Filter to a specific repo, e.g. `repo=grenade/moments`.
repo: Option<String>,
limit: Option<u32>, limit: Option<u32>,
} }
@@ -100,6 +114,7 @@ async fn list_events(
from: params.from, from: params.from,
to: params.to, to: params.to,
sources, sources,
repo: params.repo,
// Public timeline only — private events stay in the DB but are never // Public timeline only — private events stay in the DB but are never
// surfaced. A future authenticated path can flip this. // surfaced. A future authenticated path can flip this.
include_private: false, include_private: false,
@@ -116,12 +131,342 @@ async fn list_sources(
) -> Result<Json<Vec<SourceSummary>>, ApiError> { ) -> Result<Json<Vec<SourceSummary>>, ApiError> {
let summaries = state let summaries = state
.store .store
.source_summaries(/* include_private */ false) .source_summaries(/* include_private */ true)
.await .await
.map_err(internal)?; .map_err(internal)?;
Ok(Json(summaries)) Ok(Json(summaries))
} }
async fn list_projects(
State(state): State<AppState>,
) -> Result<Json<Vec<ProjectSummary>>, ApiError> {
let projects = state.store.list_projects().await.map_err(internal)?;
Ok(Json(projects))
}
#[derive(Debug, Deserialize)]
struct DailyCountsParams {
from: Option<NaiveDate>,
to: Option<NaiveDate>,
}
async fn daily_counts(
State(state): State<AppState>,
Query(params): Query<DailyCountsParams>,
) -> Result<Json<Vec<DailyCount>>, ApiError> {
let to = params.to.unwrap_or_else(|| Utc::now().date_naive());
let from = params.from.unwrap_or_else(|| to - chrono::Duration::days(365));
let counts = state.store.daily_counts(from, to, /* include_private */ true).await.map_err(internal)?;
Ok(Json(counts))
}
async fn language_daily_counts(
State(state): State<AppState>,
Query(params): Query<DailyCountsParams>,
) -> Result<Json<Vec<LanguageDailyCount>>, ApiError> {
let to = params.to.unwrap_or_else(|| Utc::now().date_naive());
let from = params.from.unwrap_or_else(|| to - chrono::Duration::days(365));
let counts = state.store.language_daily_counts(from, to, /* include_private */ true).await.map_err(internal)?;
Ok(Json(counts))
}
#[derive(Debug, Deserialize)]
struct HourlyAvgsParams {
from: Option<NaiveDate>,
to: Option<NaiveDate>,
/// IANA timezone name (e.g. "Europe/Helsinki"). Defaults to UTC.
/// Hour buckets are computed in this zone so the chart matches the
/// clock the user sees.
tz: Option<String>,
}
async fn hourly_avgs(
State(state): State<AppState>,
Query(params): Query<HourlyAvgsParams>,
) -> Result<Json<Vec<HourlyAvg>>, ApiError> {
let to = params.to.unwrap_or_else(|| Utc::now().date_naive());
let from = params.from.unwrap_or_else(|| to - chrono::Duration::days(365));
let tz = params.tz.as_deref().unwrap_or("UTC");
// Validate the tz string before handing it to postgres — a bad name
// here would surface as an opaque 500 from the DB. chrono-tz would do
// it for free but we don't depend on it; instead reject obvious shell
// injection vectors (the value is bound, not interpolated, so this is
// belt-and-braces).
if tz.len() > 64 || tz.chars().any(|c| !(c.is_ascii_alphanumeric() || matches!(c, '/' | '_' | '+' | '-'))) {
return Err(ApiError {
status: StatusCode::BAD_REQUEST,
message: "invalid tz".into(),
});
}
let avgs = state.store.hourly_avgs(from, to, tz, /* include_private */ true).await.map_err(internal)?;
Ok(Json(avgs))
}
async fn repo_languages(
State(state): State<AppState>,
) -> Result<Json<Vec<RepoLanguage>>, ApiError> {
let langs = state.store.repo_languages().await.map_err(internal)?;
Ok(Json(langs))
}
async fn og_contributions(
State(state): State<AppState>,
) -> Result<impl IntoResponse, ApiError> {
// Get date range from source summaries
let summaries = state
.store
.source_summaries(/* include_private */ true)
.await
.map_err(internal)?;
let earliest = summaries
.iter()
.filter_map(|s| s.earliest)
.min()
.unwrap_or_else(Utc::now)
.date_naive();
let today = Utc::now().date_naive();
let counts = state
.store
.daily_counts(earliest, today, /* include_private */ true)
.await
.map_err(internal)?;
let projects = state.store.list_projects().await.map_err(internal)?;
let repo_count = projects.len();
let png = render_contributions_png(&counts, earliest, today, repo_count).map_err(|e| ApiError {
status: StatusCode::INTERNAL_SERVER_ERROR,
message: e,
})?;
Ok((
StatusCode::OK,
[
(axum::http::header::CONTENT_TYPE, "image/png"),
(axum::http::header::CACHE_CONTROL, "public, max-age=3600"),
],
png,
))
}
fn render_contributions_png(
counts: &[DailyCount],
from: NaiveDate,
to: NaiveDate,
repo_count: usize,
) -> Result<Vec<u8>, String> {
use std::collections::HashMap;
let count_map: HashMap<NaiveDate, i64> = counts.iter().map(|d| (d.date, d.count)).collect();
// OG image canvas: 1200x630
let og_w = 1200_f64;
let og_h = 630_f64;
let padding = 40_f64;
let bg = "#2c3e50";
let year_label_w = 50_f64;
let max_cols = 53;
// Scale cell size to fill available width
let avail_w = og_w - 2.0 * padding - year_label_w;
let step = (avail_w / max_cols as f64).floor();
let gap = (step * 0.17).round();
let cell = step - gap;
let radius = cell / 2.0;
let colors = ["rgba(255,255,255,0.05)", "#0e4429", "#006d32", "#26a641", "#39d353"];
// Build weekly data per year
struct YearRow {
year: i32,
weeks: Vec<(NaiveDate, NaiveDate, i64)>, // start, end, count
}
let start_year = from.year();
let end_year = to.year();
let mut rows: Vec<YearRow> = Vec::new();
for yr in start_year..=end_year {
let year_start = NaiveDate::from_ymd_opt(yr, 1, 1).unwrap();
let year_end = if yr == end_year {
to
} else {
NaiveDate::from_ymd_opt(yr, 12, 31).unwrap()
};
let offset = year_start.weekday().num_days_from_sunday();
let mut cursor = year_start - chrono::Duration::days(offset as i64);
let mut weeks = Vec::new();
while cursor <= year_end {
let week_start = cursor;
let mut week_count = 0i64;
for _ in 0..7 {
week_count += count_map.get(&cursor).copied().unwrap_or(0);
cursor += chrono::Duration::days(1);
}
let week_end = cursor - chrono::Duration::days(1);
weeks.push((week_start, week_end, week_count));
}
rows.push(YearRow { year: yr, weeks });
}
// Quantile thresholds
let mut non_zero: Vec<i64> = rows
.iter()
.flat_map(|r| r.weeks.iter().map(|w| w.2))
.filter(|&c| c > 0)
.collect();
non_zero.sort();
let thresholds = if non_zero.is_empty() {
[1i64, 2, 3]
} else {
let p = |pct: f64| non_zero[(pct * non_zero.len() as f64).min(non_zero.len() as f64 - 1.0) as usize];
[p(0.25), p(0.5), p(0.75)]
};
let color_for = |count: i64| -> &str {
if count == 0 { colors[0] }
else if count <= thresholds[0] { colors[1] }
else if count <= thresholds[1] { colors[2] }
else if count <= thresholds[2] { colors[3] }
else { colors[4] }
};
let n_rows = rows.len();
let graph_h = (n_rows as f64) * step;
let total: i64 = counts.iter().map(|d| d.count).sum();
let repo_text = if repo_count > 0 {
format!(" in {repo_count} repositories")
} else {
String::new()
};
// Layout: headline at top, graph vertically centered in remaining space
let offset_x = padding;
let headline_y = padding + 36.0;
let subtitle_y = headline_y + 28.0;
let graph_top = subtitle_y + 16.0;
let avail_graph_h = og_h - graph_top - padding;
let graph_y = graph_top + (avail_graph_h - graph_h).max(0.0) / 2.0;
let mut svg = format!(
r#"<svg xmlns="http://www.w3.org/2000/svg" width="{og_w}" height="{og_h}" viewBox="0 0 {og_w} {og_h}"><rect width="100%" height="100%" fill="{bg}"/>"#,
);
// Headline
svg.push_str(&format!(
r##"<text x="{x}" y="{y}" fill="#ecf0f1" font-family="sans-serif" font-size="36" font-weight="bold">rob thijssen</text>"##,
x = offset_x + year_label_w,
y = headline_y,
));
// Subtitle
svg.push_str(&format!(
r##"<text x="{x}" y="{y}" fill="#ecf0f1" font-family="sans-serif" font-size="16" opacity="0.6">{total} contributions since {from}{repo_text}</text>"##,
x = offset_x + year_label_w,
y = subtitle_y,
));
let label_font_size = (step * 0.7).round().max(8.0).min(14.0);
for (row_idx, row) in rows.iter().enumerate() {
let y_base = graph_y + (row_idx as f64) * step;
svg.push_str(&format!(
r##"<text x="{x}" y="{y}" text-anchor="end" dominant-baseline="central" fill="#ecf0f1" font-family="sans-serif" font-size="{fs}" opacity="0.6">{yr}</text>"##,
x = offset_x + year_label_w - 6.0,
y = y_base + radius,
fs = label_font_size,
yr = row.year,
));
for (col, (_, _, count)) in row.weeks.iter().enumerate() {
let cx = offset_x + year_label_w + (col as f64) * step + radius;
let cy = y_base + radius;
let fill = color_for(*count);
svg.push_str(&format!(
r#"<circle cx="{cx}" cy="{cy}" r="{r}" fill="{fill}"/>"#,
r = radius - 1.0,
));
}
}
svg.push_str("</svg>");
// Rasterize at 1200x630
let mut fontdb = fontdb::Database::new();
fontdb.load_system_fonts();
let mut opts = resvg::usvg::Options::default();
opts.fontdb = std::sync::Arc::new(fontdb);
opts.font_family = "Noto Sans".to_owned();
let tree = resvg::usvg::Tree::from_str(&svg, &opts)
.map_err(|e| format!("svg parse: {e}"))?;
let mut pixmap =
resvg::tiny_skia::Pixmap::new(og_w as u32, og_h as u32).ok_or_else(|| "pixmap alloc failed".to_string())?;
resvg::render(&tree, resvg::tiny_skia::Transform::default(), &mut pixmap.as_mut());
pixmap
.encode_png()
.map_err(|e| format!("png encode: {e}"))
}
/// Allowlisted forge hosts that the proxy may contact.
const ALLOWED_HOSTS: &[&str] = &["api.github.com", "git.lair.cafe"];
#[derive(Debug, Deserialize)]
struct ForgeProxyParams {
host: Option<String>,
}
/// Proxy requests to forge APIs to avoid CORS issues.
/// `GET /v1/forge/{source}/{path}?host=git.lair.cafe`
async fn forge_proxy(
State(state): State<AppState>,
Path((source, rest)): Path<(String, String)>,
Query(params): Query<ForgeProxyParams>,
) -> Result<impl IntoResponse, ApiError> {
let (base, api_prefix) = match source.as_str() {
"github" => ("https://api.github.com".to_string(), ""),
"gitea" => {
let host = params.host.as_deref().unwrap_or("git.lair.cafe");
if !ALLOWED_HOSTS.contains(&host) {
return Err(ApiError::bad_request(format!("host not allowed: {host}")));
}
(format!("https://{host}"), "/api/v1")
}
_ => return Err(ApiError::bad_request(format!("unsupported source: {source}"))),
};
let url = format!("{base}{api_prefix}/{rest}");
let resp = state
.http
.get(&url)
.header("Accept", "application/json")
.header("User-Agent", "moments-api")
.send()
.await
.map_err(|e| {
tracing::warn!(url = %url, error = %e, "forge proxy request failed");
ApiError {
status: StatusCode::BAD_GATEWAY,
message: e.to_string(),
}
})?;
let status = StatusCode::from_u16(resp.status().as_u16()).unwrap_or(StatusCode::BAD_GATEWAY);
let body = resp.bytes().await.map_err(|e| ApiError {
status: StatusCode::BAD_GATEWAY,
message: e.to_string(),
})?;
Ok((
status,
[(axum::http::header::CONTENT_TYPE, "application/json")],
body,
))
}
fn parse_sources(raw: &str) -> Result<Vec<Source>, ApiError> { fn parse_sources(raw: &str) -> Result<Vec<Source>, ApiError> {
raw.split(',') raw.split(',')
.map(str::trim) .map(str::trim)

9
crates/moments-core/src/lib.rs
View File

@@ -5,7 +5,8 @@ pub use presentation::reshape;
pub use sources::{EventSource, PollerState, PollerStateStore, SourceError, run_poller}; pub use sources::{EventSource, PollerState, PollerStateStore, SourceError, run_poller};
use async_trait::async_trait; use async_trait::async_trait;
use moments_entities::{Event, EventQuery, SourceSummary}; use chrono::NaiveDate;
use moments_entities::{DailyCount, Event, EventQuery, HourlyAvg, LanguageDailyCount, ProjectSummary, RepoLanguage, SourceSummary};
#[derive(Debug, thiserror::Error)] #[derive(Debug, thiserror::Error)]
pub enum StoreError { pub enum StoreError {
@@ -18,10 +19,16 @@ pub enum StoreError {
pub trait EventReader: Send + Sync { pub trait EventReader: Send + Sync {
async fn list_events(&self, query: &EventQuery) -> Result<Vec<Event>, StoreError>; async fn list_events(&self, query: &EventQuery) -> Result<Vec<Event>, StoreError>;
async fn source_summaries(&self, include_private: bool) -> Result<Vec<SourceSummary>, StoreError>; async fn source_summaries(&self, include_private: bool) -> Result<Vec<SourceSummary>, StoreError>;
async fn list_projects(&self) -> Result<Vec<ProjectSummary>, StoreError>;
async fn daily_counts(&self, from: NaiveDate, to: NaiveDate, include_private: bool) -> Result<Vec<DailyCount>, StoreError>;
async fn language_daily_counts(&self, from: NaiveDate, to: NaiveDate, include_private: bool) -> Result<Vec<LanguageDailyCount>, StoreError>;
async fn hourly_avgs(&self, from: NaiveDate, to: NaiveDate, tz: &str, include_private: bool) -> Result<Vec<HourlyAvg>, StoreError>;
async fn repo_languages(&self) -> Result<Vec<RepoLanguage>, StoreError>;
} }
/// Write-side port consumed by `moments-worker`. Idempotent upserts on `id`. /// Write-side port consumed by `moments-worker`. Idempotent upserts on `id`.
#[async_trait] #[async_trait]
pub trait EventWriter: Send + Sync { pub trait EventWriter: Send + Sync {
async fn upsert_events(&self, events: &[Event]) -> Result<usize, StoreError>; async fn upsert_events(&self, events: &[Event]) -> Result<usize, StoreError>;
async fn upsert_repo_languages(&self, languages: &[RepoLanguage]) -> Result<usize, StoreError>;
} }

1
crates/moments-core/src/presentation/github.rs
View File

@@ -480,6 +480,7 @@ fn commit_reshape(event: &Event) -> TimelineItem {
.get("repository") .get("repository")
.and_then(|r| r.get("full_name")) .and_then(|r| r.get("full_name"))
.and_then(Value::as_str) .and_then(Value::as_str)
.or_else(|| p.get("_repo").and_then(Value::as_str))
.unwrap_or("(unknown repo)"); .unwrap_or("(unknown repo)");
let author_login = p let author_login = p
.get("author") .get("author")

1
crates/moments-data/Cargo.toml
View File

@@ -17,3 +17,4 @@ tracing.workspace = true
async-trait.workspace = true async-trait.workspace = true
reqwest.workspace = true reqwest.workspace = true
serde.workspace = true serde.workspace = true
percent-encoding = "2"

9
crates/moments-data/migrations/0004_repo_languages.sql Normal file
View File

@@ -0,0 +1,9 @@
CREATE TABLE repo_languages (
source TEXT NOT NULL,
repo TEXT NOT NULL,
language TEXT NOT NULL,
bytes BIGINT NOT NULL,
color TEXT,
fetched_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (source, repo, language)
);

55
crates/moments-data/migrations/0005_dedup_gitea_events.sql Normal file
View File

@@ -0,0 +1,55 @@
-- Collapse duplicate Gitea events introduced by polling both the user
-- activity feed and per-org activity feeds.
--
-- Gitea writes one Action row per interested user-context: a push to
-- helexa/cortex by user grenade produces two rows, one with
-- user_id=grenade and one with user_id=helexa. Everything else (op_type,
-- act_user_id, repo_id, ref_name, comment_id, created) is identical.
-- Our prior id scheme (gitea:{action_row_id}) gave them different ids,
-- so the upsert-on-id dedup never fired and the timeline rendered each
-- push twice.
--
-- This migration re-keys every existing gitea row to the same canonical
-- formula `parse_gitea_event` now emits, deleting duplicates encountered
-- along the way. Idempotent: running it again is a no-op because the
-- canonical id of a canonical id is itself.
-- Snapshot the canonical id for every gitea row.
CREATE TEMP TABLE _gitea_canonical AS
SELECT
id AS old_id,
'gitea:'
|| coalesce(payload->>'op_type', '') || ':'
|| coalesce(payload->>'act_user_id', payload->'act_user'->>'id', '0') || ':'
|| coalesce(payload->>'repo_id', payload->'repo'->>'id', '0') || ':'
|| coalesce(payload->>'ref_name', '') || ':'
|| coalesce(payload->>'comment_id', '0') || ':'
|| coalesce(payload->>'created', '')
AS new_id
FROM events
WHERE source = 'gitea';
-- For each canonical id, keep the row whose current id is lexicographically
-- smallest (stable, arbitrary tie-break) and delete the rest. The "old id
-- already matches the new id" case lands here too — DELETE skips it because
-- rn = 1 for any singleton group.
DELETE FROM events
WHERE id IN (
SELECT old_id FROM (
SELECT old_id, new_id,
row_number() OVER (PARTITION BY new_id ORDER BY old_id) AS rn
FROM _gitea_canonical
) ranked
WHERE rn > 1
);
-- Rename remaining rows to the canonical id. Postgres defers PK uniqueness
-- to statement end, so swapping ids across rows in one UPDATE is fine
-- provided the final set is unique (dedup above guarantees that).
UPDATE events e
SET id = c.new_id
FROM _gitea_canonical c
WHERE e.id = c.old_id
AND e.id <> c.new_id;
DROP TABLE _gitea_canonical;

270
crates/moments-data/src/gitea.rs
View File

@@ -9,12 +9,13 @@
//! Each item carries a self-contained payload — including the event-emitting //! Each item carries a self-contained payload — including the event-emitting
//! host — so the reshape layer can construct URLs without needing config. //! host — so the reshape layer can construct URLs without needing config.
use std::collections::HashSet;
use std::sync::Arc; use std::sync::Arc;
use async_trait::async_trait; use async_trait::async_trait;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use moments_core::{EventSource, EventWriter, PollerStateStore, SourceError}; use moments_core::{EventSource, EventWriter, PollerStateStore, SourceError};
use moments_entities::{Event, Source}; use moments_entities::{Event, RepoLanguage, Source};
use reqwest::{Client, header}; use reqwest::{Client, header};
use serde_json::Value; use serde_json::Value;
use tracing::debug; use tracing::debug;
@@ -71,10 +72,17 @@ impl GiteaSource {
} }
} }
fn page_url(&self, page: u32) -> String { fn user_feed_base_url(&self) -> String {
format!( format!(
"https://{}/api/v1/users/{}/activities/feeds?only-performed-by=true&limit={}&page={}", "https://{}/api/v1/users/{}/activities/feeds?only-performed-by=true&limit={}",
self.config.host, self.config.user, self.config.per_page, page self.config.host, self.config.user, self.config.per_page
)
}
fn org_feed_base_url(&self, org: &str) -> String {
format!(
"https://{}/api/v1/orgs/{}/activities/feeds?limit={}",
self.config.host, org, self.config.per_page
) )
} }
@@ -87,22 +95,53 @@ impl GiteaSource {
} }
req req
} }
}
#[async_trait] /// Discover organizations the authenticated user belongs to.
impl EventSource for GiteaSource { /// Returns an empty vec if no token is configured or the request fails.
fn name(&self) -> &'static str { async fn discover_orgs(&self) -> Result<Vec<String>, SourceError> {
SOURCE_NAME if self.config.token.is_none() {
return Ok(vec![]);
}
let url = format!("https://{}/api/v1/user/orgs", self.config.host);
let req = self.apply_headers(self.client.get(&url));
let resp = req
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
tracing::warn!(status = %resp.status(), "failed to discover gitea orgs");
return Ok(vec![]);
}
let orgs: Vec<Value> = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
Ok(orgs
.iter()
.filter_map(|o| o.get("username").and_then(Value::as_str).map(String::from))
.collect())
} }
async fn poll(&self) -> Result<usize, SourceError> { /// Poll a single activity feed, paginating on first run. When `filter_user`
let prior = self.state.load(SOURCE_NAME).await?; /// is true, only events performed by `self.config.user` are ingested (used
/// for org feeds which contain all members' activity).
///
/// `base_url` should contain everything except the `&page=N` suffix.
/// Returns (ingested_count, set_of_repo_full_names).
async fn poll_feed(
&self,
state_key: &str,
base_url: &str,
filter_user: bool,
) -> Result<(usize, HashSet<String>), SourceError> {
let prior = self.state.load(state_key).await?;
let first_run = prior.is_none(); let first_run = prior.is_none();
let max_pages = if first_run { MAX_BACKFILL_PAGES } else { 1 }; let max_pages = if first_run { MAX_BACKFILL_PAGES } else { 1 };
let mut total = 0usize; let mut total = 0usize;
let mut repos = HashSet::new();
for page in 1..=max_pages { for page in 1..=max_pages {
let url = self.page_url(page); let url = format!("{base_url}&page={page}");
let req = self.apply_headers(self.client.get(&url)); let req = self.apply_headers(self.client.get(&url));
let resp = req let resp = req
.send() .send()
@@ -119,8 +158,29 @@ impl EventSource for GiteaSource {
break; break;
} }
// Collect repo names from feed items
for item in &items {
if let Some(name) = item
.get("repo")
.and_then(|r| r.get("full_name"))
.and_then(Value::as_str)
{
repos.insert(name.to_string());
}
}
let events: Vec<Event> = items let events: Vec<Event> = items
.iter() .iter()
.filter(|it| {
if !filter_user {
return true;
}
it.get("act_user")
.and_then(|u| u.get("login"))
.and_then(Value::as_str)
.map(|login| login.eq_ignore_ascii_case(&self.config.user))
.unwrap_or(false)
})
.filter_map(|it| parse_gitea_event(it, &self.config.host)) .filter_map(|it| parse_gitea_event(it, &self.config.host))
.collect(); .collect();
total += self.writer.upsert_events(&events).await?; total += self.writer.upsert_events(&events).await?;
@@ -130,8 +190,85 @@ impl EventSource for GiteaSource {
} }
} }
self.state.touch(SOURCE_NAME).await?; self.state.touch(state_key).await?;
debug!(ingested = total, "gitea poll complete"); Ok((total, repos))
}
/// Fetch language breakdowns for the given repos via the Gitea REST API.
async fn fetch_languages(&self, repos: &HashSet<String>) -> Result<usize, SourceError> {
let mut total = 0usize;
for repo in repos {
let url = format!(
"https://{}/api/v1/repos/{}/languages",
self.config.host, repo
);
let req = self.apply_headers(self.client.get(&url));
let resp = req
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
tracing::warn!(repo = %repo, status = %resp.status(), "gitea language fetch failed; skipping");
continue;
}
let lang_map: std::collections::HashMap<String, i64> = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
let languages: Vec<RepoLanguage> = lang_map
.into_iter()
.map(|(language, bytes)| RepoLanguage {
source: Source::Gitea,
repo: repo.clone(),
language,
bytes,
color: None, // Gitea doesn't return colors
})
.collect();
total += self.writer.upsert_repo_languages(&languages).await?;
}
debug!(total, repos = repos.len(), "gitea repo languages updated");
Ok(total)
}
}
#[async_trait]
impl EventSource for GiteaSource {
fn name(&self) -> &'static str {
SOURCE_NAME
}
async fn poll(&self) -> Result<usize, SourceError> {
let mut all_repos = HashSet::new();
// Poll user's own activity feed (existing behavior).
let user_url = self.user_feed_base_url();
let (mut total, repos) = self.poll_feed(SOURCE_NAME, &user_url, false).await?;
all_repos.extend(repos);
// Discover orgs and poll each org's activity feed, filtering for
// events performed by this user.
let orgs = self.discover_orgs().await?;
for org in &orgs {
let state_key = format!("gitea:org:{org}");
let org_url = self.org_feed_base_url(org);
match self.poll_feed(&state_key, &org_url, true).await {
Ok((n, repos)) => {
total += n;
all_repos.extend(repos);
}
Err(e) => {
tracing::warn!(org = %org, error = %e, "failed to poll org feed");
}
}
}
if let Err(e) = self.fetch_languages(&all_repos).await {
tracing::warn!(error = %e, "gitea language fetch failed; continuing");
}
debug!(ingested = total, orgs = orgs.len(), "gitea poll complete");
Ok(total) Ok(total)
} }
} }
@@ -139,8 +276,14 @@ impl EventSource for GiteaSource {
/// Convert a Gitea activity feed item into our Event row. The host gets /// Convert a Gitea activity feed item into our Event row. The host gets
/// stamped into the payload as `_host` so the reshape layer can build /// stamped into the payload as `_host` so the reshape layer can build
/// web URLs without needing global config. /// web URLs without needing global config.
///
/// The id is content-derived rather than using Gitea's `id` field directly:
/// Gitea creates one Action row per interested user-context, so a push to
/// an org repo by user U produces two rows (one under U's context, one
/// under the org's), distinguished only by `id` and `user_id`. Keying on
/// `(op_type, act_user_id, repo_id, ref_name, comment_id, created)` makes
/// those two rows collapse to the same event on upsert.
fn parse_gitea_event(item: &Value, host: &str) -> Option<Event> { fn parse_gitea_event(item: &Value, host: &str) -> Option<Event> {
let id = item.get("id").and_then(Value::as_i64)?;
let op_type = item.get("op_type").and_then(Value::as_str)?.to_string(); let op_type = item.get("op_type").and_then(Value::as_str)?.to_string();
let created_str = item.get("created").and_then(Value::as_str)?; let created_str = item.get("created").and_then(Value::as_str)?;
let occurred_at = DateTime::parse_from_rfc3339(created_str) let occurred_at = DateTime::parse_from_rfc3339(created_str)
@@ -148,13 +291,15 @@ fn parse_gitea_event(item: &Value, host: &str) -> Option<Event> {
.with_timezone(&Utc); .with_timezone(&Utc);
let private = item.get("is_private").and_then(Value::as_bool).unwrap_or(false); let private = item.get("is_private").and_then(Value::as_bool).unwrap_or(false);
let id = gitea_canonical_id(item, &op_type, created_str);
let mut payload = item.clone(); let mut payload = item.clone();
if let Some(obj) = payload.as_object_mut() { if let Some(obj) = payload.as_object_mut() {
obj.insert("_host".into(), Value::String(host.into())); obj.insert("_host".into(), Value::String(host.into()));
} }
Some(Event { Some(Event {
id: format!("gitea:{id}"), id,
source: Source::Gitea, source: Source::Gitea,
action: op_type, action: op_type,
occurred_at, occurred_at,
@@ -163,6 +308,25 @@ fn parse_gitea_event(item: &Value, host: &str) -> Option<Event> {
}) })
} }
/// Build the canonical, content-derived id for a Gitea action. Must stay
/// in lockstep with the SQL formula in migration 0005 so back-fill and
/// new writes share the same id space.
fn gitea_canonical_id(item: &Value, op_type: &str, created: &str) -> String {
let act_user_id = item
.get("act_user_id")
.and_then(Value::as_i64)
.or_else(|| item.get("act_user").and_then(|u| u.get("id")).and_then(Value::as_i64))
.unwrap_or(0);
let repo_id = item
.get("repo_id")
.and_then(Value::as_i64)
.or_else(|| item.get("repo").and_then(|r| r.get("id")).and_then(Value::as_i64))
.unwrap_or(0);
let ref_name = item.get("ref_name").and_then(Value::as_str).unwrap_or("");
let comment_id = item.get("comment_id").and_then(Value::as_i64).unwrap_or(0);
format!("gitea:{op_type}:{act_user_id}:{repo_id}:{ref_name}:{comment_id}:{created}")
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -173,14 +337,16 @@ mod tests {
let raw = json!({ let raw = json!({
"id": 973, "id": 973,
"op_type": "commit_repo", "op_type": "commit_repo",
"act_user_id": 42,
"repo_id": 7,
"ref_name": "refs/heads/main", "ref_name": "refs/heads/main",
"is_private": false, "is_private": false,
"content": "{\"Commits\":[{\"Sha1\":\"abc123\"}],\"Len\":1}", "content": "{\"Commits\":[{\"Sha1\":\"abc123\"}],\"Len\":1}",
"created": "2026-05-03T16:37:45Z", "created": "2026-05-03T16:37:45Z",
"repo": { "full_name": "grenade/moments" } "repo": { "id": 7, "full_name": "grenade/moments" }
}); });
let ev = parse_gitea_event(&raw, "git.lair.cafe").expect("parses"); let ev = parse_gitea_event(&raw, "git.lair.cafe").expect("parses");
assert_eq!(ev.id, "gitea:973"); assert_eq!(ev.id, "gitea:commit_repo:42:7:refs/heads/main:0:2026-05-03T16:37:45Z");
assert_eq!(ev.source, Source::Gitea); assert_eq!(ev.source, Source::Gitea);
assert_eq!(ev.action, "commit_repo"); assert_eq!(ev.action, "commit_repo");
assert!(ev.public); assert!(ev.public);
@@ -191,6 +357,74 @@ mod tests {
); );
} }
#[test]
fn dup_action_rows_for_user_and_org_contexts_collapse_to_same_id() {
// Gitea creates two Action rows when grenade pushes to helexa/cortex:
// one with user_id=grenade (surfaced by the user feed), one with
// user_id=helexa (surfaced by the org feed). Everything except `id`
// and `user_id` is identical. The canonical id ignores both.
let user_ctx = json!({
"id": 1322,
"user_id": 42,
"op_type": "commit_repo",
"act_user_id": 42,
"act_user": { "login": "grenade", "id": 42 },
"repo_id": 99,
"repo": { "id": 99, "full_name": "helexa/cortex" },
"ref_name": "refs/heads/main",
"comment_id": 0,
"is_private": false,
"created": "2026-05-20T04:32:50Z"
});
let org_ctx = json!({
"id": 1323,
"user_id": 7,
"op_type": "commit_repo",
"act_user_id": 42,
"act_user": { "login": "grenade", "id": 42 },
"repo_id": 99,
"repo": { "id": 99, "full_name": "helexa/cortex" },
"ref_name": "refs/heads/main",
"comment_id": 0,
"is_private": false,
"created": "2026-05-20T04:32:50Z"
});
let a = parse_gitea_event(&user_ctx, "git.lair.cafe").expect("parses");
let b = parse_gitea_event(&org_ctx, "git.lair.cafe").expect("parses");
assert_eq!(a.id, b.id, "duplicate action rows must collide on id");
}
#[test]
fn org_event_user_filter_predicate() {
let by_user = json!({
"id": 500, "op_type": "commit_repo", "is_private": false,
"created": "2026-05-03T10:00:00Z",
"act_user": { "login": "grenade" },
"repo": { "full_name": "myorg/somerepo" }
});
let by_other = json!({
"id": 501, "op_type": "commit_repo", "is_private": false,
"created": "2026-05-03T10:01:00Z",
"act_user": { "login": "otherperson" },
"repo": { "full_name": "myorg/somerepo" }
});
// Both parse as valid events
assert!(parse_gitea_event(&by_user, "git.lair.cafe").is_some());
assert!(parse_gitea_event(&by_other, "git.lair.cafe").is_some());
// The user-filter predicate used by poll_feed
let is_user = |item: &Value, user: &str| -> bool {
item.get("act_user")
.and_then(|u| u.get("login"))
.and_then(Value::as_str)
.map(|login| login.eq_ignore_ascii_case(user))
.unwrap_or(false)
};
assert!(is_user(&by_user, "grenade"));
assert!(!is_user(&by_other, "grenade"));
// Case-insensitive match
assert!(is_user(&by_user, "Grenade"));
}
#[test] #[test]
fn private_event_marked_private() { fn private_event_marked_private() {
let raw = json!({ let raw = json!({

808
crates/moments-data/src/github_repo.rs Normal file
View File

@@ -0,0 +1,808 @@
//! Per-repo commit enumeration for full GitHub history.
//!
//! Discovers repos via two sources:
//! 1. REST `/user/repos` — repos where the user is owner, collaborator,
//! or org member.
//! 2. GraphQL `repositoriesContributedTo` — repos the user has committed
//! to, opened issues/PRs on, or reviewed, even without collaborator
//! status. No result cap (cursor-paginated).
//!
//! Then walks each branch's commit history via
//! `/repos/{owner}/{repo}/commits?author={user}&sha={branch}` with a
//! per-branch `since` cursor to avoid re-fetching known commits. Walking
//! every branch (not just the default) is what catches work-in-progress
//! on feature branches and pushes to fork branches that never get merged
//! upstream — neither the user events feed nor /search/commits surface
//! those reliably.
//!
//! Events use `github-commit:{sha}` as their ID, matching the scheme in
//! `github_search`, so duplicates are resolved via idempotent upsert
//! (the same commit reached via two branches just upserts twice).
use std::collections::HashSet;
use std::sync::Arc;
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use moments_core::{EventSource, EventWriter, PollerStateStore, SourceError};
use moments_entities::{Event, RepoLanguage, Source};
use percent_encoding::{AsciiSet, CONTROLS, utf8_percent_encode};
use reqwest::{Client, header};
use serde_json::Value;
use tracing::{debug, warn};
/// Encode characters that have meaning in a URL query — branch names can
/// contain `/`, `#`, `?`, etc. Whitelisting is too fragile; encode anything
/// outside the unreserved set plus a few safe characters.
const BRANCH_ENCODE_SET: &AsciiSet = &CONTROLS
.add(b' ')
.add(b'"')
.add(b'#')
.add(b'<')
.add(b'>')
.add(b'?')
.add(b'`')
.add(b'{')
.add(b'}')
.add(b'/')
.add(b'&')
.add(b'=')
.add(b'+')
.add(b'%');
const SOURCE_NAME: &str = "github-repo";
const USER_AGENT: &str = concat!(
"moments/",
env!("CARGO_PKG_VERSION"),
" (+https://rob.tn)"
);
const MAX_BACKFILL_PAGES: u32 = 100;
#[derive(Clone, Debug)]
pub struct GithubRepoConfig {
pub user: String,
pub token: Option<String>,
pub per_page: u32,
}
impl Default for GithubRepoConfig {
fn default() -> Self {
Self {
user: "grenade".into(),
token: None,
per_page: 100,
}
}
}
pub struct GithubRepoSource {
client: Client,
writer: Arc<dyn EventWriter>,
state: Arc<dyn PollerStateStore>,
config: GithubRepoConfig,
}
impl GithubRepoSource {
pub fn new(
client: Client,
writer: Arc<dyn EventWriter>,
state: Arc<dyn PollerStateStore>,
config: GithubRepoConfig,
) -> Self {
Self {
client,
writer,
state,
config,
}
}
fn apply_headers(&self, mut req: reqwest::RequestBuilder) -> reqwest::RequestBuilder {
req = req
.header(header::ACCEPT, "application/vnd.github+json")
.header("X-GitHub-Api-Version", "2022-11-28")
.header(header::USER_AGENT, USER_AGENT);
if let Some(token) = &self.config.token {
req = req.header(header::AUTHORIZATION, format!("Bearer {token}"));
}
req
}
/// Discover all repos the authenticated user can access.
async fn discover_repos(&self) -> Result<Vec<Repo>, SourceError> {
if self.config.token.is_none() {
return Ok(vec![]);
}
let mut repos = Vec::new();
for page in 1..=50 {
let url = format!(
"https://api.github.com/user/repos?affiliation=owner,collaborator,organization_member&visibility=all&per_page={}&page={}",
self.config.per_page, page
);
let req = self.apply_headers(self.client.get(&url));
let resp = req
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
return Err(SourceError::Http(format!("{} GET {}", resp.status(), url)));
}
let items: Vec<Value> = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
if items.is_empty() {
break;
}
for item in &items {
if let Some(r) = parse_repo(item) {
repos.push(r);
}
}
if items.len() < self.config.per_page as usize {
break;
}
}
// Supplement with repos from GraphQL repositoriesContributedTo.
// This catches repos where the user contributed via PRs but isn't
// an owner, collaborator, or org member — no result cap.
let mut known: HashSet<String> = repos.iter().map(|r| r.full_name.clone()).collect();
let contributed = self.discover_contributed_repos().await;
match contributed {
Ok(extra) => {
for r in extra {
if known.insert(r.full_name.clone()) {
repos.push(r);
}
}
}
Err(e) => {
warn!(error = %e, "GraphQL contributed-repos discovery failed; continuing with known repos");
}
}
Ok(repos)
}
/// Discover repos the user has contributed to via GraphQL.
/// Uses cursor-based pagination with no result cap.
async fn discover_contributed_repos(&self) -> Result<Vec<Repo>, SourceError> {
let token = match &self.config.token {
Some(t) => t,
None => return Ok(vec![]),
};
let mut repos = Vec::new();
let mut cursor: Option<String> = None;
loop {
let after = match &cursor {
Some(c) => format!(", after: \"{}\"", c),
None => String::new(),
};
let query = format!(
r#"{{ user(login: "{}") {{ repositoriesContributedTo(first: 100, contributionTypes: [COMMIT, PULL_REQUEST, ISSUE]{}) {{ pageInfo {{ hasNextPage endCursor }} nodes {{ nameWithOwner isPrivate }} }} }} }}"#,
self.config.user, after
);
let body = serde_json::json!({ "query": query });
let resp = self
.client
.post("https://api.github.com/graphql")
.header(header::AUTHORIZATION, format!("Bearer {token}"))
.header(header::USER_AGENT, USER_AGENT)
.header(header::CONTENT_TYPE, "application/json")
.json(&body)
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
return Err(SourceError::Http(format!(
"{} POST graphql",
resp.status()
)));
}
let data: Value = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
// Check for GraphQL-level errors
if let Some(errors) = data.get("errors").and_then(Value::as_array) {
if let Some(msg) = errors.first().and_then(|e| e.get("message")).and_then(Value::as_str) {
return Err(SourceError::Http(format!("GraphQL error: {msg}")));
}
}
let contributed = &data["data"]["user"]["repositoriesContributedTo"];
let nodes = contributed["nodes"].as_array();
if let Some(nodes) = nodes {
for node in nodes {
let full_name = node
.get("nameWithOwner")
.and_then(Value::as_str);
let private = node
.get("isPrivate")
.and_then(Value::as_bool)
.unwrap_or(false);
if let Some(name) = full_name {
repos.push(Repo {
full_name: name.to_string(),
private,
});
}
}
}
let has_next = contributed["pageInfo"]["hasNextPage"]
.as_bool()
.unwrap_or(false);
if !has_next {
break;
}
cursor = contributed["pageInfo"]["endCursor"]
.as_str()
.map(String::from);
}
debug!(repos = repos.len(), "discovered contributed repos via GraphQL");
Ok(repos)
}
/// Branch discovery via GraphQL, filtered to branches whose HEAD
/// commit was authored by the user. Skips the long tail of
/// upstream-contributor branches in large forks (e.g. azure-docs).
///
/// Why HEAD author and not `history(author:).totalCount`: the latter
/// forces GraphQL to walk full commit history per branch looking for
/// matches, which times out (502) on forks with thousands of branches.
/// Checking the HEAD commit's author is O(1) per branch. The blind
/// spot — branches with the user's older commits but a different
/// HEAD author — is rare in practice for forks/feature branches.
///
/// On any GraphQL failure, callers should fall back to `list_branches`
/// (REST, walks everything; 500s from empty branches are silenced
/// inside `scan_repo_branch`).
async fn list_branches_with_commits(
&self,
repo: &Repo,
user_login: &str,
) -> Result<Vec<String>, SourceError> {
let token = match &self.config.token {
Some(t) => t,
None => return Err(SourceError::Http("no token; graphql unavailable".into())),
};
let parts: Vec<&str> = repo.full_name.splitn(2, '/').collect();
if parts.len() != 2 {
return Ok(Vec::new());
}
let (owner, name) = (parts[0], parts[1]);
let mut branches = Vec::new();
let mut cursor: Option<String> = None;
// Cap pages to bound cost on pathological repos. 50 pages × 100
// branches = 5000; well past anything plausible for a human user.
for _ in 0..50u32 {
let after = match &cursor {
Some(c) => format!(", after: \"{}\"", c),
None => String::new(),
};
// `author.user.login` resolves the commit's GitHub user (may
// differ from the raw commit author name); falling back to
// `author.email` is intentionally omitted to keep the query
// shape minimal — false negatives there are caught by the
// REST fallback on the next poll cycle.
let query = format!(
r#"{{ repository(owner: "{owner}", name: "{name}") {{ refs(refPrefix: "refs/heads/", first: 100{after}) {{ pageInfo {{ hasNextPage endCursor }} nodes {{ name target {{ ... on Commit {{ author {{ user {{ login }} }} }} }} }} }} }} }}"#,
);
let body = serde_json::json!({ "query": query });
let resp = self
.client
.post("https://api.github.com/graphql")
.header(header::AUTHORIZATION, format!("Bearer {token}"))
.header(header::USER_AGENT, USER_AGENT)
.header(header::CONTENT_TYPE, "application/json")
.json(&body)
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
return Err(SourceError::Http(format!(
"{} POST graphql (branches {}/{})",
resp.status(),
owner,
name
)));
}
let data: Value = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
if let Some(errors) = data.get("errors").and_then(Value::as_array) {
if let Some(msg) = errors.first().and_then(|e| e.get("message")).and_then(Value::as_str) {
return Err(SourceError::Http(format!("GraphQL error listing branches: {msg}")));
}
}
let refs = &data["data"]["repository"]["refs"];
if refs.is_null() {
// Repo may be deleted or inaccessible — treat as empty.
return Ok(Vec::new());
}
if let Some(nodes) = refs["nodes"].as_array() {
for node in nodes {
let branch = node["name"].as_str();
let head_login = node["target"]["author"]["user"]["login"].as_str();
if let (Some(b), Some(login)) = (branch, head_login) {
if login.eq_ignore_ascii_case(user_login) {
branches.push(b.to_string());
}
}
}
}
let has_next = refs["pageInfo"]["hasNextPage"].as_bool().unwrap_or(false);
if !has_next {
break;
}
cursor = refs["pageInfo"]["endCursor"].as_str().map(String::from);
}
Ok(branches)
}
/// List every branch in a repo. Returns an empty vec for empty (409)
/// or missing (404) repos; surfaces rate-limit / transport errors so the
/// caller can decide whether to bail.
async fn list_branches(&self, repo: &Repo) -> Result<Vec<String>, SourceError> {
let mut branches = Vec::new();
for page in 1..=10u32 {
let url = format!(
"https://api.github.com/repos/{}/branches?per_page={}&page={}",
repo.full_name, self.config.per_page, page
);
let req = self.apply_headers(self.client.get(&url));
let resp = req
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
let status = resp.status();
if status.as_u16() == 404 || status.as_u16() == 409 {
return Ok(Vec::new());
}
if status.as_u16() == 403 || status.as_u16() == 429 {
return Err(SourceError::Http(format!("{} GET {}", status, url)));
}
if !status.is_success() {
return Err(SourceError::Http(format!("{} GET {}", status, url)));
}
let items: Vec<Value> = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
if items.is_empty() {
break;
}
for item in &items {
if let Some(name) = item.get("name").and_then(Value::as_str) {
branches.push(name.to_string());
}
}
if items.len() < self.config.per_page as usize {
break;
}
}
Ok(branches)
}
/// Fetch commits for a single repo across all branches the user has
/// touched. Per-branch state keys (`github-repo:{full_name}@{branch}`)
/// hold the newest seen commit timestamp so each branch can be
/// incremented independently — important because a brand new branch's
/// `since` cursor must start unset even when the default branch has
/// been polled many times already.
///
/// When `user_id` is supplied, branches are pre-filtered via GraphQL
/// to those with at least one commit by the user — vastly cheaper for
/// large upstream forks where most branches were never touched. On
/// GraphQL failure (or no token), falls back to the REST branch list
/// and relies on the per-branch 500-as-empty handling to discard the
/// noise.
async fn scan_repo(&self, repo: &Repo) -> Result<usize, SourceError> {
let branches = if self.config.token.is_some() {
match self.list_branches_with_commits(repo, &self.config.user).await {
Ok(b) => b,
Err(e) => {
warn!(repo = %repo.full_name, error = %e, "graphql branch filter failed; falling back to REST");
self.list_branches(repo).await?
}
}
} else {
self.list_branches(repo).await?
};
if branches.is_empty() {
return Ok(0);
}
let mut total = 0usize;
// Dedup commits seen via multiple branches in one tick. Without this
// the same SHA appears in the upsert batch twice (postgres rejects
// duplicate conflict targets in a single INSERT).
let mut seen_in_tick: HashSet<String> = HashSet::new();
for branch in &branches {
match self.scan_repo_branch(repo, branch, &mut seen_in_tick).await {
Ok(n) => total += n,
Err(SourceError::Http(ref msg)) if msg.starts_with("403") || msg.starts_with("429") => {
return Err(SourceError::Http(msg.clone()));
}
Err(e) => {
warn!(repo = %repo.full_name, branch = %branch, error = %e, "branch scan failed; continuing");
}
}
}
Ok(total)
}
async fn scan_repo_branch(
&self,
repo: &Repo,
branch: &str,
seen_in_tick: &mut HashSet<String>,
) -> Result<usize, SourceError> {
let state_key = format!("github-repo:{}@{}", repo.full_name, branch);
let prior = self.state.load(&state_key).await?;
let since = prior.as_ref().and_then(|s| s.last_modified);
let encoded_branch = utf8_percent_encode(branch, BRANCH_ENCODE_SET).to_string();
let mut total = 0usize;
let mut newest: Option<DateTime<Utc>> = since;
for page in 1..=MAX_BACKFILL_PAGES {
let mut url = format!(
"https://api.github.com/repos/{}/commits?author={}&sha={}&per_page={}&page={}",
repo.full_name, self.config.user, encoded_branch, self.config.per_page, page
);
if let Some(since_dt) = since {
url.push_str(&format!("&since={}", since_dt.to_rfc3339()));
}
let req = self.apply_headers(self.client.get(&url));
let resp = req
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
let status = resp.status();
// 409 = empty repo (no commits at all), not an error
if status.as_u16() == 409 {
break;
}
if status.as_u16() == 403 || status.as_u16() == 429 {
warn!(repo = %repo.full_name, branch = %branch, status = %status, "rate limited; stopping early");
return Err(SourceError::Http(format!("{} GET {}", status, url)));
}
if status.as_u16() == 404 {
warn!(repo = %repo.full_name, branch = %branch, "repo or branch not found; skipping");
break;
}
// GitHub's `/repos/.../commits?author=X&sha=branch` returns 500
// (not an empty array) when the user has zero commits on the
// specified branch. Treat it as "no commits on this branch"
// rather than a server error — surfacing it as a warning floods
// logs on forks whose branches were all authored by upstream.
if status.as_u16() == 500 {
debug!(repo = %repo.full_name, branch = %branch, "no commits by author on branch (500)");
break;
}
if !status.is_success() {
return Err(SourceError::Http(format!("{} GET {}", status, url)));
}
let items: Vec<Value> = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
if items.is_empty() {
break;
}
let mut events = Vec::with_capacity(items.len());
for item in &items {
if let Some(ev) = parse_commit(item, repo) {
if seen_in_tick.insert(ev.id.clone()) {
if let Some(n) = newest {
if ev.occurred_at > n {
newest = Some(ev.occurred_at);
}
} else {
newest = Some(ev.occurred_at);
}
events.push(ev);
} else {
// Already ingested via another branch this tick;
// still advance `newest` so the per-branch cursor
// doesn't get stuck behind shared history.
let occurred = parse_commit_date(item);
if let Some(t) = occurred {
newest = Some(match newest {
Some(n) if t > n => t,
Some(n) => n,
None => t,
});
}
}
}
}
total += self.writer.upsert_events(&events).await?;
if items.len() < self.config.per_page as usize {
break;
}
}
self.state.save(&state_key, None, newest).await?;
Ok(total)
}
/// Batch-fetch language breakdowns for repos via GraphQL, upserting
/// into repo_languages. Repos are batched using GraphQL aliases to
/// minimise round trips.
async fn fetch_languages(&self, repos: &[Repo]) -> Result<usize, SourceError> {
let token = match &self.config.token {
Some(t) => t,
None => return Ok(0),
};
let mut total = 0usize;
for chunk in repos.chunks(20) {
let mut fragments = Vec::with_capacity(chunk.len());
for (i, repo) in chunk.iter().enumerate() {
let parts: Vec<&str> = repo.full_name.splitn(2, '/').collect();
if parts.len() != 2 {
continue;
}
fragments.push(format!(
r#"r{i}: repository(owner: "{}", name: "{}") {{ languages(first: 20, orderBy: {{field: SIZE, direction: DESC}}) {{ edges {{ size node {{ name color }} }} }} }}"#,
parts[0], parts[1]
));
}
if fragments.is_empty() {
continue;
}
let query = format!("{{ {} }}", fragments.join(" "));
let body = serde_json::json!({ "query": query });
let resp = self
.client
.post("https://api.github.com/graphql")
.header(header::AUTHORIZATION, format!("Bearer {token}"))
.header(header::USER_AGENT, USER_AGENT)
.header(header::CONTENT_TYPE, "application/json")
.json(&body)
.send()
.await
.map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() {
warn!(status = %resp.status(), "GraphQL language fetch failed");
break;
}
let data: Value = resp
.json()
.await
.map_err(|e| SourceError::Parse(e.to_string()))?;
if let Some(errors) = data.get("errors").and_then(Value::as_array) {
if let Some(msg) = errors.first().and_then(|e| e.get("message")).and_then(Value::as_str) {
warn!(error = %msg, "GraphQL language fetch had errors");
}
}
let data_obj = match data.get("data") {
Some(d) => d,
None => continue,
};
let mut languages = Vec::new();
for (i, repo) in chunk.iter().enumerate() {
let alias = format!("r{i}");
let edges = data_obj
.get(&alias)
.and_then(|r| r.get("languages"))
.and_then(|l| l.get("edges"))
.and_then(Value::as_array);
if let Some(edges) = edges {
for edge in edges {
let size = edge.get("size").and_then(Value::as_i64).unwrap_or(0);
let name = edge
.get("node")
.and_then(|n| n.get("name"))
.and_then(Value::as_str);
let color = edge
.get("node")
.and_then(|n| n.get("color"))
.and_then(Value::as_str);
if let Some(name) = name {
languages.push(RepoLanguage {
source: Source::Github,
repo: repo.full_name.clone(),
language: name.to_string(),
bytes: size,
color: color.map(String::from),
});
}
}
}
}
total += self.writer.upsert_repo_languages(&languages).await?;
}
debug!(total, "repo languages updated");
Ok(total)
}
}
#[async_trait]
impl EventSource for GithubRepoSource {
fn name(&self) -> &'static str {
SOURCE_NAME
}
async fn poll(&self) -> Result<usize, SourceError> {
let repos = self.discover_repos().await?;
debug!(repos = repos.len(), "discovered github repos");
let mut total = 0usize;
for repo in &repos {
match self.scan_repo(repo).await {
Ok(n) => {
if n > 0 {
debug!(repo = %repo.full_name, ingested = n, "repo commit scan complete");
}
total += n;
}
Err(SourceError::Http(ref msg)) if msg.starts_with("403") || msg.starts_with("429") => {
warn!("rate limited during repo scan; ending poll early");
break;
}
Err(e) => {
warn!(repo = %repo.full_name, error = %e, "repo scan failed; continuing");
}
}
}
if let Err(e) = self.fetch_languages(&repos).await {
warn!(error = %e, "language fetch failed; continuing");
}
self.state.touch(SOURCE_NAME).await?;
debug!(ingested = total, repos = repos.len(), "github-repo poll complete");
Ok(total)
}
}
#[derive(Debug, Clone)]
struct Repo {
full_name: String,
private: bool,
}
fn parse_repo(item: &Value) -> Option<Repo> {
let full_name = item.get("full_name").and_then(Value::as_str)?;
let private = item.get("private").and_then(Value::as_bool).unwrap_or(false);
Some(Repo {
full_name: full_name.to_string(),
private,
})
}
fn parse_commit_date(item: &Value) -> Option<DateTime<Utc>> {
let date_str = item
.get("commit")
.and_then(|c| c.get("author"))
.and_then(|a| a.get("date"))
.and_then(Value::as_str)
.or_else(|| {
item.get("commit")
.and_then(|c| c.get("committer"))
.and_then(|c| c.get("date"))
.and_then(Value::as_str)
})?;
Some(
DateTime::parse_from_rfc3339(date_str)
.ok()?
.with_timezone(&Utc),
)
}
fn parse_commit(item: &Value, repo: &Repo) -> Option<Event> {
let sha = item.get("sha").and_then(Value::as_str)?;
let occurred_at = parse_commit_date(item)?;
let mut payload = item.clone();
if let Some(obj) = payload.as_object_mut() {
obj.insert("_repo".into(), Value::String(repo.full_name.clone()));
}
Some(Event {
id: format!("github-commit:{sha}"),
source: Source::Github,
action: "Commit".into(),
occurred_at,
public: !repo.private,
payload,
})
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn parse_commit_uses_sha_as_id() {
let repo = Repo {
full_name: "grenade/moments".into(),
private: false,
};
let raw = json!({
"sha": "abc123",
"commit": {
"author": { "date": "2024-01-15T10:30:00Z" },
"message": "fix something"
}
});
let ev = parse_commit(&raw, &repo).expect("parses");
assert_eq!(ev.id, "github-commit:abc123");
assert_eq!(ev.action, "Commit");
assert!(ev.public);
}
#[test]
fn parse_commit_private_repo() {
let repo = Repo {
full_name: "grenade/secret".into(),
private: true,
};
let raw = json!({
"sha": "def456",
"commit": {
"author": { "date": "2024-01-15T10:30:00Z" },
"message": "secret change"
}
});
let ev = parse_commit(&raw, &repo).expect("parses");
assert!(!ev.public);
}
#[test]
fn parse_commit_falls_back_to_committer_date() {
let repo = Repo {
full_name: "grenade/moments".into(),
private: false,
};
let raw = json!({
"sha": "ghi789",
"commit": {
"committer": { "date": "2024-02-01T12:00:00Z" },
"message": "no author date"
}
});
let ev = parse_commit(&raw, &repo).expect("parses");
assert_eq!(ev.id, "github-commit:ghi789");
}
#[test]
fn parse_repo_extracts_fields() {
let raw = json!({
"full_name": "grenade/moments",
"private": false
});
let repo = parse_repo(&raw).expect("parses");
assert_eq!(repo.full_name, "grenade/moments");
assert!(!repo.private);
}
}

5
crates/moments-data/src/github_search.rs
View File

@@ -113,8 +113,11 @@ impl GithubSearchSource {
) -> Result<usize, SourceError> { ) -> Result<usize, SourceError> {
let mut total = 0usize; let mut total = 0usize;
for page in 1..=self.config.max_pages { for page in 1..=self.config.max_pages {
// `fork:true` opts forks into the search — by default GitHub's
// search API excludes them entirely, which means commits on a
// user's fork (regardless of branch) never surface here.
let url = format!( let url = format!(
"https://api.github.com/search/commits?q=author:{}&sort=author-date&order=desc&per_page={}&page={}", "https://api.github.com/search/commits?q=author:{}+fork:true&sort=author-date&order=desc&per_page={}&page={}",
self.config.user, self.config.per_page, page self.config.user, self.config.per_page, page
); );
let req = self.apply_headers(self.client.get(&url)); let req = self.apply_headers(self.client.get(&url));

235
crates/moments-data/src/hg.rs
View File

@@ -1,14 +1,14 @@
//! hg-edge.mozilla.org pushlog ingestion. //! hg-edge.mozilla.org changeset ingestion via `json-log` revset queries.
//! //!
//! mozilla's hg pushlog filter `user=` matches the *pusher*, not the //! Uses the `json-log?rev=author(term)` endpoint which returns changesets
//! changeset author. As a community-level contributor whose code was //! by the *author* (not the pusher), so it captures commits landed by
//! reviewed and pushed by sheriffs/reviewers, the user filter returns 0 //! sheriffs on behalf of the contributor.
//! results. So this source pulls full pushlogs from a configured set of
//! repos and filters changeset authors client-side by substring match.
//! //!
//! The result set is historical (mozilla retired hg) — no new events //! Repos are discovered within configured groups (e.g. `build`) via the
//! expected after the initial backfill. Daily polling keeps the upserts //! `/{group}/?style=json` index, plus any individually listed repos
//! cheap and idempotent. //! (e.g. `mozilla-central`). Once the first successful scan completes
//! (poller state is touched), all subsequent polls are skipped — the
//! data is historical and will not change.
use std::sync::Arc; use std::sync::Arc;
@@ -26,13 +26,17 @@ const USER_AGENT: &str = concat!(
env!("CARGO_PKG_VERSION"), env!("CARGO_PKG_VERSION"),
" (+https://rob.tn)" " (+https://rob.tn)"
); );
/// Maximum changesets returned per json-log request.
const REV_COUNT: u32 = 500;
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct HgConfig { pub struct HgConfig {
pub host: String, pub host: String,
/// Case-insensitive substrings matched against changeset author fields. /// Substrings matched via `author(term)` revset queries.
pub author_terms: Vec<String>, pub author_terms: Vec<String>,
/// Repo paths under host, e.g. "build/puppet". /// Repo groups to scan — each is enumerated via `/{group}/?style=json`.
pub groups: Vec<String>,
/// Individual repos to scan (e.g. `mozilla-central`).
pub repos: Vec<String>, pub repos: Vec<String>,
} }
@@ -40,12 +44,9 @@ impl Default for HgConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
host: "hg-edge.mozilla.org".into(), host: "hg-edge.mozilla.org".into(),
author_terms: vec!["thijssen".into(), "grenade".into()], author_terms: vec!["rthijssen".into(), "grenade".into()],
repos: vec![ groups: vec!["build".into(), "integration".into()],
"build/puppet".into(), repos: vec!["mozilla-central".into()],
"build/tools".into(),
"build/buildbot-configs".into(),
],
} }
} }
} }
@@ -72,23 +73,9 @@ impl HgSource {
} }
} }
fn pushlog_url(&self, repo: &str) -> String { /// Discover repos in a group via `/{group}/?style=json`.
format!( async fn discover_repos(&self, group: &str) -> Result<Vec<String>, SourceError> {
"https://{}/{}/json-pushes?version=2&full=1", let url = format!("https://{}/{}/?style=json", self.config.host, group);
self.config.host, repo
)
}
fn matches_author(&self, author: &str) -> bool {
let lower = author.to_lowercase();
self.config
.author_terms
.iter()
.any(|t| lower.contains(&t.to_lowercase()))
}
async fn scan_repo(&self, repo: &str) -> Result<usize, SourceError> {
let url = self.pushlog_url(repo);
let resp = self let resp = self
.client .client
.get(&url) .get(&url)
@@ -97,69 +84,87 @@ impl HgSource {
.await .await
.map_err(|e| SourceError::Http(e.to_string()))?; .map_err(|e| SourceError::Http(e.to_string()))?;
if !resp.status().is_success() { if !resp.status().is_success() {
return Err(SourceError::Http(format!("{} GET {}", resp.status(), url))); warn!(group, status = %resp.status(), "failed to discover repos in group");
return Ok(vec![]);
} }
let body: Value = resp let body: Value = resp
.json() .json()
.await .await
.map_err(|e| SourceError::Parse(e.to_string()))?; .map_err(|e| SourceError::Parse(e.to_string()))?;
Ok(body
.get("entries")
.and_then(Value::as_array)
.map(|entries| {
entries
.iter()
.filter_map(|e| {
e.get("name")
.and_then(Value::as_str)
.map(|name| format!("{group}/{name}"))
})
.collect()
})
.unwrap_or_default())
}
let mut events = Vec::new(); fn log_url(&self, repo: &str, author_term: &str) -> String {
if let Some(pushes) = body.get("pushes").and_then(Value::as_object) { format!(
for (pushid, push) in pushes { "https://{}/{}/json-log?rev=author({})&style=json&revcount={}",
let pushed_at_secs = push.get("date").and_then(Value::as_i64).unwrap_or(0); self.config.host, repo, author_term, REV_COUNT
let pushed_at = Utc )
.timestamp_opt(pushed_at_secs, 0) }
.single()
.unwrap_or_else(Utc::now);
let pusher = push
.get("user")
.and_then(Value::as_str)
.unwrap_or("")
.to_string();
if let Some(changesets) = push.get("changesets").and_then(Value::as_array) {
for cs in changesets {
let author = cs.get("author").and_then(Value::as_str).unwrap_or("");
if !self.matches_author(author) {
continue;
}
let node = cs.get("node").and_then(Value::as_str).unwrap_or("");
if node.is_empty() {
continue;
}
let occurred_at = cs
.get("date")
.and_then(Value::as_array)
.and_then(|a| parse_hg_date(a))
.unwrap_or(pushed_at);
let mut payload = cs.clone(); async fn scan_repo(&self, repo: &str) -> Result<usize, SourceError> {
if let Some(obj) = payload.as_object_mut() { let mut all_events = Vec::new();
obj.insert("_repo".into(), Value::String(repo.into())); for term in &self.config.author_terms {
obj.insert( let url = self.log_url(repo, term);
"_host".into(), let resp = self
Value::String(self.config.host.clone()), .client
); .get(&url)
obj.insert("_pusher".into(), Value::String(pusher.clone())); .header(header::USER_AGENT, USER_AGENT)
obj.insert( .send()
"_pushid".into(), .await
Value::String(pushid.clone()), .map_err(|e| SourceError::Http(e.to_string()))?;
); if !resp.status().is_success() {
} return Err(SourceError::Http(format!("{} GET {}", resp.status(), url)));
events.push(Event { }
id: format!("hg:{repo}:{node}"), let body: Value = resp
source: Source::Hg, .json()
action: "Commit".into(), .await
occurred_at, .map_err(|e| SourceError::Parse(e.to_string()))?;
// mozilla hg-edge is exclusively public.
public: true, if let Some(entries) = body.get("entries").and_then(Value::as_array) {
payload, for entry in entries {
}); let node = entry.get("node").and_then(Value::as_str).unwrap_or("");
if node.is_empty() {
continue;
} }
let occurred_at = entry
.get("date")
.and_then(Value::as_array)
.and_then(|a| parse_hg_date(a))
.unwrap_or_else(Utc::now);
let mut payload = entry.clone();
if let Some(obj) = payload.as_object_mut() {
obj.insert("_repo".into(), Value::String(repo.into()));
obj.insert(
"_host".into(),
Value::String(self.config.host.clone()),
);
}
all_events.push(Event {
id: format!("hg:{repo}:{node}"),
source: Source::Hg,
action: "Commit".into(),
occurred_at,
public: true,
payload,
});
} }
} }
} }
Ok(self.writer.upsert_events(&events).await?) Ok(self.writer.upsert_events(&all_events).await?)
} }
} }
@@ -170,11 +175,26 @@ impl EventSource for HgSource {
} }
async fn poll(&self) -> Result<usize, SourceError> { async fn poll(&self) -> Result<usize, SourceError> {
// hg repos are archived — one complete scan is sufficient.
if self.state.load(SOURCE_NAME).await?.is_some() {
debug!("hg already backfilled, skipping");
return Ok(0);
}
let mut repos: Vec<String> = self.config.repos.clone();
for group in &self.config.groups {
let discovered = self.discover_repos(group).await?;
debug!(group, repos = discovered.len(), "discovered hg repos");
repos.extend(discovered);
}
let mut total = 0usize; let mut total = 0usize;
for repo in &self.config.repos { for repo in &repos {
match self.scan_repo(repo).await { match self.scan_repo(repo).await {
Ok(n) => { Ok(n) => {
debug!(repo, ingested = n, "hg repo scan complete"); if n > 0 {
debug!(repo, ingested = n, "hg repo scan complete");
}
total += n; total += n;
} }
Err(e) => { Err(e) => {
@@ -183,11 +203,12 @@ impl EventSource for HgSource {
} }
} }
self.state.touch(SOURCE_NAME).await?; self.state.touch(SOURCE_NAME).await?;
debug!(ingested = total, "hg backfill complete");
Ok(total) Ok(total)
} }
} }
/// Parse a pushlog date array `[seconds, tz_offset_secs]` into UTC. /// Parse a hgweb date array `[seconds, tz_offset_secs]` into UTC.
fn parse_hg_date(arr: &[Value]) -> Option<DateTime<Utc>> { fn parse_hg_date(arr: &[Value]) -> Option<DateTime<Utc>> {
let secs = arr.first()?.as_f64()? as i64; let secs = arr.first()?.as_f64()? as i64;
Utc.timestamp_opt(secs, 0).single() Utc.timestamp_opt(secs, 0).single()
@@ -197,23 +218,6 @@ fn parse_hg_date(arr: &[Value]) -> Option<DateTime<Utc>> {
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn matches_author_substring_case_insensitive() {
let s = HgSource {
client: Client::new(),
writer: Arc::new(NoopWriter),
state: Arc::new(NoopState),
config: HgConfig {
author_terms: vec!["thijssen".into(), "grenade".into()],
..Default::default()
},
};
assert!(s.matches_author("Rob Thijssen <rob@example.com>"));
assert!(s.matches_author("grenade@example"));
assert!(s.matches_author("THIJSSEN"));
assert!(!s.matches_author("Other Person <other@example>"));
}
#[test] #[test]
fn parse_hg_date_handles_seconds() { fn parse_hg_date_handles_seconds() {
let arr = vec![Value::from(1_700_000_000_f64), Value::from(0_f64)]; let arr = vec![Value::from(1_700_000_000_f64), Value::from(0_f64)];
@@ -221,6 +225,19 @@ mod tests {
assert_eq!(dt.timestamp(), 1_700_000_000); assert_eq!(dt.timestamp(), 1_700_000_000);
} }
#[test]
fn log_url_uses_revset_author_query() {
let src = HgSource {
client: Client::new(),
writer: Arc::new(NoopWriter),
state: Arc::new(NoopState),
config: HgConfig::default(),
};
let url = src.log_url("mozilla-central", "thijssen");
assert!(url.contains("json-log?rev=author(thijssen)"));
assert!(url.contains("revcount=500"));
}
// Tiny stub impls just so we can construct an HgSource for unit tests. // Tiny stub impls just so we can construct an HgSource for unit tests.
struct NoopWriter; struct NoopWriter;
#[async_trait] #[async_trait]
@@ -231,6 +248,12 @@ mod tests {
) -> Result<usize, moments_core::StoreError> { ) -> Result<usize, moments_core::StoreError> {
Ok(0) Ok(0)
} }
async fn upsert_repo_languages(
&self,
_languages: &[moments_entities::RepoLanguage],
) -> Result<usize, moments_core::StoreError> {
Ok(0)
}
} }
struct NoopState; struct NoopState;
#[async_trait] #[async_trait]

291
crates/moments-data/src/lib.rs
View File

@@ -1,13 +1,15 @@
pub mod bugzilla; pub mod bugzilla;
pub mod gitea; pub mod gitea;
pub mod github; pub mod github;
pub mod github_repo;
pub mod github_search; pub mod github_search;
pub mod hg; pub mod hg;
use async_trait::async_trait; use async_trait::async_trait;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use moments_core::{EventReader, EventWriter, PollerState, PollerStateStore, StoreError}; use moments_core::{EventReader, EventWriter, PollerState, PollerStateStore, StoreError};
use moments_entities::{Event, EventQuery, Source, SourceSummary}; use chrono::NaiveDate;
use moments_entities::{DailyCount, Event, EventQuery, HourlyAvg, LanguageDailyCount, ProjectSummary, RepoLanguage, Source, SourceSummary};
use sqlx::Row; use sqlx::Row;
use sqlx::postgres::{PgPool, PgPoolOptions}; use sqlx::postgres::{PgPool, PgPoolOptions};
use std::str::FromStr; use std::str::FromStr;
@@ -53,6 +55,20 @@ impl EventReader for PgStore {
AND ($2::timestamptz IS NULL OR occurred_at < $2) AND ($2::timestamptz IS NULL OR occurred_at < $2)
AND ($3::text[] IS NULL OR source = ANY($3)) AND ($3::text[] IS NULL OR source = ANY($3))
AND ($4::bool OR public = true) AND ($4::bool OR public = true)
AND ($6::text IS NULL OR (CASE source
WHEN 'github' THEN COALESCE(
payload->'repo'->>'name',
payload->'repository'->>'full_name',
payload->>'_repo'
)
WHEN 'gitea' THEN COALESCE(
payload->'repo'->>'full_name',
payload->'repo'->>'name'
)
WHEN 'hg' THEN payload->>'_repo'
WHEN 'bugzilla' THEN payload->>'product'
ELSE NULL
END) = $6)
ORDER BY occurred_at DESC ORDER BY occurred_at DESC
LIMIT $5 LIMIT $5
"#, "#,
@@ -62,6 +78,7 @@ impl EventReader for PgStore {
.bind(sources.as_deref()) .bind(sources.as_deref())
.bind(query.include_private) .bind(query.include_private)
.bind(query.limit as i64) .bind(query.limit as i64)
.bind(query.repo.as_deref())
.fetch_all(&self.pool) .fetch_all(&self.pool)
.await .await
.map_err(map_err)?; .map_err(map_err)?;
@@ -114,6 +131,245 @@ impl EventReader for PgStore {
}) })
.collect() .collect()
} }
async fn list_projects(&self) -> Result<Vec<ProjectSummary>, StoreError> {
let rows = sqlx::query(
r#"
SELECT source, repo, host,
SUM(commits)::bigint AS commit_count,
SUM(issues)::bigint AS issue_count,
SUM(prs)::bigint AS pr_count,
MIN(occurred_at) AS first_activity,
MAX(occurred_at) AS last_activity
FROM (
SELECT source, occurred_at,
CASE source
WHEN 'github' THEN COALESCE(
payload->'repo'->>'name',
payload->'repository'->>'full_name',
payload->>'_repo'
)
WHEN 'gitea' THEN COALESCE(
payload->'repo'->>'full_name',
payload->'repo'->>'name'
)
WHEN 'hg' THEN payload->>'_repo'
WHEN 'bugzilla' THEN payload->>'product'
ELSE NULL
END AS repo,
CASE source
WHEN 'github' THEN 'github.com'
WHEN 'gitea' THEN COALESCE(payload->>'_host', 'git.lair.cafe')
WHEN 'hg' THEN COALESCE(payload->>'_host', 'hg-edge.mozilla.org')
WHEN 'bugzilla' THEN 'bugzilla.mozilla.org'
ELSE 'unknown'
END AS host,
CASE WHEN action IN ('Commit', 'PushEvent', 'commit_repo') THEN 1 ELSE 0 END AS commits,
CASE WHEN action IN ('Issue', 'IssuesEvent') THEN 1 ELSE 0 END AS issues,
CASE WHEN action IN ('PullRequest', 'PullRequestEvent') THEN 1 ELSE 0 END AS prs
FROM events
WHERE public = true
) sub
WHERE repo IS NOT NULL AND repo != ''
GROUP BY source, repo, host
ORDER BY MAX(occurred_at) DESC
"#,
)
.fetch_all(&self.pool)
.await
.map_err(map_err)?;
rows.into_iter()
.map(|r| {
let source_str: String = r.try_get("source").map_err(map_err)?;
Ok(ProjectSummary {
source: Source::from_str(&source_str).map_err(map_err)?,
repo: r.try_get("repo").map_err(map_err)?,
host: r.try_get("host").map_err(map_err)?,
commit_count: r.try_get::<i64, _>("commit_count").map_err(map_err).unwrap_or(0),
issue_count: r.try_get::<i64, _>("issue_count").map_err(map_err).unwrap_or(0),
pr_count: r.try_get::<i64, _>("pr_count").map_err(map_err).unwrap_or(0),
first_activity: r.try_get("first_activity").map_err(map_err)?,
last_activity: r.try_get("last_activity").map_err(map_err)?,
})
})
.collect()
}
async fn daily_counts(&self, from: NaiveDate, to: NaiveDate, include_private: bool) -> Result<Vec<DailyCount>, StoreError> {
let rows = sqlx::query(
r#"
SELECT d::date AS date,
COUNT(e.id)::bigint AS count
FROM generate_series($1::date, $2::date, '1 day') d
LEFT JOIN events e
ON e.occurred_at >= (d::date || 'T00:00:00Z')::timestamptz
AND e.occurred_at < ((d::date + 1) || 'T00:00:00Z')::timestamptz
AND ($3::bool OR e.public = true)
GROUP BY d::date
ORDER BY d::date
"#,
)
.bind(from)
.bind(to)
.bind(include_private)
.fetch_all(&self.pool)
.await
.map_err(map_err)?;
rows.into_iter()
.map(|r| {
Ok(DailyCount {
date: r.try_get("date").map_err(map_err)?,
count: r.try_get("count").map_err(map_err)?,
})
})
.collect()
}
async fn language_daily_counts(&self, from: NaiveDate, to: NaiveDate, include_private: bool) -> Result<Vec<LanguageDailyCount>, StoreError> {
let rows = sqlx::query(
r#"
SELECT date, language, color,
ROUND(SUM(weight))::bigint AS commits
FROM (
SELECT d::date AS date,
rl.language,
COALESCE(rl.color,
(SELECT color FROM repo_languages
WHERE language = rl.language AND color IS NOT NULL
LIMIT 1)
) AS color,
rl.bytes::float / NULLIF(rt.total, 0) AS weight
FROM generate_series($1::date, $2::date, '1 day') d
JOIN events e
ON e.occurred_at >= (d::date || 'T00:00:00Z')::timestamptz
AND e.occurred_at < ((d::date + 1) || 'T00:00:00Z')::timestamptz
AND ($3::bool OR e.public = true)
AND e.action IN ('Commit', 'PushEvent', 'commit_repo')
JOIN repo_languages rl
ON rl.source = e.source
AND rl.repo = CASE e.source
WHEN 'github' THEN COALESCE(
e.payload->'repo'->>'name',
e.payload->'repository'->>'full_name',
e.payload->>'_repo'
)
WHEN 'gitea' THEN COALESCE(
e.payload->'repo'->>'full_name',
e.payload->'repo'->>'name'
)
ELSE NULL
END
JOIN LATERAL (
SELECT SUM(bytes)::float AS total
FROM repo_languages r2
WHERE r2.source = rl.source AND r2.repo = rl.repo
) rt ON true
) weighted
GROUP BY date, language, color
HAVING ROUND(SUM(weight)) > 0
ORDER BY date, commits DESC
"#,
)
.bind(from)
.bind(to)
.bind(include_private)
.fetch_all(&self.pool)
.await
.map_err(map_err)?;
rows.into_iter()
.map(|r| {
Ok(LanguageDailyCount {
date: r.try_get("date").map_err(map_err)?,
language: r.try_get("language").map_err(map_err)?,
color: r.try_get("color").map_err(map_err)?,
commits: r.try_get("commits").map_err(map_err)?,
})
})
.collect()
}
async fn hourly_avgs(
&self,
from: NaiveDate,
to: NaiveDate,
tz: &str,
include_private: bool,
) -> Result<Vec<HourlyAvg>, StoreError> {
// GREATEST guards against from > to (returns NaN-via-div-by-zero
// otherwise). EXTRACT(hour FROM tz-shifted timestamp) buckets each
// event into the user's local hour rather than UTC, so the chart
// matches the labels they'd see on a clock.
let rows = sqlx::query(
r#"
WITH params AS (
SELECT GREATEST(($2::date - $1::date + 1), 1)::float8 AS day_count
),
bucketed AS (
SELECT EXTRACT(hour FROM (occurred_at AT TIME ZONE $3))::int AS hour
FROM events
WHERE occurred_at >= ($1::date::timestamp AT TIME ZONE 'UTC')
AND occurred_at < (($2::date + 1)::timestamp AT TIME ZONE 'UTC')
AND ($4::bool OR public = true)
)
SELECT g.h::int AS hour,
(COUNT(b.hour)::float8 / (SELECT day_count FROM params)) AS avg
FROM generate_series(0, 23) AS g(h)
LEFT JOIN bucketed b ON b.hour = g.h
GROUP BY g.h
ORDER BY g.h
"#,
)
.bind(from)
.bind(to)
.bind(tz)
.bind(include_private)
.fetch_all(&self.pool)
.await
.map_err(map_err)?;
rows.into_iter()
.map(|r| {
Ok(HourlyAvg {
hour: r.try_get("hour").map_err(map_err)?,
avg: r.try_get("avg").map_err(map_err)?,
})
})
.collect()
}
async fn repo_languages(&self) -> Result<Vec<RepoLanguage>, StoreError> {
let rows = sqlx::query(
r#"
SELECT source, repo, language, bytes,
COALESCE(color,
(SELECT color FROM repo_languages r2
WHERE r2.language = repo_languages.language AND r2.color IS NOT NULL
LIMIT 1)
) AS color
FROM repo_languages
ORDER BY repo, bytes DESC
"#,
)
.fetch_all(&self.pool)
.await
.map_err(map_err)?;
rows.into_iter()
.map(|r| {
let source_str: String = r.try_get("source").map_err(map_err)?;
Ok(RepoLanguage {
source: Source::from_str(&source_str).map_err(map_err)?,
repo: r.try_get("repo").map_err(map_err)?,
language: r.try_get("language").map_err(map_err)?,
bytes: r.try_get("bytes").map_err(map_err)?,
color: r.try_get("color").map_err(map_err)?,
})
})
.collect()
}
} }
#[async_trait] #[async_trait]
@@ -221,4 +477,37 @@ impl EventWriter for PgStore {
tx.commit().await.map_err(map_err)?; tx.commit().await.map_err(map_err)?;
Ok(inserted) Ok(inserted)
} }
async fn upsert_repo_languages(&self, languages: &[RepoLanguage]) -> Result<usize, StoreError> {
if languages.is_empty() {
return Ok(0);
}
let mut tx = self.pool.begin().await.map_err(map_err)?;
let mut count = 0usize;
for lang in languages {
let n = sqlx::query(
r#"
INSERT INTO repo_languages (source, repo, language, bytes, color, fetched_at)
VALUES ($1, $2, $3, $4, $5, now())
ON CONFLICT (source, repo, language) DO UPDATE
SET bytes = EXCLUDED.bytes,
color = EXCLUDED.color,
fetched_at = EXCLUDED.fetched_at
"#,
)
.bind(lang.source.as_str())
.bind(&lang.repo)
.bind(&lang.language)
.bind(lang.bytes)
.bind(&lang.color)
.execute(&mut *tx)
.await
.map_err(map_err)?
.rows_affected();
count += n as usize;
}
tx.commit().await.map_err(map_err)?;
Ok(count)
}
} }

49
crates/moments-entities/src/lib.rs
View File

@@ -67,6 +67,8 @@ pub struct EventQuery {
pub from: Option<DateTime<Utc>>, pub from: Option<DateTime<Utc>>,
pub to: Option<DateTime<Utc>>, pub to: Option<DateTime<Utc>>,
pub sources: Option<Vec<Source>>, pub sources: Option<Vec<Source>>,
/// Filter to events matching a specific repo (matched against payload).
pub repo: Option<String>,
/// When false (default), only `public = true` rows are returned. The API /// When false (default), only `public = true` rows are returned. The API
/// pins this to false today; a future authenticated path can flip it. /// pins this to false today; a future authenticated path can flip it.
pub include_private: bool, pub include_private: bool,
@@ -82,6 +84,53 @@ pub struct SourceSummary {
pub latest: Option<DateTime<Utc>>, pub latest: Option<DateTime<Utc>>,
} }
/// Per-day event count for the contribution graph.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DailyCount {
pub date: chrono::NaiveDate,
pub count: i64,
}
/// Average events per day at a given hour of the day, computed in a
/// caller-supplied IANA timezone. 24 entries (0..=23).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HourlyAvg {
pub hour: i32,
pub avg: f64,
}
/// Per-repo activity rollup for the dashboard.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProjectSummary {
pub repo: String,
pub source: Source,
pub host: String,
pub commit_count: i64,
pub issue_count: i64,
pub pr_count: i64,
pub first_activity: Option<DateTime<Utc>>,
pub last_activity: Option<DateTime<Utc>>,
}
/// Per-language daily commit count for the language stream graph.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LanguageDailyCount {
pub date: chrono::NaiveDate,
pub language: String,
pub color: Option<String>,
pub commits: i64,
}
/// Per-repo language breakdown from the forge.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RepoLanguage {
pub source: Source,
pub repo: String,
pub language: String,
pub bytes: i64,
pub color: Option<String>,
}
// --------------------------------------------------------------------- // ---------------------------------------------------------------------
// Presentation shape — what `GET /v1/events` actually returns. // Presentation shape — what `GET /v1/events` actually returns.
// The API reshapes raw payloads into these so the frontend stays dumb. // The API reshapes raw payloads into these so the frontend stays dumb.

43
crates/moments-worker/src/main.rs
View File

@@ -7,6 +7,7 @@ use moments_data::{
bugzilla::{BugzillaConfig, BugzillaSource}, bugzilla::{BugzillaConfig, BugzillaSource},
gitea::{GiteaConfig, GiteaSource}, gitea::{GiteaConfig, GiteaSource},
github::{GithubConfig, GithubSource}, github::{GithubConfig, GithubSource},
github_repo::{GithubRepoConfig, GithubRepoSource},
github_search::{GithubSearchConfig, GithubSearchSource}, github_search::{GithubSearchConfig, GithubSearchSource},
hg::{HgConfig, HgSource}, hg::{HgConfig, HgSource},
}; };
@@ -35,6 +36,11 @@ struct Args {
#[arg(long, env = "SEARCH_POLL_INTERVAL_SECS", default_value = "86400")] #[arg(long, env = "SEARCH_POLL_INTERVAL_SECS", default_value = "86400")]
search_interval_secs: u64, search_interval_secs: u64,
/// Seconds between per-repo commit enumeration polls (full history backfill).
/// Defaults to weekly — expensive initial scan, cheap afterwards.
#[arg(long, env = "REPO_POLL_INTERVAL_SECS", default_value = "604800")]
repo_interval_secs: u64,
#[arg(long, env = "GITEA_HOST", default_value = "git.lair.cafe")] #[arg(long, env = "GITEA_HOST", default_value = "git.lair.cafe")]
gitea_host: String, gitea_host: String,
@@ -51,21 +57,31 @@ struct Args {
#[arg(long, env = "HG_HOST", default_value = "hg-edge.mozilla.org")] #[arg(long, env = "HG_HOST", default_value = "hg-edge.mozilla.org")]
hg_host: String, hg_host: String,
/// Comma-separated mozilla hg repo paths to scan, e.g. "build/puppet,build/tools". /// Comma-separated repo groups to scan. Repos within each group are
/// discovered via `/{group}/?style=json`.
#[arg(
long,
env = "HG_GROUPS",
value_delimiter = ',',
default_value = "build,integration"
)]
hg_groups: Vec<String>,
/// Comma-separated individual repos to scan (e.g. `mozilla-central`).
#[arg( #[arg(
long, long,
env = "HG_REPOS", env = "HG_REPOS",
value_delimiter = ',', value_delimiter = ',',
default_value = "build/puppet,build/tools,build/buildbot-configs" default_value = "mozilla-central"
)] )]
hg_repos: Vec<String>, hg_repos: Vec<String>,
/// Comma-separated case-insensitive substrings matched against changeset author fields. /// Comma-separated author substrings for `author()` revset queries.
#[arg( #[arg(
long, long,
env = "HG_AUTHOR_TERMS", env = "HG_AUTHOR_TERMS",
value_delimiter = ',', value_delimiter = ',',
default_value = "thijssen,grenade" default_value = "rthijssen,grenade"
)] )]
hg_author_terms: Vec<String>, hg_author_terms: Vec<String>,
@@ -122,6 +138,17 @@ async fn main() -> anyhow::Result<()> {
}, },
)) as Arc<dyn EventSource>; )) as Arc<dyn EventSource>;
let github_repo = Arc::new(GithubRepoSource::new(
http.clone(),
store.clone(),
store.clone(),
GithubRepoConfig {
user: args.github_user.clone(),
token: args.github_token.clone(),
..Default::default()
},
)) as Arc<dyn EventSource>;
let gitea = Arc::new(GiteaSource::new( let gitea = Arc::new(GiteaSource::new(
http.clone(), http.clone(),
store.clone(), store.clone(),
@@ -141,6 +168,7 @@ async fn main() -> anyhow::Result<()> {
HgConfig { HgConfig {
host: args.hg_host.clone(), host: args.hg_host.clone(),
author_terms: args.hg_author_terms.clone(), author_terms: args.hg_author_terms.clone(),
groups: args.hg_groups.clone(),
repos: args.hg_repos.clone(), repos: args.hg_repos.clone(),
}, },
)) as Arc<dyn EventSource>; )) as Arc<dyn EventSource>;
@@ -162,11 +190,14 @@ async fn main() -> anyhow::Result<()> {
gitea_host = args.gitea_host, gitea_host = args.gitea_host,
gitea_user = args.gitea_user, gitea_user = args.gitea_user,
hg_host = args.hg_host, hg_host = args.hg_host,
hg_groups = ?args.hg_groups,
hg_repos = ?args.hg_repos, hg_repos = ?args.hg_repos,
hg_author_terms = ?args.hg_author_terms,
bugzilla_host = args.bugzilla_host, bugzilla_host = args.bugzilla_host,
bugzilla_email = args.bugzilla_email, bugzilla_email = args.bugzilla_email,
events_interval_secs = args.interval_secs, events_interval_secs = args.interval_secs,
search_interval_secs = args.search_interval_secs, search_interval_secs = args.search_interval_secs,
repo_interval_secs = args.repo_interval_secs,
gitea_interval_secs = args.gitea_interval_secs, gitea_interval_secs = args.gitea_interval_secs,
hg_interval_secs = args.hg_interval_secs, hg_interval_secs = args.hg_interval_secs,
bugzilla_interval_secs = args.bugzilla_interval_secs, bugzilla_interval_secs = args.bugzilla_interval_secs,
@@ -175,6 +206,7 @@ async fn main() -> anyhow::Result<()> {
let interval = Duration::from_secs(args.interval_secs); let interval = Duration::from_secs(args.interval_secs);
let search_interval = Duration::from_secs(args.search_interval_secs); let search_interval = Duration::from_secs(args.search_interval_secs);
let repo_interval = Duration::from_secs(args.repo_interval_secs);
let gitea_interval = Duration::from_secs(args.gitea_interval_secs); let gitea_interval = Duration::from_secs(args.gitea_interval_secs);
let hg_interval = Duration::from_secs(args.hg_interval_secs); let hg_interval = Duration::from_secs(args.hg_interval_secs);
let bugzilla_interval = Duration::from_secs(args.bugzilla_interval_secs); let bugzilla_interval = Duration::from_secs(args.bugzilla_interval_secs);
@@ -182,6 +214,8 @@ async fn main() -> anyhow::Result<()> {
let github_task = tokio::spawn(async move { run_poller(github, interval).await }); let github_task = tokio::spawn(async move { run_poller(github, interval).await });
let github_search_task = let github_search_task =
tokio::spawn(async move { run_poller(github_search, search_interval).await }); tokio::spawn(async move { run_poller(github_search, search_interval).await });
let github_repo_task =
tokio::spawn(async move { run_poller(github_repo, repo_interval).await });
let gitea_task = tokio::spawn(async move { run_poller(gitea, gitea_interval).await }); let gitea_task = tokio::spawn(async move { run_poller(gitea, gitea_interval).await });
let hg_task = tokio::spawn(async move { run_poller(hg, hg_interval).await }); let hg_task = tokio::spawn(async move { run_poller(hg, hg_interval).await });
let bugzilla_task = let bugzilla_task =
@@ -191,6 +225,7 @@ async fn main() -> anyhow::Result<()> {
info!("shutdown signal received"); info!("shutdown signal received");
github_task.abort(); github_task.abort();
github_search_task.abort(); github_search_task.abort();
github_repo_task.abort();
gitea_task.abort(); gitea_task.abort();
hg_task.abort(); hg_task.abort();
bugzilla_task.abort(); bugzilla_task.abort();

139
readme.md
View File

@@ -1,75 +1,136 @@
# moments # moments
Personal activity timeline for [rob.tn](https://rob.tn). Polls public sources (GitHub, Gitea, hg-edge.mozilla.org, bugzilla.mozilla.org), stores raw payloads in Postgres, and serves a reshaped timeline to a static React frontend. personal activity timeline and portfolio site. polls public sources (github, gitea, mercurial, bugzilla), stores raw payloads in postgres, and serves a dashboard + project detail views to a react frontend.
Successor to the now-defunct [grenade-events-react](https://github.com/grenade/grenade-events-react), which depended on MongoDB Stitch (retired by MongoDB in September 2022). successor to the now-defunct [grenade-events-react](https://github.com/grenade/grenade-events-react), which depended on mongodb stitch (retired by mongodb in september 2022).
## Layout ## layout
``` ```
crates/ crates/
moments-entities/ # types and DTOs moments-entities/ # types and dtos (event, source, project/daily summaries)
moments-core/ # ingestion + reshape logic moments-core/ # ingestion traits, presentation reshape, poller loop
moments-data/ # postgres adapter + migrations moments-data/ # postgres adapter, migrations, all event-source impls
moments-api/ # axum read-only HTTP API (binary) moments-api/ # axum read-only http api + forge proxy + og image (binary)
moments-worker/ # ingestion daemon (binary) moments-worker/ # ingestion daemon (binary)
ui/ # vite + react + swc + ts frontend ui/ # vite + react + swc + typescript frontend
asset/ # systemd, nginx, firewalld, manifest.yml asset/ # systemd, nginx, firewalld, manifest.yml
script/deploy.sh script/
deploy.sh # manifest-driven deploy to prod
hg-ingest.sh # one-shot local hg clone + psql ingest
certify.sh # letsencrypt cert management
teardown.sh # service removal
db-perms.sh # postgres role + ident setup
``` ```
Architectural conventions follow [grenade/architecture/generic.md](https://git.lair.cafe/grenade/architecture/src/branch/main/generic.md). architectural conventions follow [grenade/architecture/generic.md](https://git.lair.cafe/grenade/architecture/src/branch/main/generic.md).
## Local development ## data sources
| source | impl | endpoint | notes |
|--------|------|----------|-------|
| github events | `github.rs` | `/users/{user}/events` | last 90 days, etag-optimised polling |
| github search | `github_search.rs` | `/search/commits` + `/search/issues` | historical backfill, 1000-result cap |
| github repo | `github_repo.rs` | `/user/repos` + `/repos/{o}/{r}/commits` | full commit history, no cap, weekly poll |
| gitea | `gitea.rs` | user + org activity feeds | auto-discovers orgs, filters by user |
| mercurial | `hg.rs` | `json-log?rev=author()` | revset-based, one-shot backfill then skip |
| bugzilla | `bugzilla.rs` | `/rest/bug?creator=` | mozilla bugzilla |
hg repos are archived (mozilla retired hg). the worker skips hg after the first successful scan. for bulk ingestion, `script/hg-ingest.sh` clones repos locally and inserts via psql, avoiding rate limits on hg-edge.mozilla.org.
## frontend routes
| path | page | description |
|------|------|-------------|
| `/` or `/dash` | dashboard | contribution graphs (daily + all-time weekly) + ranked project cards with forge icons and language info |
| `/activity` | timeline | filterable activity feed with source toggles, date range slider, and event limit |
| `/activity/:timespan` | timeline | pre-filtered by date (`YYYY-MM-DD`) or range (`YYYY-MM-DD..YYYY-MM-DD`) |
| `/project/:source/*` | project detail | repo readme, language breakdown bar, per-repo activity timeline |
| `/cv` | resume | loaded from github gist, markdown-rendered |
shared layout provides nav header (dash, activity, cv + external links) and footer across all routes.
## api endpoints
| method | path | description |
|--------|------|-------------|
| GET | `/v1/healthz` | liveness probe |
| GET | `/v1/events?from=&to=&source=&repo=&limit=` | reshaped timeline items |
| GET | `/v1/sources` | per-source summary (count, earliest, latest) |
| GET | `/v1/projects` | per-repo aggregated stats (commits, issues, prs, date range) |
| GET | `/v1/activity/daily?from=&to=` | per-day event counts for contribution graphs |
| GET | `/v1/forge/{source}/*?host=` | proxy to github/gitea apis (avoids cors) |
| GET | `/v1/og/contributions.png` | server-rendered contribution graph as png (resvg) |
the og image endpoint renders the all-time weekly contribution graph as svg, rasterizes to png via resvg, and serves it with a 1-hour cache. used as the `og:image` meta tag for social media previews.
## local development
```sh ```sh
cargo build --workspace cargo build --workspace
cargo run -p moments-api # serves on 127.0.0.1:8080 cargo run -p moments-api # serves on 127.0.0.1:8080
cargo run -p moments-worker # one-shot ingest tick (until --interval is wired up) cargo run -p moments-worker # starts all pollers
cd ui && npm install && npm run dev # vite dev server on :5173
``` ```
The API expects a Postgres reachable at `DATABASE_URL`. For magrathea, that's an mTLS connection using the host cert. For local dev against a throwaway database: the api expects a postgres reachable at `DATABASE_URL`. in production this is an mtls connection using the host cert. for local dev against a throwaway database:
```sh ```sh
DATABASE_URL=postgres://localhost/moments cargo run -p moments-api DATABASE_URL=postgres://localhost/moments cargo run -p moments-api
``` ```
Migrations live in `crates/moments-data/migrations/` and run automatically on worker startup. The API connects as `moments_ro` and never runs migrations — the worker (as `moments_rw`) is the schema owner. migrations live in `crates/moments-data/migrations/` and run automatically on worker startup. the api connects as `moments_ro` and never runs migrations — the worker (as `moments_rw`) is the schema owner.
## Deployment ## deployment
```sh ```sh
./script/deploy.sh prod all # api + worker + web ./script/deploy.sh <env> all # api + worker + web
./script/deploy.sh prod api worker # subset ./script/deploy.sh <env> api worker # subset
./script/deploy.sh prod default # api + web only (worker untouched) ./script/deploy.sh <env> default # api + web only (worker untouched)
./script/deploy.sh prod all --dry-run ./script/deploy.sh <env> all --dry-run
``` ```
Topology: concrete hosts, ports, and the site's `server_name` live in `asset/manifest.yml`. the shape of the deployment:
| Component | Host | Notes | | component | notes |
| --------- | --------------------------------- | ------------------------------------------------------------------ | |-----------|-------|
| api | `anjie.kosherinata.internal` | binds `0.0.0.0:42424`; firewalld service `moments-api` | | api | binds the port from `api.config.bind`; firewalld service `moments-api` |
| worker | `anjie.kosherinata.internal` | no listening port; pollers only | | worker | no listening port; pollers only |
| web | `oolon.kosherinata.internal` | per-site nginx ingress for rob.tn; `/api/*` → anjie across the WG | | web | per-site nginx ingress; `/api/*` reverse-proxies to the api host |
| db | `magrathea.kosherinata.internal` | postgres mTLS, passwordless | | db | postgres mtls, passwordless |
api and worker are co-located on `anjie` while `nikola` and `frootmig` are out for drive replacement. postgres roles `moments_rw` and `moments_ro` must exist on the primary, with `pg_ident.conf.d/<host>.conf` mapping the api host's fqdn to `moments_ro` and the worker host's fqdn to `moments_rw`. see `asset/sql/bootstrap-moments.sql`, `asset/postgres/ident.conf.tmpl`, and `script/db-perms.sh`.
Postgres roles `moments_rw` and `moments_ro` must exist on the primary, with `pg_ident.conf` mapping `anjie.kosherinata.internal` to **both** roles (one cert_cn line per mapping). See `asset/sql/bootstrap-moments.sql` and `asset/postgres/ident.conf.tmpl`. secrets are resolved at deploy time via `pass`. the mapping of env-var name to pass-store path lives under `worker.secrets` in `manifest.yml`; `deploy.sh` iterates the map, fetches each secret, and substitutes the matching `{{NAME}}` placeholder in `worker.env.tmpl`.
Inter-host traffic over the WG mesh: oolon's nginx connects to `http://anjie.kosherinata.internal:42424` in plaintext. The mesh provides the encryption layer; per-hop TLS for an internal HTTP read-only API on already-public data is deferred. If that changes, swap the api binary to rustls + the host cert pair, and update the nginx upstream to `https://`. ## environment variables
Secrets resolved by `deploy.sh` via `pass`: ### worker
- `github.com/grenade/admin-token` — GitHub PAT for events + search APIs (worker only). | variable | default | description |
|----------|---------|-------------|
| `DATABASE_URL` | required | postgres connection string |
| `GITHUB_USER` | `grenade` | github username |
| `GITHUB_TOKEN` | optional | github pat for higher rate limits + private events |
| `POLL_INTERVAL_SECS` | `600` | github events api poll interval |
| `SEARCH_POLL_INTERVAL_SECS` | `86400` | github search backfill interval |
| `REPO_POLL_INTERVAL_SECS` | `604800` | github per-repo commit enumeration (weekly) |
| `GITEA_HOST` | `git.lair.cafe` | gitea instance hostname |
| `GITEA_USER` | `grenade` | gitea username |
| `GITEA_TOKEN` | optional | gitea token for org discovery |
| `GITEA_POLL_INTERVAL_SECS` | `600` | gitea activity feed poll interval |
| `HG_HOST` | `hg-edge.mozilla.org` | mercurial host |
| `HG_GROUPS` | `build,integration` | hg repo groups to discover |
| `HG_REPOS` | `mozilla-central` | individual hg repos |
| `HG_AUTHOR_TERMS` | `rthijssen,grenade` | author substrings for revset queries |
| `HG_POLL_INTERVAL_SECS` | `86400` | hg poll interval (skips after first scan) |
| `BUGZILLA_HOST` | `bugzilla.mozilla.org` | bugzilla instance |
| `BUGZILLA_EMAIL` | `rthijssen@mozilla.com` | bugzilla creator email filter |
| `BUGZILLA_POLL_INTERVAL_SECS` | `86400` | bugzilla poll interval |
Optional, set if needed in `worker.env`: `GITEA_TOKEN`, `BUGZILLA_API_KEY`. ### api
### DNS cutover | variable | default | description |
|----------|---------|-------------|
`rob.tn` currently resolves to GitHub Pages. After the first successful prod deploy: | `DATABASE_URL` | required | postgres connection string (read-only role) |
| `BIND_ADDR` | `127.0.0.1:8080` | api listen address |
1. Update Cloudflare DNS for `rob.tn` to the WAN IP that fronts `oolon` (unproxied — see architecture doc §11).
2. Confirm `curl -fsS https://rob.tn/api/v1/healthz` returns `ok`.
3. Add an archival notice to the top of [grenade-events-react/readme.md](https://github.com/grenade/grenade-events-react) pointing at this repo, and archive the GitHub repo.

63
script/db-perms.sh Executable file
View File

@@ -0,0 +1,63 @@
#!/usr/bin/env bash
#
# Idempotently add cert_cn → role mappings to pg_ident.conf.d on the moments
# postgres primary and standby, then reload postgres so the changes take
# effect. Re-running is a no-op (no duplicate lines, no spurious reload).
#
# Run from a workstation with ssh access to both pg hosts. This script ssh's
# out; do NOT run it on magrathea/frankie directly.
set -euo pipefail
api_host=nikola.kosherinata.internal
worker_host=frootmig.kosherinata.internal
pg_hosts=(
magrathea.kosherinata.internal
frankie.kosherinata.internal
)
# Each (cert_cn host, role) pair becomes one cert_cn line in
# pg_ident.conf.d/<cert_cn host>.conf on every pg host listed above.
mapping_pairs=(
"$api_host" moments_ro
"$worker_host" moments_rw
)
ident_dir=/var/lib/pgsql/18/data/pg_ident.conf.d
for pg_host in "${pg_hosts[@]}"; do
printf '==> %s\n' "$pg_host"
ssh -o BatchMode=yes "$pg_host" "sudo bash -s -- ${ident_dir@Q} ${mapping_pairs[@]@Q}" <<'REMOTE_EOF'
set -euo pipefail
ident_dir="$1"; shift
changed=0
while [[ $# -gt 0 ]]; do
cert_cn_host="$1"
role="$2"
shift 2
line="cert_cn ${cert_cn_host} ${role}"
file="${ident_dir}/${cert_cn_host}.conf"
# The heredoc runs as root via sudo bash, so [[ -f ]] and grep are fine
# without dropping privs. tee --append runs as postgres so a newly-created
# file lands with the conventional postgres:postgres ownership.
if [[ -f "$file" ]] && grep --fixed-strings --line-regexp --quiet -- "$line" "$file"; then
printf ' present: %s\n' "$line"
else
printf '%s\n' "$line" | sudo -u postgres tee --append "$file" >/dev/null
printf ' added: %s\n' "$line"
changed=1
fi
done
if (( changed )); then
systemctl reload postgresql-18
echo " reloaded postgresql-18"
else
echo " no changes; reload skipped"
fi
REMOTE_EOF
done

463
script/deploy.sh
View File

@@ -48,6 +48,31 @@ ssh_run() {
fi fi
} }
# Ensure /tmp on the remote is world-writable + sticky (mode 1777). Some
# hosts in this fleet have had /tmp reset to root-owned 0755 by an
# unrelated configuration step, which silently breaks the rsync of the
# deploy stage dir under our unprivileged user. Check the mode first so a
# correctly-configured host doesn't incur a needless sudo call.
ensure_tmp_writable() {
local host="$1"
if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m ssh %s -- stat /tmp; chmod 1777 if needed\n' "$host" >&2
return 0
fi
local mode
mode="$(ssh -o BatchMode=yes "$host" 'stat -c %a /tmp')" || {
warn "could not stat /tmp on $host"
return 1
}
if [[ "$mode" != "1777" ]]; then
warn "/tmp on $host is mode $mode; fixing to 1777"
ssh -o BatchMode=yes "$host" 'sudo chmod 1777 /tmp' || {
warn "failed to chmod /tmp on $host"
return 1
}
fi
}
[[ $# -ge 1 ]] || usage [[ $# -ge 1 ]] || usage
environment="$1"; shift environment="$1"; shift
components=() components=()
@@ -60,18 +85,32 @@ while [[ $# -gt 0 ]]; do
done done
[[ -f "$manifest" ]] || die "manifest not found: $manifest" [[ -f "$manifest" ]] || die "manifest not found: $manifest"
command -v yq >/dev/null 2>&1 || die "yq is required" command -v yq >/dev/null 2>&1 || die "yq is required"
command -v pass >/dev/null 2>&1 || die "pass is required" command -v pass >/dev/null 2>&1 || die "pass is required"
command -v rsync >/dev/null 2>&1 || die "rsync is required" command -v rsync >/dev/null 2>&1 || die "rsync is required"
command -v cargo >/dev/null 2>&1 || die "cargo is required" command -v cargo >/dev/null 2>&1 || die "cargo is required"
command -v podman >/dev/null 2>&1 || die "podman is required (used for the deploy build container)"
# Rust binaries are built inside a Debian container so the resulting ELF
# links against an older glibc than this workstation's. Building natively
# on f44 (glibc 2.43) produces binaries that won't load on f42 / f43
# servers — the dynamic loader refuses them outright. Debian bookworm's
# glibc 2.36 is older than every Fedora release we deploy to, so its
# binaries are forward-compatible.
#
# The artifacts land in target/deploy/release/ so a native `cargo build`
# in this checkout (for tests, clippy, dev runs) doesn't compete with
# the container for incremental state, and vice-versa.
rust_build_image="docker.io/library/rust:1-bookworm"
rust_target_dir="${repo_root}/target/deploy"
# Resolve component list ---------------------------------------------------- # Resolve component list ----------------------------------------------------
env_path=".environments.${environment}" env_path=".environments.${environment}"
yq -e "${env_path}" "$manifest" >/dev/null \ yq --exit-status "${env_path}" "$manifest" >/dev/null \
|| die "environment '$environment' not found in manifest" || die "environment '$environment' not found in manifest"
mapfile -t all_components < <(yq -r "${env_path}.components | keys | .[]" "$manifest") mapfile -t all_components < <(yq --raw-output "${env_path}.components | keys | .[]" "$manifest")
if [[ ${#components[@]} -eq 0 ]]; then if [[ ${#components[@]} -eq 0 ]]; then
usage usage
@@ -93,8 +132,20 @@ for c in "${components[@]}"; do
done done
if (( needs_rust )); then if (( needs_rust )); then
log "cargo build --release (api, worker)" log "cargo build --release in ${rust_build_image} (api, worker)"
run cargo build --release --bin moments-api --bin moments-worker --manifest-path "${repo_root}/Cargo.toml" install --directory "$rust_target_dir"
# Named volumes cache the cargo registry and git index across runs so
# subsequent builds don't re-fetch every crate. CARGO_TARGET_DIR
# redirects build output into the host-mounted target/deploy.
# :Z relabels the bind mount for SELinux on Fedora hosts.
run podman run --rm \
--volume "${repo_root}:/workspace:Z" \
--volume moments-deploy-cargo-registry:/usr/local/cargo/registry \
--volume moments-deploy-cargo-git:/usr/local/cargo/git \
--workdir /workspace \
--env CARGO_TARGET_DIR=/workspace/target/deploy \
"$rust_build_image" \
cargo build --release --bin moments-api --bin moments-worker
fi fi
if (( needs_web )); then if (( needs_web )); then
@@ -108,75 +159,118 @@ deploy_api() {
local host="$1" local host="$1"
log "api -> $host" log "api -> $host"
local bind
bind="$(yq --raw-output "${env_path}.components.api.config.bind" "$manifest")"
[[ -n "$bind" && "$bind" != "null" ]] || die "api.config.bind missing in manifest"
[[ "$bind" == *:* ]] \
|| die "api.config.bind must be host:port form: '$bind'"
local api_port
api_port="${bind##*:}"
[[ "$api_port" =~ ^[0-9]+$ ]] \
|| die "api.config.bind port is not numeric: '$api_port'"
if (( dry_run )); then if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m render api.env (HOSTNAME=%s) + units, rsync to %s:/, run sysusers/restorecon/semanage/systemctl on %s\n' \ printf '\033[2m[dry-run]\033[0m render api.env (HOSTNAME=%s, BIND=%s) + firewalld svc (port=%s) + units, stage to %s:/tmp/, install via heredoc, run sysusers/restorecon/semanage/systemctl on %s\n' \
"$host" "$host" "$host" >&2 "$host" "$bind" "$api_port" "$host" "$host" >&2
return 0 return 0
fi fi
local fqdn local fqdn="$host"
fqdn="$host"
local stage local stage
stage="$(mktemp -d)" stage="$(mktemp --directory)"
trap "rm -rf '$stage'" RETURN trap "rm --recursive --force '$stage'" RETURN
install -d "$stage/etc/moments" "$stage/etc/systemd/system" "$stage/etc/sysusers.d" "$stage/etc/firewalld/services" "$stage/usr/local/bin" install --directory \
"$stage/etc/moments" \
"$stage/etc/systemd/system" \
"$stage/etc/sysusers.d" \
"$stage/etc/firewalld/services" \
"$stage/usr/local/bin"
# Render env file with hostname substitution. local rendered
sed "s|{{HOSTNAME}}|${fqdn}|g" "${repo_root}/asset/config/api.env.tmpl" \ rendered="$(<"${repo_root}/asset/config/api.env.tmpl")"
> "$stage/etc/moments/api.env" rendered=${rendered//'{{HOSTNAME}}'/$fqdn}
rendered=${rendered//'{{BIND}}'/$bind}
printf '%s\n' "$rendered" > "$stage/etc/moments/api.env"
sed "s|{{HOSTNAME}}|${fqdn}|g" "${repo_root}/asset/systemd/moments-api-cert.path" \ rendered="$(<"${repo_root}/asset/systemd/moments-api-cert.path")"
> "$stage/etc/systemd/system/moments-api-cert.path" rendered=${rendered//'{{HOSTNAME}}'/$fqdn}
printf '%s\n' "$rendered" > "$stage/etc/systemd/system/moments-api-cert.path"
install -m 0644 "${repo_root}/asset/systemd/moments-api.service" "$stage/etc/systemd/system/" rendered="$(<"${repo_root}/asset/firewalld/moments-api.xml.tmpl")"
install -m 0644 "${repo_root}/asset/systemd/moments-api-cert-reload.service" "$stage/etc/systemd/system/" rendered=${rendered//'{{API_PORT}}'/$api_port}
install -m 0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf" printf '%s\n' "$rendered" > "$stage/etc/firewalld/services/moments-api.xml"
install -m 0644 "${repo_root}/asset/firewalld/moments-api.xml" "$stage/etc/firewalld/services/moments-api.xml" chmod 0644 "$stage/etc/firewalld/services/moments-api.xml"
install -m 0755 "${repo_root}/target/release/moments-api" "$stage/usr/local/bin/moments-api"
install --mode=0644 "${repo_root}/asset/systemd/moments-api.service" "$stage/etc/systemd/system/"
install --mode=0644 "${repo_root}/asset/systemd/moments-api-cert-reload.service" "$stage/etc/systemd/system/"
install --mode=0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf"
install --mode=0755 "${rust_target_dir}/release/moments-api" "$stage/usr/local/bin/moments-api"
# Permissions on the rendered env: root-owned, moments group readable.
chmod 0640 "$stage/etc/moments/api.env" chmod 0640 "$stage/etc/moments/api.env"
if (( dry_run )); then # Stage to a tmpdir on the remote, then `install` each file at its final
printf '\033[2m[dry-run]\033[0m rsync staged -> %s:/\n' "$host" >&2 # path via the heredoc. Never rsync into /, since rsync of staged parent
else # dirs (etc/, usr/, ...) can leak ownership, ACLs and xattrs onto the
rsync \ # live system dirs.
--archive \ local remote_stage="/tmp/moments-deploy.api.${$}.${RANDOM}"
--hard-links \
--acls \
--xattrs \
--numeric-ids \
--chown root:root \
--rsync-path 'sudo rsync' \
"$stage/" \
"${host}:/"
fi
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF' ensure_tmp_writable "$host" || return 1
rsync \
--archive \
--hard-links \
--numeric-ids \
--rsh='ssh -o BatchMode=yes' \
"$stage/" \
"${host}:${remote_stage}/"
ssh_run "$host" "sudo bash -s -- ${remote_stage@Q} ${api_port@Q}" <<'REMOTE_EOF'
set -euo pipefail set -euo pipefail
fqdn="$(hostname -f)" remote_stage="$1"
api_port="$2"
trap 'rm --recursive --force "$remote_stage"' EXIT
fqdn="$(hostname --fqdn)"
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/sysusers.d/moments.conf" \
/etc/sysusers.d/moments.conf
systemd-sysusers /etc/sysusers.d/moments.conf systemd-sysusers /etc/sysusers.d/moments.conf
install -d -o root -g moments -m 0750 /etc/moments install --directory --owner=root --group=moments --mode=0750 /etc/moments
install -d -o moments -g moments -m 0750 /var/lib/moments install --directory --owner=moments --group=moments --mode=0750 /var/lib/moments
chown root:moments /etc/moments/api.env
chmod 0640 /etc/moments/api.env
# Grant the moments user read access to the host private key — required for install --owner=root --group=moments --mode=0640 \
# the postgres mTLS connection. "$remote_stage/etc/moments/api.env" \
setfacl -m u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true /etc/moments/api.env
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-api.service" \
/etc/systemd/system/moments-api.service
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-api-cert.path" \
/etc/systemd/system/moments-api-cert.path
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-api-cert-reload.service" \
/etc/systemd/system/moments-api-cert-reload.service
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/firewalld/services/moments-api.xml" \
/etc/firewalld/services/moments-api.xml
install --owner=root --group=root --mode=0755 \
"$remote_stage/usr/local/bin/moments-api" \
/usr/local/bin/moments-api
# Label the API port. Idempotent — the -m fallback turns "already labelled" # Grant the moments user read access to the host private key for the
# into a no-op. # postgres mTLS connection.
if ! semanage port -l | awk '{print $1, $3}' | grep -qE "^http_port_t .*42424"; then setfacl --modify=u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true
semanage port -a -t http_port_t -p tcp 42424 || \
semanage port -m -t http_port_t -p tcp 42424 # Idempotent label: --add fails if the port is already labelled (we suppress
fi # that one stderr line); --modify is then a no-op or fixes a stale type.
semanage port --add --type=http_port_t --proto=tcp "$api_port" 2>/dev/null \
|| semanage port --modify --type=http_port_t --proto=tcp "$api_port"
# Firewalld: install the named service and enable it in the default zone.
firewall-cmd --reload firewall-cmd --reload
zone="$(firewall-cmd --get-default-zone)" zone="$(firewall-cmd --get-default-zone)"
if ! firewall-cmd --zone="$zone" --query-service=moments-api >/dev/null 2>&1; then if ! firewall-cmd --zone="$zone" --query-service=moments-api >/dev/null 2>&1; then
@@ -191,17 +285,18 @@ systemctl enable --now moments-api-cert.path
systemctl enable --now moments-api.service systemctl enable --now moments-api.service
systemctl restart moments-api.service systemctl restart moments-api.service
# Health probe — hit the bound interface, not loopback, so we exercise the # Quietly retry while the service binds; only show curl's diagnostics if
# same path nginx will use from oolon. # every attempt fails. The journalctl tail on failure is the verbose source.
for i in 1 2 3 4 5 6 7 8 9 10; do for i in 1 2 3 4 5 6 7 8 9 10; do
if curl -fsS "http://${fqdn}:42424/v1/healthz" >/dev/null; then if curl --fail --silent "http://${fqdn}:${api_port}/v1/healthz" >/dev/null 2>&1; then
echo "moments-api healthy" echo "moments-api healthy"
exit 0 exit 0
fi fi
sleep 1 sleep 1
done done
echo "moments-api did not become healthy" >&2 echo "moments-api did not become healthy" >&2
journalctl -u moments-api.service -n 50 --no-pager >&2 curl --fail --silent --show-error "http://${fqdn}:${api_port}/v1/healthz" >/dev/null || true
journalctl --unit=moments-api.service --lines=50 --no-pager >&2
exit 1 exit 1
REMOTE_EOF REMOTE_EOF
} }
@@ -210,69 +305,114 @@ deploy_worker() {
local host="$1" local host="$1"
log "worker -> $host" log "worker -> $host"
# Manifest entries under `worker.secrets` map env-var name -> pass store path.
# The script fetches each via `pass` and substitutes the matching {{NAME}}
# placeholder in worker.env.tmpl. Adding a new secret is then a manifest +
# template change; no script edit required.
local -a secret_lines secret_keys
mapfile -t secret_lines < <(yq --raw-output \
"${env_path}.components.worker.secrets // {} | to_entries | .[] | \"\(.key)=\(.value)\"" \
"$manifest")
local line
for line in "${secret_lines[@]}"; do
[[ -n "$line" ]] && secret_keys+=("${line%%=*}")
done
if (( dry_run )); then if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m render worker.env (HOSTNAME=%s, GITHUB_TOKEN from pass) + units, rsync to %s:/, run sysusers/restorecon/systemctl on %s\n' \ printf '\033[2m[dry-run]\033[0m render worker.env (HOSTNAME=%s, secrets [%s] from pass) + units, stage to %s:/tmp/, install via heredoc, run sysusers/restorecon/systemctl on %s\n' \
"$host" "$host" "$host" >&2 "$host" "${secret_keys[*]:-none}" "$host" "$host" >&2
return 0 return 0
fi fi
local fqdn local fqdn="$host"
fqdn="$host"
local github_token=""
if pass show github.com/grenade/admin-token >/dev/null 2>&1; then
github_token="$(pass show github.com/grenade/admin-token)"
else
warn "no github admin-token in pass; worker will run without GITHUB_TOKEN"
fi
local stage local stage
stage="$(mktemp -d)" stage="$(mktemp --directory)"
trap "rm -rf '$stage'" RETURN trap "rm --recursive --force '$stage'" RETURN
install -d "$stage/etc/moments" "$stage/etc/systemd/system" "$stage/etc/sysusers.d" "$stage/usr/local/bin" install --directory \
"$stage/etc/moments" \
"$stage/etc/systemd/system" \
"$stage/etc/sysusers.d" \
"$stage/usr/local/bin"
sed -e "s|{{HOSTNAME}}|${fqdn}|g" \ # Render templates in-memory so secrets never appear on a command line
-e "s|{{GITHUB_TOKEN}}|${github_token}|g" \ # (sed would expose them to anything that can read /proc/<pid>/cmdline).
"${repo_root}/asset/config/worker.env.tmpl" > "$stage/etc/moments/worker.env" local rendered
rendered="$(<"${repo_root}/asset/config/worker.env.tmpl")"
rendered=${rendered//'{{HOSTNAME}}'/$fqdn}
local key pass_path value
for line in "${secret_lines[@]}"; do
[[ -z "$line" ]] && continue
key="${line%%=*}"
pass_path="${line#*=}"
if pass show "$pass_path" >/dev/null 2>&1; then
value="$(pass show "$pass_path")"
else
warn "no secret in pass at '${pass_path}' for ${key}; worker will run without ${key}"
value=""
fi
rendered=${rendered//"{{${key}}}"/$value}
done
printf '%s\n' "$rendered" > "$stage/etc/moments/worker.env"
sed "s|{{HOSTNAME}}|${fqdn}|g" "${repo_root}/asset/systemd/moments-worker-cert.path" \ rendered="$(<"${repo_root}/asset/systemd/moments-worker-cert.path")"
> "$stage/etc/systemd/system/moments-worker-cert.path" rendered=${rendered//'{{HOSTNAME}}'/$fqdn}
printf '%s\n' "$rendered" > "$stage/etc/systemd/system/moments-worker-cert.path"
install -m 0644 "${repo_root}/asset/systemd/moments-worker.service" "$stage/etc/systemd/system/" install --mode=0644 "${repo_root}/asset/systemd/moments-worker.service" "$stage/etc/systemd/system/"
install -m 0644 "${repo_root}/asset/systemd/moments-worker-cert-reload.service" "$stage/etc/systemd/system/" install --mode=0644 "${repo_root}/asset/systemd/moments-worker-cert-reload.service" "$stage/etc/systemd/system/"
install -m 0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf" install --mode=0644 "${repo_root}/asset/systemd/moments.sysusers.conf" "$stage/etc/sysusers.d/moments.conf"
install -m 0755 "${repo_root}/target/release/moments-worker" "$stage/usr/local/bin/moments-worker" install --mode=0755 "${rust_target_dir}/release/moments-worker" "$stage/usr/local/bin/moments-worker"
chmod 0640 "$stage/etc/moments/worker.env" chmod 0640 "$stage/etc/moments/worker.env"
if (( dry_run )); then # Stage to a tmpdir on the remote, then `install` each file at its final
printf '\033[2m[dry-run]\033[0m rsync staged -> %s:/\n' "$host" >&2 # path via the heredoc. Never rsync into /.
else local remote_stage="/tmp/moments-deploy.worker.${$}.${RANDOM}"
rsync \
--archive \
--hard-links \
--acls \
--xattrs \
--numeric-ids \
--chown root:root \
--rsync-path 'sudo rsync' \
"$stage/" \
"${host}:/"
fi
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF' ensure_tmp_writable "$host" || return 1
rsync \
--archive \
--hard-links \
--numeric-ids \
--rsh='ssh -o BatchMode=yes' \
"$stage/" \
"${host}:${remote_stage}/"
ssh_run "$host" "sudo bash -s -- ${remote_stage@Q}" <<'REMOTE_EOF'
set -euo pipefail set -euo pipefail
fqdn="$(hostname -f)" remote_stage="$1"
trap 'rm --recursive --force "$remote_stage"' EXIT
fqdn="$(hostname --fqdn)"
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/sysusers.d/moments.conf" \
/etc/sysusers.d/moments.conf
systemd-sysusers /etc/sysusers.d/moments.conf systemd-sysusers /etc/sysusers.d/moments.conf
install -d -o root -g moments -m 0750 /etc/moments install --directory --owner=root --group=moments --mode=0750 /etc/moments
install -d -o moments -g moments -m 0750 /var/lib/moments install --directory --owner=moments --group=moments --mode=0750 /var/lib/moments
chown root:moments /etc/moments/worker.env
chmod 0640 /etc/moments/worker.env
setfacl -m u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true install --owner=root --group=moments --mode=0640 \
"$remote_stage/etc/moments/worker.env" \
/etc/moments/worker.env
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-worker.service" \
/etc/systemd/system/moments-worker.service
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-worker-cert.path" \
/etc/systemd/system/moments-worker-cert.path
install --owner=root --group=root --mode=0644 \
"$remote_stage/etc/systemd/system/moments-worker-cert-reload.service" \
/etc/systemd/system/moments-worker-cert-reload.service
install --owner=root --group=root --mode=0755 \
"$remote_stage/usr/local/bin/moments-worker" \
/usr/local/bin/moments-worker
setfacl --modify=u:moments:r "/etc/pki/tls/private/${fqdn}.pem" || true
restorecon -Rv /usr/local/bin/moments-worker /etc/moments /var/lib/moments restorecon -Rv /usr/local/bin/moments-worker /etc/moments /var/lib/moments
@@ -281,9 +421,8 @@ systemctl enable --now moments-worker-cert.path
systemctl enable --now moments-worker.service systemctl enable --now moments-worker.service
systemctl restart moments-worker.service systemctl restart moments-worker.service
# Liveness probe — worker doesn't expose a port, so check is-active.
if ! systemctl is-active --quiet moments-worker.service; then if ! systemctl is-active --quiet moments-worker.service; then
journalctl -u moments-worker.service -n 50 --no-pager >&2 journalctl --unit=moments-worker.service --lines=50 --no-pager >&2
exit 1 exit 1
fi fi
echo "moments-worker active" echo "moments-worker active"
@@ -294,61 +433,93 @@ deploy_web() {
local host="$1" local host="$1"
log "web -> $host" log "web -> $host"
local server_name web_root api_upstream
server_name="$(yq --raw-output "${env_path}.components.web.config.server_name" "$manifest")"
web_root="$(yq --raw-output "${env_path}.components.web.config.root" "$manifest")"
api_upstream="$(yq --raw-output "${env_path}.components.web.config.api_upstream" "$manifest")"
[[ -n "$server_name" && "$server_name" != "null" ]] || die "web.config.server_name missing in manifest"
[[ -n "$web_root" && "$web_root" != "null" ]] || die "web.config.root missing in manifest"
[[ -n "$api_upstream" && "$api_upstream" != "null" ]] || die "web.config.api_upstream missing in manifest"
[[ "$web_root" == /* ]] \
|| die "web.config.root must be an absolute path: '$web_root'"
[[ "$api_upstream" == http://* || "$api_upstream" == https://* ]] \
|| die "web.config.api_upstream must be a http(s) URL: '$api_upstream'"
local api_upstream_scheme api_upstream_addr api_upstream_port
api_upstream_scheme="${api_upstream%%://*}"
api_upstream_addr="${api_upstream#*://}"
[[ "$api_upstream_addr" == *:* ]] \
|| die "web.config.api_upstream must include an explicit port: '$api_upstream'"
api_upstream_port="${api_upstream_addr##*:}"
[[ "$api_upstream_port" =~ ^[0-9]+$ ]] \
|| die "extracted upstream port is not numeric: '$api_upstream_port'"
local site_conf_path="/etc/nginx/conf.d/${server_name}.conf"
if (( dry_run )); then if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m rsync ui/dist/ to %s:/var/www/rob.tn/ + nginx config, run nginx -t/reload on %s\n' \ printf '\033[2m[dry-run]\033[0m render %s (server_name=%s, docroot=%s, upstream=%s://%s) + rsync ui/dist/ to %s:%s/, run nginx -t/reload on %s\n' \
"$host" "$host" >&2 "$site_conf_path" "$server_name" "$web_root" \
"$api_upstream_scheme" "$api_upstream_addr" \
"$host" "$web_root" "$host" >&2
return 0 return 0
fi fi
local stage local stage
stage="$(mktemp -d)" stage="$(mktemp --directory)"
trap "rm -rf '$stage'" RETURN trap "rm --recursive --force '$stage'" RETURN
install -d "$stage/var/www/rob.tn" "$stage/etc/nginx/conf.d" install --directory "${stage}${web_root}" "$stage/etc/nginx/conf.d"
rsync -a "${repo_root}/ui/dist/" "$stage/var/www/rob.tn/" rsync --archive "${repo_root}/ui/dist/" "${stage}${web_root}/"
install -m 0644 "${repo_root}/asset/nginx/rob.tn.conf" "$stage/etc/nginx/conf.d/rob.tn.conf"
if (( dry_run )); then local rendered
printf '\033[2m[dry-run]\033[0m rsync staged -> %s:/\n' "$host" >&2 rendered="$(<"${repo_root}/asset/nginx/site.conf.tmpl")"
else rendered=${rendered//'{{SERVER_NAME}}'/$server_name}
rsync \ rendered=${rendered//'{{DOCROOT}}'/$web_root}
--archive \ rendered=${rendered//'{{API_UPSTREAM_SCHEME}}'/$api_upstream_scheme}
--hard-links \ rendered=${rendered//'{{API_UPSTREAM_ADDR}}'/$api_upstream_addr}
--acls \ printf '%s\n' "$rendered" > "${stage}${site_conf_path}"
--xattrs \ chmod 0644 "${stage}${site_conf_path}"
--numeric-ids \
--chown root:root \
--rsync-path 'sudo rsync' \
--delete \
"$stage/var/www/rob.tn/" \
"${host}:/var/www/rob.tn/"
rsync \
--archive \
--hard-links \
--acls \
--xattrs \
--numeric-ids \
--chown root:root \
--rsync-path 'sudo rsync' \
"$stage/etc/nginx/conf.d/rob.tn.conf" \
"${host}:/etc/nginx/conf.d/rob.tn.conf"
fi
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF' # Both targets are leaf paths (the docroot itself, and a single named
# file) so rsync does not traverse /var or /etc parents — `--chown` is
# enough; -A/-X are intentionally absent.
rsync \
--archive \
--hard-links \
--numeric-ids \
--chown root:root \
--rsh='ssh -o BatchMode=yes' \
--rsync-path 'sudo rsync' \
--delete \
"${stage}${web_root}/" \
"${host}:${web_root}/"
rsync \
--archive \
--hard-links \
--numeric-ids \
--chown root:root \
--rsh='ssh -o BatchMode=yes' \
--rsync-path 'sudo rsync' \
"${stage}${site_conf_path}" \
"${host}:${site_conf_path}"
ssh_run "$host" "sudo bash -s -- ${web_root@Q} ${site_conf_path@Q} ${api_upstream_port@Q}" <<'REMOTE_EOF'
set -euo pipefail set -euo pipefail
web_root="$1"
site_conf_path="$2"
api_upstream_port="$3"
# Allow nginx to make outbound connections to the moments-api upstream # Allow nginx to make outbound connections to the moments-api upstream
# across the WG mesh. # across the WG mesh.
setsebool -P httpd_can_network_connect on setsebool -P httpd_can_network_connect on
# Label the upstream port so httpd_t may name_connect to it. # Idempotent label: --add fails if the port is already labelled (we suppress
if ! semanage port -l | awk '{print $1, $3}' | grep -qE "^http_port_t .*42424"; then # that one stderr line); --modify is then a no-op or fixes a stale type.
semanage port -a -t http_port_t -p tcp 42424 || \ semanage port --add --type=http_port_t --proto=tcp "$api_upstream_port" 2>/dev/null \
semanage port -m -t http_port_t -p tcp 42424 || semanage port --modify --type=http_port_t --proto=tcp "$api_upstream_port"
fi
restorecon -Rv /var/www/rob.tn /etc/nginx/conf.d/rob.tn.conf restorecon -Rv "$web_root" "$site_conf_path"
if ! nginx -t; then if ! nginx -t; then
echo "nginx config check failed" >&2 echo "nginx config check failed" >&2
@@ -363,7 +534,7 @@ REMOTE_EOF
failed=() failed=()
for component in "${components[@]}"; do for component in "${components[@]}"; do
mapfile -t hosts < <(yq -r "${env_path}.components.${component}.hosts[]" "$manifest") mapfile -t hosts < <(yq --raw-output "${env_path}.components.${component}.hosts[]" "$manifest")
for host in "${hosts[@]}"; do for host in "${hosts[@]}"; do
case "$component" in case "$component" in
api) deploy_api "$host" || failed+=("api@$host") ;; api) deploy_api "$host" || failed+=("api@$host") ;;

141
script/hg-ingest.sh Executable file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env bash
#
# One-shot hg changeset ingestion via local clones.
#
# Bare-clones each hg repo, extracts changesets matching author terms,
# and inserts them into the moments database. Sets poller_state so the
# worker won't re-scan.
#
# Requirements: hg (mercurial), psql, jq
#
# Usage:
# DATABASE_URL="postgres://..." ./script/hg-ingest.sh
#
set -euo pipefail
DATABASE_URL="${DATABASE_URL:-postgres://moments_rw@magrathea.kosherinata.internal:5432/moments?sslmode=verify-full&sslrootcert=/etc/pki/ca-trust/source/anchors/root-internal.pem&sslcert=/etc/pki/tls/misc/$(hostname -f).pem&sslkey=/etc/pki/tls/private/$(hostname -f).pem}"
HG_HOST="${HG_HOST:-hg-edge.mozilla.org}"
WORK_DIR="${HG_WORK_DIR:-$HOME/hg}"
# Repos to clone (groups are expanded inline)
REPOS=(
mozilla-central
integration/mozilla-inbound
integration/autoland
integration/fx-team
integration/b2g-inbound
build/puppet
build/tools
build/buildbot
build/buildbot-configs
build/slave_health
build/mozharness
build/braindump
build/cloud-tools
build/compare-locales
build/nagios-core
build/partner-repacks
build/preproduction
build/rpm-sources
build/talos
build/tupperware
build/ash-mozharness
build/autoland
build/opsi-package-sources
)
# Author terms — matched case-insensitively against changeset author fields
AUTHOR_TERMS=("rthijssen" "grenade")
: "${DATABASE_URL:?DATABASE_URL must be set}"
mkdir -p "$WORK_DIR"
total=0
CLONE_DIR="$WORK_DIR/clone"
CACHE_DIR="$WORK_DIR/cache"
mkdir -p "$CACHE_DIR"
cd "$WORK_DIR"
for repo in "${REPOS[@]}"; do
cache_file="$CACHE_DIR/$(echo "$repo" | tr '/' '_').tsv"
# Skip repos already cached (re-run safe)
if [ -f "$cache_file" ]; then
echo "[hg-ingest] $repo: using cached results"
else
# Remove any previous clone to keep only one on disk
rm -rf "$CLONE_DIR"
echo "[hg-ingest] cloning $repo"
if ! hg clone --noupdate "https://$HG_HOST/$repo" "$CLONE_DIR"; then
echo "[hg-ingest] clone failed: $repo (skipping)"
continue
fi
# Build revset: author(term1) or author(term2) ...
revset=""
for term in "${AUTHOR_TERMS[@]}"; do
if [ -z "$revset" ]; then
revset="author('$term')"
else
revset="$revset or author('$term')"
fi
done
# Extract matching changesets to cache file
hg log -R "$CLONE_DIR" -r "$revset" \
--template '{node}\t{author}\t{date|hgdate}\t{desc|firstline}\n' \
> "$cache_file" || true
# Free disk immediately
rm -rf "$CLONE_DIR"
fi
# Ingest cached results into the database
count=0
while IFS=$'\t' read -r node author date_raw desc; do
[ -z "$node" ] && continue
# {date|hgdate} outputs "timestamp offset" — take just the timestamp
date_ts="${date_raw%% *}"
# Build ISO timestamp from unix epoch
occurred_at=$(date -u -d "@${date_ts}" '+%Y-%m-%dT%H:%M:%SZ')
event_id="hg:${repo}:${node}"
# Build payload JSON (jq handles all escaping)
payload=$(jq -n \
--arg node "$node" \
--arg user "$author" \
--arg desc "$desc" \
--arg repo "$repo" \
--arg host "$HG_HOST" \
'{node: $node, user: $user, desc: $desc, _repo: $repo, _host: $host}')
# Upsert into events table
psql "$DATABASE_URL" -q -c "
INSERT INTO events (id, source, action, occurred_at, public, payload)
VALUES (\$\$${event_id}\$\$, 'hg', 'Commit', '${occurred_at}', true, \$\$${payload}\$\$::jsonb)
ON CONFLICT (id) DO NOTHING;
"
count=$((count + 1))
done < "$cache_file"
if [ "$count" -gt 0 ]; then
echo "[hg-ingest] $repo: $count changesets ingested"
fi
total=$((total + count))
done
# Mark poller state so the worker skips hg
psql "$DATABASE_URL" -q -c "
INSERT INTO poller_state (source, last_fetched)
VALUES ('hg', now())
ON CONFLICT (source) DO UPDATE SET last_fetched = now();
"
echo "[hg-ingest] done. total: $total changesets"

288
script/teardown.sh Executable file
View File

@@ -0,0 +1,288 @@
#!/usr/bin/env bash
#
# moments teardown script.
#
# ./script/teardown.sh <environment> <host> [component...] [--dry-run]
# ./script/teardown.sh prod anjie.kosherinata.internal api worker
# ./script/teardown.sh prod oolon.kosherinata.internal web --remove-docroot
# ./script/teardown.sh prod anjie.kosherinata.internal all --dry-run
#
# Removes moments unit files, binaries, env files, firewalld service +
# definition, SELinux port label, and (when no moments component env files
# remain) the shared /etc/moments + /var/lib/moments dirs and the sysusers
# entry. Idempotent — safe to re-run.
#
# Notes:
# - The host argument is explicit on purpose: you typically tear down on
# hosts you've already removed from manifest.components.<c>.hosts.
# - Manifest is still read for env-wide config (api port, server_name,
# docroot path), so $environment must still resolve.
# - The `moments` user/group is intentionally NOT removed: any leftover
# file owned by it would become orphan-owned. Run `userdel moments`
# manually if you're certain there are none.
# - Web docroot is left intact unless --remove-docroot is given.
set -euo pipefail
shopt -s nullglob
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
manifest="${repo_root}/asset/manifest.yml"
dry_run=0
remove_docroot=0
usage() {
cat <<EOF >&2
usage: $(basename "$0") <environment> <host> [component...] [--dry-run] [--remove-docroot]
$(basename "$0") prod anjie.kosherinata.internal api worker
$(basename "$0") prod oolon.kosherinata.internal web --remove-docroot
$(basename "$0") prod anjie.kosherinata.internal all
components: api | worker | web | all
EOF
exit 2
}
log() { printf '\033[1;34m[teardown]\033[0m %s\n' "$*" >&2; }
warn() { printf '\033[1;33m[teardown]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[teardown]\033[0m %s\n' "$*" >&2; exit 1; }
ssh_run() {
local host="$1"; shift
if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m ssh %s -- %s\n' "$host" "$*" >&2
else
ssh -o BatchMode=yes "$host" "$@"
fi
}
[[ $# -ge 2 ]] || usage
environment="$1"; shift
target_host="$1"; shift
components=()
while [[ $# -gt 0 ]]; do
case "$1" in
--dry-run) dry_run=1 ;;
--remove-docroot) remove_docroot=1 ;;
*) components+=("$1") ;;
esac
shift
done
[[ -f "$manifest" ]] || die "manifest not found: $manifest"
command -v yq >/dev/null 2>&1 || die "yq is required"
env_path=".environments.${environment}"
yq --exit-status "${env_path}" "$manifest" >/dev/null \
|| die "environment '$environment' not found in manifest"
if [[ ${#components[@]} -eq 0 ]]; then
usage
fi
if [[ "${components[0]:-}" == "all" ]]; then
components=(api worker web)
fi
teardown_api() {
local host="$1"
log "api -> $host"
local bind api_port=""
bind="$(yq --raw-output "${env_path}.components.api.config.bind" "$manifest")"
if [[ -n "$bind" && "$bind" != "null" && "$bind" == *:* ]]; then
api_port="${bind##*:}"
[[ "$api_port" =~ ^[0-9]+$ ]] || api_port=""
fi
if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m stop+disable moments-api units, remove unit files, /etc/moments/api.env, /usr/local/bin/moments-api, firewalld svc moments-api, SELinux label tcp/%s on %s\n' \
"${api_port:-<unknown>}" "$host" >&2
return 0
fi
ssh_run "$host" "sudo bash -s -- ${api_port@Q}" <<'REMOTE_EOF'
set -euo pipefail
api_port="$1"
# Stop + disable units. `disable --now` quietly does nothing on a unit that
# isn't loaded, but emits non-zero exit on some systemd versions when the
# file is already gone — swallow that so re-runs are clean.
for unit in moments-api.service moments-api-cert.path moments-api-cert-reload.service; do
systemctl disable --now "$unit" 2>/dev/null || true
done
rm --force \
/etc/systemd/system/moments-api.service \
/etc/systemd/system/moments-api-cert.path \
/etc/systemd/system/moments-api-cert-reload.service
systemctl daemon-reload
rm --force /etc/moments/api.env /usr/local/bin/moments-api
# Firewalld: remove service from default zone, then drop service definition.
zone="$(firewall-cmd --get-default-zone)"
if firewall-cmd --zone="$zone" --query-service=moments-api >/dev/null 2>&1; then
firewall-cmd --permanent --zone="$zone" --remove-service=moments-api
firewall-cmd --zone="$zone" --remove-service=moments-api 2>/dev/null || true
fi
rm --force /etc/firewalld/services/moments-api.xml
firewall-cmd --reload
# SELinux: remove the port label, if we know which port. --delete fails when
# the port wasn't user-labelled — that's fine, swallow it.
if [[ -n "$api_port" ]]; then
semanage port --delete --proto=tcp "$api_port" 2>/dev/null || true
fi
echo "moments-api torn down"
REMOTE_EOF
}
teardown_worker() {
local host="$1"
log "worker -> $host"
if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m stop+disable moments-worker units, remove unit files, /etc/moments/worker.env, /usr/local/bin/moments-worker on %s\n' \
"$host" >&2
return 0
fi
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF'
set -euo pipefail
for unit in moments-worker.service moments-worker-cert.path moments-worker-cert-reload.service; do
systemctl disable --now "$unit" 2>/dev/null || true
done
rm --force \
/etc/systemd/system/moments-worker.service \
/etc/systemd/system/moments-worker-cert.path \
/etc/systemd/system/moments-worker-cert-reload.service
systemctl daemon-reload
rm --force /etc/moments/worker.env /usr/local/bin/moments-worker
echo "moments-worker torn down"
REMOTE_EOF
}
teardown_web() {
local host="$1"
log "web -> $host"
local server_name web_root
server_name="$(yq --raw-output "${env_path}.components.web.config.server_name" "$manifest")"
web_root="$(yq --raw-output "${env_path}.components.web.config.root" "$manifest")"
[[ -n "$server_name" && "$server_name" != "null" ]] || die "web.config.server_name missing in manifest"
[[ -n "$web_root" && "$web_root" != "null" ]] || die "web.config.root missing in manifest"
[[ "$web_root" == /* ]] || die "web.config.root must be an absolute path: '$web_root'"
# Refuse to recursively remove a shallow or system path even if the
# manifest says so.
if (( remove_docroot )); then
case "$web_root" in
/|/bin|/bin/*|/boot|/boot/*|/dev|/dev/*|/etc|/etc/*|/home|/home/*|/lib|/lib/*|/lib64|/lib64/*|/proc|/proc/*|/root|/root/*|/run|/run/*|/sbin|/sbin/*|/srv|/srv/*|/sys|/sys/*|/tmp|/tmp/*|/usr|/usr/*|/var|/var/lib|/var/log|/var/run|/var/spool|/var/www)
die "refusing to recursively remove a system path: '$web_root'"
;;
esac
# Require at least three path components (e.g. /var/www/<site>) to
# rule out things like /opt or /srv directly.
[[ "$web_root" =~ ^/[^/]+/[^/]+/[^/]+ ]] \
|| die "refusing to recursively remove a path with fewer than 3 components: '$web_root'"
fi
local site_conf_path="/etc/nginx/conf.d/${server_name}.conf"
if (( dry_run )); then
if (( remove_docroot )); then
printf '\033[2m[dry-run]\033[0m remove %s, recursively remove %s, nginx -t/reload on %s\n' \
"$site_conf_path" "$web_root" "$host" >&2
else
printf '\033[2m[dry-run]\033[0m remove %s, nginx -t/reload on %s (docroot %s left intact; pass --remove-docroot to also clear it)\n' \
"$site_conf_path" "$host" "$web_root" >&2
fi
return 0
fi
ssh_run "$host" "sudo bash -s -- ${site_conf_path@Q} ${web_root@Q} ${remove_docroot@Q}" <<'REMOTE_EOF'
set -euo pipefail
site_conf_path="$1"
web_root="$2"
remove_docroot="$3"
rm --force "$site_conf_path"
if nginx -t 2>&1; then
systemctl reload nginx
echo "nginx reloaded without ${site_conf_path}"
else
echo "nginx -t failed AFTER removing ${site_conf_path}; check other site configs" >&2
exit 1
fi
if [[ "$remove_docroot" == "1" && -d "$web_root" ]]; then
rm --recursive --force "$web_root"
echo "removed docroot ${web_root}"
fi
REMOTE_EOF
}
teardown_shared() {
local host="$1"
log "shared (post-component cleanup) -> $host"
if (( dry_run )); then
printf '\033[2m[dry-run]\033[0m if no api.env/worker.env remain: remove /etc/sysusers.d/moments.conf and rmdir /etc/moments + /var/lib/moments on %s (moments user left in place)\n' \
"$host" >&2
return 0
fi
ssh_run "$host" "sudo bash -s" <<'REMOTE_EOF'
set -euo pipefail
# If any component env still exists, leave shared state alone — another
# moments component is still using /etc/moments and the moments user.
if [[ -e /etc/moments/api.env || -e /etc/moments/worker.env ]]; then
echo "moments env files still present; leaving /etc/moments + /var/lib/moments + sysusers entry in place"
exit 0
fi
# rmdir refuses non-empty dirs — defensive against unknown stragglers.
rmdir /etc/moments 2>/dev/null || true
rmdir /var/lib/moments 2>/dev/null || true
rm --force /etc/sysusers.d/moments.conf
echo "shared state cleared (where empty); moments user/group intentionally left in place"
REMOTE_EOF
}
# Dispatch ------------------------------------------------------------------
failed=()
did_app=0
for component in "${components[@]}"; do
case "$component" in
api) teardown_api "$target_host" || failed+=("api@$target_host") ;;
worker) teardown_worker "$target_host" || failed+=("worker@$target_host") ;;
web) teardown_web "$target_host" || failed+=("web@$target_host") ;;
*) warn "unknown component: $component" ;;
esac
case "$component" in
api|worker) did_app=1 ;;
esac
done
# Shared cleanup runs after api/worker teardown. It's a no-op if either
# component still has its env file present on the host.
if (( did_app )); then
teardown_shared "$target_host" || failed+=("shared@$target_host")
fi
if [[ ${#failed[@]} -gt 0 ]]; then
die "failed: ${failed[*]}"
fi
log "teardown complete on $target_host"

81
ui/index.html
View File

@@ -1,13 +1,74 @@
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>rob.tn</title> <title>rob.tn</title>
<link rel="icon" type="image/x-icon" href="/favicon.ico" /> <meta
</head> name="description"
<body> content="a timeline of open source contributions across github, gitea, and mozilla hg. ranked projects, language trends, and commit activity since 2012."
<div id="root"></div> />
<script type="module" src="/src/main.tsx"></script> <meta
</body> property="og:title"
content="rob thijssen: developer activity and contribution history"
/>
<meta
property="og:description"
content="a timeline of open source contributions across github, gitea, and mozilla hg. ranked projects, language trends, and commit activity since 2012."
/>
<meta
property="og:image"
content="https://rob.tn/api/v1/og/contributions.png"
width="1200"
height="630"
/>
<meta property="og:type" content="website" />
<meta property="og:url" content="https://rob.tn/" />
<meta property="og:site_name" content="rob.tn" />
<meta property="og:locale" content="en_US" />
<meta property="og:logo" content="https://rob.tn/icon-512.png" />
<meta name="twitter:card" content="summary_large_image" />
<meta
name="twitter:image"
content="https://rob.tn/api/v1/og/contributions.png"
width="1200"
height="630"
/>
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
<link
rel="icon"
type="image/png"
sizes="16x16"
href="/favicon-16.png"
/>
<link
rel="icon"
type="image/png"
sizes="32x32"
href="/favicon-32.png"
/>
<link
rel="icon"
type="image/png"
sizes="48x48"
href="/favicon-48.png"
/>
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
<link
rel="icon"
type="image/png"
sizes="192x192"
href="/icon-192.png"
/>
<link
rel="icon"
type="image/png"
sizes="512x512"
href="/icon-512.png"
/>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html> </html>

6
ui/package.json
View File

@@ -18,7 +18,11 @@
"react-bootstrap-icons": "^1.11.4", "react-bootstrap-icons": "^1.11.4",
"react-dom": "^19.0.0", "react-dom": "^19.0.0",
"react-markdown": "^9.0.1", "react-markdown": "^9.0.1",
"react-vertical-timeline-component": "^3.6.0" "react-router-dom": "^7.14.2",
"react-vertical-timeline-component": "^3.6.0",
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1"
}, },
"devDependencies": { "devDependencies": {
"@types/react": "^19.0.0", "@types/react": "^19.0.0",

373
ui/pnpm-lock.yaml generated
View File

@@ -32,9 +32,21 @@ importers:
react-markdown: react-markdown:
specifier: ^9.0.1 specifier: ^9.0.1
version: 9.1.0(@types/react@19.2.14)(react@19.2.5) version: 9.1.0(@types/react@19.2.14)(react@19.2.5)
react-router-dom:
specifier: ^7.14.2
version: 7.14.2(react-dom@19.2.5(react@19.2.5))(react@19.2.5)
react-vertical-timeline-component: react-vertical-timeline-component:
specifier: ^3.6.0 specifier: ^3.6.0
version: 3.6.0(react@19.2.5) version: 3.6.0(react@19.2.5)
rehype-raw:
specifier: ^7.0.0
version: 7.0.0
rehype-sanitize:
specifier: ^6.0.0
version: 6.0.0
remark-gfm:
specifier: ^4.0.1
version: 4.0.1
devDependencies: devDependencies:
'@types/react': '@types/react':
specifier: ^19.0.0 specifier: ^19.0.0
@@ -593,6 +605,10 @@ packages:
comma-separated-tokens@2.0.3: comma-separated-tokens@2.0.3:
resolution: {integrity: sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg==} resolution: {integrity: sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg==}
cookie@1.1.1:
resolution: {integrity: sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==}
engines: {node: '>=18'}
csstype@3.2.3: csstype@3.2.3:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
@@ -618,11 +634,19 @@ packages:
dom-helpers@5.2.1: dom-helpers@5.2.1:
resolution: {integrity: sha512-nRCa7CK3VTrM2NmGkIy4cbK7IZlgBE/PYMn55rrXefr5xXDP0LdtfPnblFDoVdcAfslJ7or6iqAUnx0CCGIWQA==} resolution: {integrity: sha512-nRCa7CK3VTrM2NmGkIy4cbK7IZlgBE/PYMn55rrXefr5xXDP0LdtfPnblFDoVdcAfslJ7or6iqAUnx0CCGIWQA==}
entities@6.0.1:
resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==}
engines: {node: '>=0.12'}
esbuild@0.25.12: esbuild@0.25.12:
resolution: {integrity: sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg==} resolution: {integrity: sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg==}
engines: {node: '>=18'} engines: {node: '>=18'}
hasBin: true hasBin: true
escape-string-regexp@5.0.0:
resolution: {integrity: sha512-/veY75JbMK4j1yjvuUxuVsiS/hr/4iHs9FTT6cgTexxdE0Ly/glccBAkloH/DofkjRbZU3bnoj38mOmhkZ0lHw==}
engines: {node: '>=12'}
estree-util-is-identifier-name@3.0.0: estree-util-is-identifier-name@3.0.0:
resolution: {integrity: sha512-hFtqIDZTIUZ9BXLb8y4pYGyk6+wekIivNVTcmvk8NoOh+VeRn5y6cEHzbURrWbfp1fIqdVipilzj+lfaadNZmg==} resolution: {integrity: sha512-hFtqIDZTIUZ9BXLb8y4pYGyk6+wekIivNVTcmvk8NoOh+VeRn5y6cEHzbURrWbfp1fIqdVipilzj+lfaadNZmg==}
@@ -643,15 +667,36 @@ packages:
engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0}
os: [darwin] os: [darwin]
hast-util-from-parse5@8.0.3:
resolution: {integrity: sha512-3kxEVkEKt0zvcZ3hCRYI8rqrgwtlIOFMWkbclACvjlDw8Li9S2hk/d51OI0nr/gIpdMHNepwgOKqZ/sy0Clpyg==}
hast-util-parse-selector@4.0.0:
resolution: {integrity: sha512-wkQCkSYoOGCRKERFWcxMVMOcYE2K1AaNLU8DXS9arxnLOUEWbOXKXiJUNzEpqZ3JOKpnha3jkFrumEjVliDe7A==}
hast-util-raw@9.1.0:
resolution: {integrity: sha512-Y8/SBAHkZGoNkpzqqfCldijcuUKh7/su31kEBp67cFY09Wy0mTRgtsLYsiIxMJxlu0f6AA5SUTbDR8K0rxnbUw==}
hast-util-sanitize@5.0.2:
resolution: {integrity: sha512-3yTWghByc50aGS7JlGhk61SPenfE/p1oaFeNwkOOyrscaOkMGrcW9+Cy/QAIOBpZxP1yqDIzFMR0+Np0i0+usg==}
hast-util-to-jsx-runtime@2.3.6: hast-util-to-jsx-runtime@2.3.6:
resolution: {integrity: sha512-zl6s8LwNyo1P9uw+XJGvZtdFF1GdAkOg8ujOw+4Pyb76874fLps4ueHXDhXWdk6YHQ6OgUtinliG7RsYvCbbBg==} resolution: {integrity: sha512-zl6s8LwNyo1P9uw+XJGvZtdFF1GdAkOg8ujOw+4Pyb76874fLps4ueHXDhXWdk6YHQ6OgUtinliG7RsYvCbbBg==}
hast-util-to-parse5@8.0.1:
resolution: {integrity: sha512-MlWT6Pjt4CG9lFCjiz4BH7l9wmrMkfkJYCxFwKQic8+RTZgWPuWxwAfjJElsXkex7DJjfSJsQIt931ilUgmwdA==}
hast-util-whitespace@3.0.0: hast-util-whitespace@3.0.0:
resolution: {integrity: sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw==} resolution: {integrity: sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw==}
hastscript@9.0.1:
resolution: {integrity: sha512-g7df9rMFX/SPi34tyGCyUBREQoKkapwdY/T04Qn9TDWfHhAYt4/I0gMVirzK5wEzeUqIjEB+LXC/ypb7Aqno5w==}
html-url-attributes@3.0.1: html-url-attributes@3.0.1:
resolution: {integrity: sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ==} resolution: {integrity: sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ==}
html-void-elements@3.0.0:
resolution: {integrity: sha512-bEqo66MRXsUGxWHV5IP0PUiAWwoEjba4VCzg0LjFJBpchPaTfyfCKTG6bc5F8ucKec3q5y6qOdGyYTSBEvhCrg==}
inline-style-parser@0.2.7: inline-style-parser@0.2.7:
resolution: {integrity: sha512-Nb2ctOyNR8DqQoR0OwRG95uNWIC0C1lCgf5Naz5H6Ji72KZ8OcFZLz2P5sNgwlyoJ8Yif11oMuYs5pBQa86csA==} resolution: {integrity: sha512-Nb2ctOyNR8DqQoR0OwRG95uNWIC0C1lCgf5Naz5H6Ji72KZ8OcFZLz2P5sNgwlyoJ8Yif11oMuYs5pBQa86csA==}
@@ -684,9 +729,33 @@ packages:
resolution: {integrity: sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+fRLevlw1HgMySw7f611UIY408EtxRSoK3Q==} resolution: {integrity: sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+fRLevlw1HgMySw7f611UIY408EtxRSoK3Q==}
hasBin: true hasBin: true
markdown-table@3.0.4:
resolution: {integrity: sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw==}
mdast-util-find-and-replace@3.0.2:
resolution: {integrity: sha512-Tmd1Vg/m3Xz43afeNxDIhWRtFZgM2VLyaf4vSTYwudTyeuTneoL3qtWMA5jeLyz/O1vDJmmV4QuScFCA2tBPwg==}
mdast-util-from-markdown@2.0.3: mdast-util-from-markdown@2.0.3:
resolution: {integrity: sha512-W4mAWTvSlKvf8L6J+VN9yLSqQ9AOAAvHuoDAmPkz4dHf553m5gVj2ejadHJhoJmcmxEnOv6Pa8XJhpxE93kb8Q==} resolution: {integrity: sha512-W4mAWTvSlKvf8L6J+VN9yLSqQ9AOAAvHuoDAmPkz4dHf553m5gVj2ejadHJhoJmcmxEnOv6Pa8XJhpxE93kb8Q==}
mdast-util-gfm-autolink-literal@2.0.1:
resolution: {integrity: sha512-5HVP2MKaP6L+G6YaxPNjuL0BPrq9orG3TsrZ9YXbA3vDw/ACI4MEsnoDpn6ZNm7GnZgtAcONJyPhOP8tNJQavQ==}
mdast-util-gfm-footnote@2.1.0:
resolution: {integrity: sha512-sqpDWlsHn7Ac9GNZQMeUzPQSMzR6Wv0WKRNvQRg0KqHh02fpTz69Qc1QSseNX29bhz1ROIyNyxExfawVKTm1GQ==}
mdast-util-gfm-strikethrough@2.0.0:
resolution: {integrity: sha512-mKKb915TF+OC5ptj5bJ7WFRPdYtuHv0yTRxK2tJvi+BDqbkiG7h7u/9SI89nRAYcmap2xHQL9D+QG/6wSrTtXg==}
mdast-util-gfm-table@2.0.0:
resolution: {integrity: sha512-78UEvebzz/rJIxLvE7ZtDd/vIQ0RHv+3Mh5DR96p7cS7HsBhYIICDBCu8csTNWNO6tBWfqXPWekRuj2FNOGOZg==}
mdast-util-gfm-task-list-item@2.0.0:
resolution: {integrity: sha512-IrtvNvjxC1o06taBAVJznEnkiHxLFTzgonUdy8hzFVeDun0uTjxxrRGVaNFqkU1wJR3RBPEfsxmU6jDWPofrTQ==}
mdast-util-gfm@3.1.0:
resolution: {integrity: sha512-0ulfdQOM3ysHhCJ1p06l0b0VKlhU0wuQs3thxZQagjcjPrlFRqY215uZGHHJan9GEAXd9MbfPjFJz+qMkVR6zQ==}
mdast-util-mdx-expression@2.0.1: mdast-util-mdx-expression@2.0.1:
resolution: {integrity: sha512-J6f+9hUp+ldTZqKRSg7Vw5V6MqjATc+3E4gf3CFNcuZNWD8XdyI6zQ8GqH7f8169MM6P7hMBRDVGnn7oHB9kXQ==} resolution: {integrity: sha512-J6f+9hUp+ldTZqKRSg7Vw5V6MqjATc+3E4gf3CFNcuZNWD8XdyI6zQ8GqH7f8169MM6P7hMBRDVGnn7oHB9kXQ==}
@@ -711,6 +780,27 @@ packages:
micromark-core-commonmark@2.0.3: micromark-core-commonmark@2.0.3:
resolution: {integrity: sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg==} resolution: {integrity: sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg==}
micromark-extension-gfm-autolink-literal@2.1.0:
resolution: {integrity: sha512-oOg7knzhicgQ3t4QCjCWgTmfNhvQbDDnJeVu9v81r7NltNCVmhPy1fJRX27pISafdjL+SVc4d3l48Gb6pbRypw==}
micromark-extension-gfm-footnote@2.1.0:
resolution: {integrity: sha512-/yPhxI1ntnDNsiHtzLKYnE3vf9JZ6cAisqVDauhp4CEHxlb4uoOTxOCJ+9s51bIB8U1N1FJ1RXOKTIlD5B/gqw==}
micromark-extension-gfm-strikethrough@2.1.0:
resolution: {integrity: sha512-ADVjpOOkjz1hhkZLlBiYA9cR2Anf8F4HqZUO6e5eDcPQd0Txw5fxLzzxnEkSkfnD0wziSGiv7sYhk/ktvbf1uw==}
micromark-extension-gfm-table@2.1.1:
resolution: {integrity: sha512-t2OU/dXXioARrC6yWfJ4hqB7rct14e8f7m0cbI5hUmDyyIlwv5vEtooptH8INkbLzOatzKuVbQmAYcbWoyz6Dg==}
micromark-extension-gfm-tagfilter@2.0.0:
resolution: {integrity: sha512-xHlTOmuCSotIA8TW1mDIM6X2O1SiX5P9IuDtqGonFhEK0qgRI4yeC6vMxEV2dgyr2TiD+2PQ10o+cOhdVAcwfg==}
micromark-extension-gfm-task-list-item@2.1.0:
resolution: {integrity: sha512-qIBZhqxqI6fjLDYFTBIa4eivDMnP+OZqsNwmQ3xNLE4Cxwc+zfQEfbs6tzAo2Hjq+bh6q5F+Z8/cksrLFYWQQw==}
micromark-extension-gfm@3.0.0:
resolution: {integrity: sha512-vsKArQsicm7t0z2GugkCKtZehqUm31oeGBV/KVSorWSy8ZlNAv7ytjFhvaryUiCUJYqs+NoE6AFhpQvBTM6Q4w==}
micromark-factory-destination@2.0.1: micromark-factory-destination@2.0.1:
resolution: {integrity: sha512-Xe6rDdJlkmbFRExpTOmRj9N3MaWmbAgdpSrBQvCFqhezUn4AHqJHbaEnfbVYYiexVSs//tqOdY/DxhjdCiJnIA==} resolution: {integrity: sha512-Xe6rDdJlkmbFRExpTOmRj9N3MaWmbAgdpSrBQvCFqhezUn4AHqJHbaEnfbVYYiexVSs//tqOdY/DxhjdCiJnIA==}
@@ -786,6 +876,9 @@ packages:
parse-entities@4.0.2: parse-entities@4.0.2:
resolution: {integrity: sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw==} resolution: {integrity: sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw==}
parse5@7.3.0:
resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==}
picocolors@1.1.1: picocolors@1.1.1:
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==} resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
@@ -867,6 +960,23 @@ packages:
'@types/react': '>=18' '@types/react': '>=18'
react: '>=18' react: '>=18'
react-router-dom@7.14.2:
resolution: {integrity: sha512-YZcM5ES8jJSM+KrJ9BdvHHqlnGTg5tH3sC5ChFRj4inosKctdyzBDhOyyHdGk597q2OT6NTrCA1OvB/YDwfekQ==}
engines: {node: '>=20.0.0'}
peerDependencies:
react: '>=18'
react-dom: '>=18'
react-router@7.14.2:
resolution: {integrity: sha512-yCqNne6I8IB6rVCH7XUvlBK7/QKyqypBFGv+8dj4QBFJiiRX+FG7/nkdAvGElyvVZ/HQP5N19wzteuTARXi5Gw==}
engines: {node: '>=20.0.0'}
peerDependencies:
react: '>=18'
react-dom: '>=18'
peerDependenciesMeta:
react-dom:
optional: true
react-stately@3.46.0: react-stately@3.46.0:
resolution: {integrity: sha512-OdxhWvHgs2L4OJGIs7hnuTr5WjjMM6enhNEAMRqiekhF8+ITvA2LRwNftOZwcogaoCslGYq5S2VQTQwnm0GbCA==} resolution: {integrity: sha512-OdxhWvHgs2L4OJGIs7hnuTr5WjjMM6enhNEAMRqiekhF8+ITvA2LRwNftOZwcogaoCslGYq5S2VQTQwnm0GbCA==}
peerDependencies: peerDependencies:
@@ -885,12 +995,24 @@ packages:
resolution: {integrity: sha512-llUJLzz1zTUBrskt2pwZgLq59AemifIftw4aB7JxOqf1HY2FDaGDxgwpAPVzHU1kdWabH7FauP4i1oEeer2WCA==} resolution: {integrity: sha512-llUJLzz1zTUBrskt2pwZgLq59AemifIftw4aB7JxOqf1HY2FDaGDxgwpAPVzHU1kdWabH7FauP4i1oEeer2WCA==}
engines: {node: '>=0.10.0'} engines: {node: '>=0.10.0'}
rehype-raw@7.0.0:
resolution: {integrity: sha512-/aE8hCfKlQeA8LmyeyQvQF3eBiLRGNlfBJEvWH7ivp9sBqs7TNqBL5X3v157rM4IFETqDnIOO+z5M/biZbo9Ww==}
rehype-sanitize@6.0.0:
resolution: {integrity: sha512-CsnhKNsyI8Tub6L4sm5ZFsme4puGfc6pYylvXo1AeqaGbjOYyzNv3qZPwvs0oMJ39eryyeOdmxwUIo94IpEhqg==}
remark-gfm@4.0.1:
resolution: {integrity: sha512-1quofZ2RQ9EWdeN34S79+KExV1764+wCUGop5CPL1WGdD0ocPpu91lzPGbwWMECpEpd42kJGQwzRfyov9j4yNg==}
remark-parse@11.0.0: remark-parse@11.0.0:
resolution: {integrity: sha512-FCxlKLNGknS5ba/1lmpYijMUzX2esxW5xQqjWxw2eHFfS2MSdaHVINFmhjo+qN1WhZhNimq0dZATN9pH0IDrpA==} resolution: {integrity: sha512-FCxlKLNGknS5ba/1lmpYijMUzX2esxW5xQqjWxw2eHFfS2MSdaHVINFmhjo+qN1WhZhNimq0dZATN9pH0IDrpA==}
remark-rehype@11.1.2: remark-rehype@11.1.2:
resolution: {integrity: sha512-Dh7l57ianaEoIpzbp0PC9UKAdCSVklD8E5Rpw7ETfbTl3FqcOOgq5q2LVDhgGCkaBv7p24JXikPdvhhmHvKMsw==} resolution: {integrity: sha512-Dh7l57ianaEoIpzbp0PC9UKAdCSVklD8E5Rpw7ETfbTl3FqcOOgq5q2LVDhgGCkaBv7p24JXikPdvhhmHvKMsw==}
remark-stringify@11.0.0:
resolution: {integrity: sha512-1OSmLd3awB/t8qdoEOMazZkNsfVTeY4fTsgzcQFdXNq8ToTN4ZGwrMnlda4K6smTFKD+GRV6O48i6Z4iKgPPpw==}
rollup@4.60.2: rollup@4.60.2:
resolution: {integrity: sha512-J9qZyW++QK/09NyN/zeO0dG/1GdGfyp9lV8ajHnRVLfo/uFsbji5mHnDgn/qYdUHyCkM2N+8VyspgZclfAh0eQ==} resolution: {integrity: sha512-J9qZyW++QK/09NyN/zeO0dG/1GdGfyp9lV8ajHnRVLfo/uFsbji5mHnDgn/qYdUHyCkM2N+8VyspgZclfAh0eQ==}
engines: {node: '>=18.0.0', npm: '>=8.0.0'} engines: {node: '>=18.0.0', npm: '>=8.0.0'}
@@ -899,6 +1021,9 @@ packages:
scheduler@0.27.0: scheduler@0.27.0:
resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==} resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==}
set-cookie-parser@2.7.2:
resolution: {integrity: sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==}
source-map-js@1.2.1: source-map-js@1.2.1:
resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==}
engines: {node: '>=0.10.0'} engines: {node: '>=0.10.0'}
@@ -966,6 +1091,9 @@ packages:
peerDependencies: peerDependencies:
react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
vfile-location@5.0.3:
resolution: {integrity: sha512-5yXvWDEgqeiYiBe1lbxYF7UMAIm/IcopxMHrMQDq3nvKcjPKIhZklUKL+AE7J7uApI4kwe2snsK+eI6UTj9EHg==}
vfile-message@4.0.3: vfile-message@4.0.3:
resolution: {integrity: sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==} resolution: {integrity: sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==}
@@ -1015,6 +1143,9 @@ packages:
warning@4.0.3: warning@4.0.3:
resolution: {integrity: sha512-rpJyN222KWIvHJ/F53XSZv0Zl/accqHR8et1kpaMTD/fLCRxtV8iX8czMzY7sVZupTI3zcUTg8eycS2kNF9l6w==} resolution: {integrity: sha512-rpJyN222KWIvHJ/F53XSZv0Zl/accqHR8et1kpaMTD/fLCRxtV8iX8czMzY7sVZupTI3zcUTg8eycS2kNF9l6w==}
web-namespaces@2.0.1:
resolution: {integrity: sha512-bKr1DkiNa2krS7qxNtdrtHAmzuYGFQLiQ13TsorsdT6ULTkPLKuu5+GsFpDlg6JFjUTwX2DyhMPG2be8uPrqsQ==}
zwitch@2.0.4: zwitch@2.0.4:
resolution: {integrity: sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A==} resolution: {integrity: sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A==}
@@ -1378,6 +1509,8 @@ snapshots:
comma-separated-tokens@2.0.3: {} comma-separated-tokens@2.0.3: {}
cookie@1.1.1: {}
csstype@3.2.3: {} csstype@3.2.3: {}
debug@4.4.3: debug@4.4.3:
@@ -1399,6 +1532,8 @@ snapshots:
'@babel/runtime': 7.29.2 '@babel/runtime': 7.29.2
csstype: 3.2.3 csstype: 3.2.3
entities@6.0.1: {}
esbuild@0.25.12: esbuild@0.25.12:
optionalDependencies: optionalDependencies:
'@esbuild/aix-ppc64': 0.25.12 '@esbuild/aix-ppc64': 0.25.12
@@ -1428,6 +1563,8 @@ snapshots:
'@esbuild/win32-ia32': 0.25.12 '@esbuild/win32-ia32': 0.25.12
'@esbuild/win32-x64': 0.25.12 '@esbuild/win32-x64': 0.25.12
escape-string-regexp@5.0.0: {}
estree-util-is-identifier-name@3.0.0: {} estree-util-is-identifier-name@3.0.0: {}
extend@3.0.2: {} extend@3.0.2: {}
@@ -1439,6 +1576,43 @@ snapshots:
fsevents@2.3.3: fsevents@2.3.3:
optional: true optional: true
hast-util-from-parse5@8.0.3:
dependencies:
'@types/hast': 3.0.4
'@types/unist': 3.0.3
devlop: 1.1.0
hastscript: 9.0.1
property-information: 7.1.0
vfile: 6.0.3
vfile-location: 5.0.3
web-namespaces: 2.0.1
hast-util-parse-selector@4.0.0:
dependencies:
'@types/hast': 3.0.4
hast-util-raw@9.1.0:
dependencies:
'@types/hast': 3.0.4
'@types/unist': 3.0.3
'@ungap/structured-clone': 1.3.0
hast-util-from-parse5: 8.0.3
hast-util-to-parse5: 8.0.1
html-void-elements: 3.0.0
mdast-util-to-hast: 13.2.1
parse5: 7.3.0
unist-util-position: 5.0.0
unist-util-visit: 5.1.0
vfile: 6.0.3
web-namespaces: 2.0.1
zwitch: 2.0.4
hast-util-sanitize@5.0.2:
dependencies:
'@types/hast': 3.0.4
'@ungap/structured-clone': 1.3.0
unist-util-position: 5.0.0
hast-util-to-jsx-runtime@2.3.6: hast-util-to-jsx-runtime@2.3.6:
dependencies: dependencies:
'@types/estree': 1.0.8 '@types/estree': 1.0.8
@@ -1459,12 +1633,32 @@ snapshots:
transitivePeerDependencies: transitivePeerDependencies:
- supports-color - supports-color
hast-util-to-parse5@8.0.1:
dependencies:
'@types/hast': 3.0.4
comma-separated-tokens: 2.0.3
devlop: 1.1.0
property-information: 7.1.0
space-separated-tokens: 2.0.2
web-namespaces: 2.0.1
zwitch: 2.0.4
hast-util-whitespace@3.0.0: hast-util-whitespace@3.0.0:
dependencies: dependencies:
'@types/hast': 3.0.4 '@types/hast': 3.0.4
hastscript@9.0.1:
dependencies:
'@types/hast': 3.0.4
comma-separated-tokens: 2.0.3
hast-util-parse-selector: 4.0.0
property-information: 7.1.0
space-separated-tokens: 2.0.2
html-url-attributes@3.0.1: {} html-url-attributes@3.0.1: {}
html-void-elements@3.0.0: {}
inline-style-parser@0.2.7: {} inline-style-parser@0.2.7: {}
invariant@2.2.4: invariant@2.2.4:
@@ -1492,6 +1686,15 @@ snapshots:
dependencies: dependencies:
js-tokens: 4.0.0 js-tokens: 4.0.0
markdown-table@3.0.4: {}
mdast-util-find-and-replace@3.0.2:
dependencies:
'@types/mdast': 4.0.4
escape-string-regexp: 5.0.0
unist-util-is: 6.0.1
unist-util-visit-parents: 6.0.2
mdast-util-from-markdown@2.0.3: mdast-util-from-markdown@2.0.3:
dependencies: dependencies:
'@types/mdast': 4.0.4 '@types/mdast': 4.0.4
@@ -1509,6 +1712,63 @@ snapshots:
transitivePeerDependencies: transitivePeerDependencies:
- supports-color - supports-color
mdast-util-gfm-autolink-literal@2.0.1:
dependencies:
'@types/mdast': 4.0.4
ccount: 2.0.1
devlop: 1.1.0
mdast-util-find-and-replace: 3.0.2
micromark-util-character: 2.1.1
mdast-util-gfm-footnote@2.1.0:
dependencies:
'@types/mdast': 4.0.4
devlop: 1.1.0
mdast-util-from-markdown: 2.0.3
mdast-util-to-markdown: 2.1.2
micromark-util-normalize-identifier: 2.0.1
transitivePeerDependencies:
- supports-color
mdast-util-gfm-strikethrough@2.0.0:
dependencies:
'@types/mdast': 4.0.4
mdast-util-from-markdown: 2.0.3
mdast-util-to-markdown: 2.1.2
transitivePeerDependencies:
- supports-color
mdast-util-gfm-table@2.0.0:
dependencies:
'@types/mdast': 4.0.4
devlop: 1.1.0
markdown-table: 3.0.4
mdast-util-from-markdown: 2.0.3
mdast-util-to-markdown: 2.1.2
transitivePeerDependencies:
- supports-color
mdast-util-gfm-task-list-item@2.0.0:
dependencies:
'@types/mdast': 4.0.4
devlop: 1.1.0
mdast-util-from-markdown: 2.0.3
mdast-util-to-markdown: 2.1.2
transitivePeerDependencies:
- supports-color
mdast-util-gfm@3.1.0:
dependencies:
mdast-util-from-markdown: 2.0.3
mdast-util-gfm-autolink-literal: 2.0.1
mdast-util-gfm-footnote: 2.1.0
mdast-util-gfm-strikethrough: 2.0.0
mdast-util-gfm-table: 2.0.0
mdast-util-gfm-task-list-item: 2.0.0
mdast-util-to-markdown: 2.1.2
transitivePeerDependencies:
- supports-color
mdast-util-mdx-expression@2.0.1: mdast-util-mdx-expression@2.0.1:
dependencies: dependencies:
'@types/estree-jsx': 1.0.5 '@types/estree-jsx': 1.0.5
@@ -1600,6 +1860,64 @@ snapshots:
micromark-util-symbol: 2.0.1 micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2 micromark-util-types: 2.0.2
micromark-extension-gfm-autolink-literal@2.1.0:
dependencies:
micromark-util-character: 2.1.1
micromark-util-sanitize-uri: 2.0.1
micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2
micromark-extension-gfm-footnote@2.1.0:
dependencies:
devlop: 1.1.0
micromark-core-commonmark: 2.0.3
micromark-factory-space: 2.0.1
micromark-util-character: 2.1.1
micromark-util-normalize-identifier: 2.0.1
micromark-util-sanitize-uri: 2.0.1
micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2
micromark-extension-gfm-strikethrough@2.1.0:
dependencies:
devlop: 1.1.0
micromark-util-chunked: 2.0.1
micromark-util-classify-character: 2.0.1
micromark-util-resolve-all: 2.0.1
micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2
micromark-extension-gfm-table@2.1.1:
dependencies:
devlop: 1.1.0
micromark-factory-space: 2.0.1
micromark-util-character: 2.1.1
micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2
micromark-extension-gfm-tagfilter@2.0.0:
dependencies:
micromark-util-types: 2.0.2
micromark-extension-gfm-task-list-item@2.1.0:
dependencies:
devlop: 1.1.0
micromark-factory-space: 2.0.1
micromark-util-character: 2.1.1
micromark-util-symbol: 2.0.1
micromark-util-types: 2.0.2
micromark-extension-gfm@3.0.0:
dependencies:
micromark-extension-gfm-autolink-literal: 2.1.0
micromark-extension-gfm-footnote: 2.1.0
micromark-extension-gfm-strikethrough: 2.1.0
micromark-extension-gfm-table: 2.1.1
micromark-extension-gfm-tagfilter: 2.0.0
micromark-extension-gfm-task-list-item: 2.1.0
micromark-util-combine-extensions: 2.0.1
micromark-util-types: 2.0.2
micromark-factory-destination@2.0.1: micromark-factory-destination@2.0.1:
dependencies: dependencies:
micromark-util-character: 2.1.1 micromark-util-character: 2.1.1
@@ -1730,6 +2048,10 @@ snapshots:
is-decimal: 2.0.1 is-decimal: 2.0.1
is-hexadecimal: 2.0.1 is-hexadecimal: 2.0.1
parse5@7.3.0:
dependencies:
entities: 6.0.1
picocolors@1.1.1: {} picocolors@1.1.1: {}
picomatch@4.0.4: {} picomatch@4.0.4: {}
@@ -1841,6 +2163,20 @@ snapshots:
transitivePeerDependencies: transitivePeerDependencies:
- supports-color - supports-color
react-router-dom@7.14.2(react-dom@19.2.5(react@19.2.5))(react@19.2.5):
dependencies:
react: 19.2.5
react-dom: 19.2.5(react@19.2.5)
react-router: 7.14.2(react-dom@19.2.5(react@19.2.5))(react@19.2.5)
react-router@7.14.2(react-dom@19.2.5(react@19.2.5))(react@19.2.5):
dependencies:
cookie: 1.1.1
react: 19.2.5
set-cookie-parser: 2.7.2
optionalDependencies:
react-dom: 19.2.5(react@19.2.5)
react-stately@3.46.0(react@19.2.5): react-stately@3.46.0(react@19.2.5):
dependencies: dependencies:
'@internationalized/date': 3.12.1 '@internationalized/date': 3.12.1
@@ -1870,6 +2206,28 @@ snapshots:
react@19.2.5: {} react@19.2.5: {}
rehype-raw@7.0.0:
dependencies:
'@types/hast': 3.0.4
hast-util-raw: 9.1.0
vfile: 6.0.3
rehype-sanitize@6.0.0:
dependencies:
'@types/hast': 3.0.4
hast-util-sanitize: 5.0.2
remark-gfm@4.0.1:
dependencies:
'@types/mdast': 4.0.4
mdast-util-gfm: 3.1.0
micromark-extension-gfm: 3.0.0
remark-parse: 11.0.0
remark-stringify: 11.0.0
unified: 11.0.5
transitivePeerDependencies:
- supports-color
remark-parse@11.0.0: remark-parse@11.0.0:
dependencies: dependencies:
'@types/mdast': 4.0.4 '@types/mdast': 4.0.4
@@ -1887,6 +2245,12 @@ snapshots:
unified: 11.0.5 unified: 11.0.5
vfile: 6.0.3 vfile: 6.0.3
remark-stringify@11.0.0:
dependencies:
'@types/mdast': 4.0.4
mdast-util-to-markdown: 2.1.2
unified: 11.0.5
rollup@4.60.2: rollup@4.60.2:
dependencies: dependencies:
'@types/estree': 1.0.8 '@types/estree': 1.0.8
@@ -1920,6 +2284,8 @@ snapshots:
scheduler@0.27.0: {} scheduler@0.27.0: {}
set-cookie-parser@2.7.2: {}
source-map-js@1.2.1: {} source-map-js@1.2.1: {}
space-separated-tokens@2.0.2: {} space-separated-tokens@2.0.2: {}
@@ -1999,6 +2365,11 @@ snapshots:
dependencies: dependencies:
react: 19.2.5 react: 19.2.5
vfile-location@5.0.3:
dependencies:
'@types/unist': 3.0.3
vfile: 6.0.3
vfile-message@4.0.3: vfile-message@4.0.3:
dependencies: dependencies:
'@types/unist': 3.0.3 '@types/unist': 3.0.3
@@ -2024,4 +2395,6 @@ snapshots:
dependencies: dependencies:
loose-envify: 1.4.0 loose-envify: 1.4.0
web-namespaces@2.0.1: {}
zwitch@2.0.4: {} zwitch@2.0.4: {}

BIN
ui/public/apple-touch-icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

BIN
ui/public/bean.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 134 KiB

BIN
ui/public/favicon-16.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

BIN
ui/public/favicon-32.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

BIN
ui/public/favicon-48.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.4 KiB

BIN
ui/public/favicon.ico Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

23
ui/public/gitea.svg Normal file
View File

@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="utf-8" ?>
<svg
xmlns="http://www.w3.org/2000/svg"
aria-label="Gitea"
role="img"
viewBox="0 0 512 512"
>
<rect rx="15%" height="512" width="512" fill="#ffffff" />
<path
d="M419 150c-98 7-186 2-276-1-27 0-63 19-61 67 3 75 71 82 99 83 3 14 35 62 59 65h104c63-5 109-213 75-214zm-311 67c-3-21 7-42 42-42 3 39 10 61 22 96-32-5-59-15-64-54z"
fill="#592"
/>
<path d="m293 152v70" stroke="#ffffff" stroke-width="9" />
<g transform="rotate(25.7 496 -423)" stroke-width="7" fill="#592">
<path d="M561 246h97" stroke="#592" />
<rect x="561" y="246" width="97" height="97" rx="16" fill="#ffffff" />
<path d="M592 245v75" stroke="#592" />
<path d="M592 273c45 0 38-5 38 48" fill="none" stroke="#592" />
<circle cx="592" cy="320" r="10" />
<circle cx="630" cy="320" r="10" />
<circle cx="592" cy="273" r="10" />
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 951 B

14
ui/public/github.svg Normal file
View File

@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="utf-8" ?>
<svg
fill="#ffffff"
width="800px"
height="800px"
viewBox="0 0 24 24"
role="img"
xmlns="http://www.w3.org/2000/svg"
>
<title>github</title>
<path
d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12"
/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 956 B

BIN
ui/public/icon-192.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 41 KiB

BIN
ui/public/icon-512.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 218 KiB

BIN
ui/public/icon-maskable-512.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 153 KiB

30
ui/public/mozilla.svg Normal file
View File

@@ -0,0 +1,30 @@
<?xml version="1.0" standalone="no" ?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg
width="800px"
height="800px"
viewBox="-10 -5 1034 1034"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="1.1"
>
<path
fill="#ffffff"
d="M411 237q-31 1 -64 10q-27 8 -54 22q-20 9 -38 20l-14 10q-50 33 -120 94q-67 57 -108 100q8 1 23 -5q12 -4 18 -5l99 -27l-32 26q-47 38 -69 59q-39 36 -52 62q11 -4 42 -19t46 -21q26 -11 40 -12l-18 16q-33 30 -49 46q-28 27 -38 44q1 1 1 4v2l10 -6q33 -20 52 -29
q32 -16 47 -18q-77 83 -89 125q19 -13 55 -32.5t55 -26.5q-3 8 -15 23q-9 12 -12.5 18.5t-15.5 21.5t-17 23q-8 14 -9 25q30 -29 73 -53q-4 10 -15 27q-10 15 -14 23q-7 13 -9 26q9 -10 27 -18q11 -5 33 -12l11 -4l-4 4q-27 21 -42 39l8 -1q2 -1 9 1l6 1l-32 13
q143 188 395 166l-13 -28q10 -18 19 -63t19 -66q16 -34 48 -51l25 -14q24 -14 37 -17q18 -4 49 2q8 1 32 7q35 9 52 12q30 6 46 2l10 -5q36 -22 58 -39q24 5 34.5 1.5t18.5 -16.5l5 -14q18 -50 28 -72l1 -15q-7 -21 -11 -75q-2 -32 -5 -36q-13 -18 -45 -34q-20 -10 -67 -27
q-54 -20 -79 -33q-43 -21 -66 -47q8 -25 -5 -51q-10 -19 -32 -40q-25 -18 -63 -16q-24 1 -64 12q-30 -6 -86 -24q-29 -9 -44 -14q-8 -1 -18 -1h-4zM410 285q22 0 64 12q27 7 41 9q-1 4 -6.5 8t-6.5 7l-4 6q-4 5 -5.5 8.5t0.5 7.5l32 -12q13 -4 38 -12q32 -11 48 -14
q28 -6 51 -3l24 31q-24 -20 -55 -15q-19 3 -56 20q-33 15 -49 18q-11 2 -33 8q-17 5 -25 7q29 22 75 17q3 15 18 34.5t29 22.5q-1 -6 -7 -20.5t-7 -22.5q-3 -14 4 -25l9 -6q17 -10 26 -13q16 -5 27 4q-10 1 -26 11l-8 5q-6 4 -8.5 14t-0.5 14q7 20 39 33q36 14 90 14
q-3 -4 -16 -18.5t-18 -21.5q-9 -11 -8 -17q30 30 81 55q30 14 89 35q35 12 49 18q22 10 31 19q1 5 -13 22q-7 9 -9 13q-4 7 -2 10q-3 7 7 22q5 8 22 29l3 4q-3 -25 -1.5 -36.5t6 -9.5t8.5 14.5t3 28.5q0 19 -7 37l-35 -8q-47 -3 -84 -15q-33 -9 -66 -28q-21 -12 -65 -42
l-30 -20q-27 -18 -79 -60l-14 -11l-50 -19q14 17 36 41q20 22 25 30q7 12 5 24.5t-16 36.5l-17 -31l-7 -55q-11 40 -13 66q-2 36 13 62q17 30 58 49l50 7q-22 -18 -30 -29q-12 -16 -8 -31q15 30 78 55q35 14 105 32l24 6q15 6 24 15q-7 6 -13 9q-4 2 -13 4.5t-15 4.5l-28 -9
q-36 -11 -51 -18q-26 -11 -36 -25q4 6 -27 14q-18 4 -63 13q-29 5 -37 7q-13 3 -3 4q-35 -1 -72 -23q-33 -19 -61 -50l-9 -3q0 17 11 35q6 11 22.5 32t22.5 31l38 17l-10 6q-16 10 -23 16q-13 10 -19 21l-2 6q-5 9 -6 14q-2 8 1 14q21 -23 61 -33l-16 25q2 18 -9 49
q-2 -5 -5 -10q-47 -20 -71 -39q6 20 25 44q16 21 37 38q-5 12 -12 23q-36 -25 -54 -42q5 9 11 27q9 24 15 34q-133 -10 -217 -82l60 -34q-10 2 -47 11l-30 8l-7 -8l24 -12q29 -14 42 -22q23 -14 29 -26v0l-6 2l-61 3q-4 -13 0 -27q2 -9 10.5 -24.5t9.5 -23.5q2 -13 -7 -25
q-35 -46 -49 -80q-20 -46 -15 -90l1 2q7 15 13 20l2 -17q2 -18 5 -26v1q6 20 11 29q8 16 21 24l1 -2q-14 -50 -2 -95q13 -51 56 -78q-6 2 -23 4q-26 4 -41 10q2 -1 5 -10q11 -29 23 -39q6 -4 15 -13q11 -10 19 -15q24 -19 43 -28q26 -14 59 -19q8 -1 16 -1h4zM425 332
q-28 0 -60 12l-13 10q-14 17 -19 26q-10 14 -12 29l3 4q13 -16 37 -28q0 18 9.5 33.5t25.5 24.5h3q-3 -19 -1 -39.5t9 -37.5q7 -6 23 -14q18 -10 25 -17q-14 -4 -30 -3zM625 391v0q8 0 15 3q-6 3 -8 11q-1 4 0 7q-11 -8 -11 -21h4zM462 392q2 7 14 22l9 11q-28 2 -60 11
q5 4 14.5 9t13.5 9l-16 6q-21 7 -31 11q-16 8 -28 19l5 2q31 -6 66 -5.5t67 7.5l3 -2l-25 -46l38 -8q-39 -37 -70 -46zM662 406q8 7 15 16q-11 3 -22 0q5 -3 7 -10v-6zM599 430q13 39 37 41q33 6 62 1q-12 -7 -39 -13q-22 -6 -32 -10q-17 -7 -28 -19zM368 492
q-23 17 -32.5 42t-5.5 53q2 7 1 17q-1 6 -4 17q-4 17 -3 25q0 13 8 22q22 25 63 55l-2 -8q-35 -41 -46 -74l17 6q36 13 55 16q-6 -10 -22 -29q-28 -34 -36 -53q-14 -32 -4 -62q0 -5 6 -12q7 -10 5 -15zM728 621q2 0 5 1q8 2 9 4q-3 2 -8.5 7t-8.5 6l1 -6q0 -9 2 -12z
M768 642v0q4 0 9 4h1l-13 13l1 -17h2zM798 653v0l10 6q-1 0 -6 7t-5.5 6.5t-0.5 -6.5q0 -13 2 -13zM829 667v0l9 2q-1 1 -4 7q-6 11 -11 11l4 -8v-9q0 -3 2 -3zM861 673l12 3l-13 22zM896 679q1 0 8 2l4 2l-15 11q-1 -9 -0.5 -12t3.5 -3zM933 685q5 0 7 2q0 5 -7 12
q-4 4 -5 7v-6q2 -10 0 -15h5z"
/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.8 KiB

14
ui/public/robots.txt Normal file
View File

@@ -0,0 +1,14 @@
User-agent: *
Allow: /
User-agent: facebookexternalhit
Allow: /
User-agent: Twitterbot
Allow: /
User-agent: LinkedInBot
Allow: /
User-agent: WhatsApp
Allow: /

149
ui/src/App.css
View File

@@ -1,6 +1,7 @@
body { body {
background-color: #2c3e50; background-color: #2c3e50;
color: #ecf0f1; color: #ecf0f1;
text-transform: lowercase;
} }
.container { .container {
@@ -33,6 +34,154 @@ a.hot-pink {
color: #2c3e50; color: #2c3e50;
} }
.vertical-timeline-element-content h4.vertical-timeline-element-title {
font-size: 0.85rem;
}
.vertical-timeline-element-content h5.vertical-timeline-element-subtitle {
font-size: 0.75rem;
}
.vertical-timeline-element-content p,
.vertical-timeline-element-content ul,
.vertical-timeline-element-content li,
.vertical-timeline-element-content code {
font-size: 0.75rem;
}
.vertical-timeline-element-content a { .vertical-timeline-element-content a {
color: #1565c0; color: #1565c0;
} }
.site-header h1 {
font-size: 1.75rem;
}
.site-header nav a {
color: #ecf0f1;
opacity: 0.7;
}
.site-header nav a:hover {
color: #ff4081;
opacity: 1;
text-decoration: none;
}
.site-header nav a.active {
color: #ff4081;
opacity: 1;
}
.nav-divider {
opacity: 0.3;
}
.graph-label {
fill: #ecf0f1;
font-size: 9px;
opacity: 0.6;
}
.graph-cell {
cursor: pointer;
transition: opacity 0.15s;
}
.graph-cell:hover {
opacity: 0.8;
stroke: #ecf0f1;
stroke-width: 1;
}
.project-card {
background: rgba(255, 255, 255, 0.05);
border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 6px;
height: 100%;
}
.project-card h5 {
font-size: 0.9rem;
}
.forge-icon {
width: 16px;
height: 16px;
margin-right: 6px;
vertical-align: -2px;
opacity: 0.7;
}
.project-card a {
color: #ff4081;
}
.project-card .text-muted {
color: rgba(236, 240, 241, 0.5) !important;
font-size: 0.7rem;
}
.project-card h5,
.project-card .text-muted,
.project-card span {
color: #ecf0f1;
}
.language-bar {
display: flex;
height: 6px;
border-radius: 3px;
overflow: hidden;
}
.language-bar-segment {
min-width: 2px;
}
.language-dot {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 4px;
}
.project-readme {
background: rgba(255, 255, 255, 0.05);
border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 6px;
padding: 1.5rem;
font-size: 0.85rem;
line-height: 1.5;
}
.project-readme h1,
.project-readme h2,
.project-readme h3 {
font-size: 1.1rem;
margin-top: 1rem;
}
.project-readme pre {
background: rgba(0, 0, 0, 0.2);
padding: 0.75rem;
border-radius: 4px;
overflow-x: auto;
}
.project-readme code {
font-size: 0.8rem;
}
.project-readme img {
max-width: 100%;
}
.site-footer {
margin-top: 3rem;
padding: 1rem 0;
font-size: 0.75rem;
opacity: 0.6;
text-align: center;
}

137
ui/src/App.tsx
View File

@@ -1,132 +1,27 @@
import { useMemo, useState } from 'react'; import { Routes, Route } from 'react-router-dom';
import { useQuery } from '@tanstack/react-query';
import Col from 'react-bootstrap/Col';
import Container from 'react-bootstrap/Container';
import Row from 'react-bootstrap/Row';
import { VerticalTimeline } from 'react-vertical-timeline-component';
import 'bootstrap/dist/css/bootstrap.min.css'; import 'bootstrap/dist/css/bootstrap.min.css';
import 'rc-slider/assets/index.css'; import 'rc-slider/assets/index.css';
import 'react-vertical-timeline-component/style.min.css'; import 'react-vertical-timeline-component/style.min.css';
import './App.css'; import './App.css';
import { fetchEvents, fetchSources, type Source } from './api/client'; import { Layout } from './components/Layout';
import { Filters } from './components/Filters'; import { DashPage } from './pages/DashPage';
import { TimelineEntry } from './components/TimelineEntry'; import { TimelineHome } from './pages/TimelineHome';
import { ProjectPage } from './pages/ProjectPage';
const RANGE_MIN = new Date('2010-01-01T00:00:00Z').getTime(); import { CvPage } from './pages/CvPage';
const RANGE_MAX = Date.now();
const externalLinks: { url: string; alt: string }[] = [
{ url: 'https://linkedin.com/in/thijssen/', alt: 'linkedin' },
{ url: 'https://stackoverflow.com/users/68115/grenade', alt: 'stackoverflow' },
{ url: 'https://github.com/grenade', alt: 'github' },
{ url: 'https://git.lair.cafe/grenade', alt: 'gitea' },
];
export default function App() { export default function App() {
const [enabledSources, setEnabledSources] = useState<Record<Source, boolean>>({
github: true,
gitea: true,
hg: true,
bugzilla: true,
});
const [rangeValue, setRangeValue] = useState<[number, number]>(() => {
const now = Date.now();
const thirtyDaysAgo = now - 30 * 24 * 60 * 60 * 1000;
return [thirtyDaysAgo, now];
});
const [limit, setLimit] = useState<number>(100);
const sourcesQ = useQuery({
queryKey: ['sources'],
queryFn: fetchSources,
refetchInterval: 60_000,
});
const activeSources = useMemo(
() =>
(Object.keys(enabledSources) as Source[]).filter((s) => enabledSources[s]),
[enabledSources],
);
const eventsQ = useQuery({
queryKey: ['events', rangeValue, activeSources, limit],
queryFn: () =>
fetchEvents({
from: new Date(rangeValue[0]),
to: new Date(rangeValue[1]),
sources: activeSources,
limit,
}),
refetchInterval: 60_000,
});
const events = eventsQ.data ?? [];
return ( return (
<Container className="py-4"> <Routes>
<Row className="mb-3"> <Route element={<Layout />}>
<Col> <Route index element={<DashPage />} />
<h1>hi, i'm rob</h1> <Route path="/dash" element={<DashPage />} />
</Col> <Route path="/activity" element={<TimelineHome />} />
<Col className="d-flex flex-wrap gap-3 justify-content-end align-items-center"> <Route path="/activity/:timespan" element={<TimelineHome />} />
{externalLinks.map((el) => ( <Route path="/project/:source/*" element={<ProjectPage />} />
<a <Route path="/cv" element={<CvPage />} />
key={el.url} </Route>
href={el.url} </Routes>
title={el.alt}
target="_blank"
rel="noopener noreferrer"
>
{el.alt}
</a>
))}
</Col>
</Row>
<Row className="mb-4">
<Col>
<p>
i rarely say anything that warrants capital letters. if you're here
to see my resume, please go to{' '}
<a className="hot-pink" href="https://rob.tn/cv/">
https://rob.tn/cv
</a>
. a peek into the projects i'm working on is below.
</p>
</Col>
</Row>
<Filters
enabledSources={enabledSources}
onSourceToggle={(s, on) =>
setEnabledSources((prev) => ({ ...prev, [s]: on }))
}
rangeMin={RANGE_MIN}
rangeMax={RANGE_MAX}
rangeValue={rangeValue}
onRangeChange={setRangeValue}
limit={limit}
onLimitChange={setLimit}
summaries={sourcesQ.data}
/>
<Row>
<Col>
<p className="text-center" style={{ fontSize: '85%' }}>
{eventsQ.isLoading
? 'loading…'
: eventsQ.isError
? `error: ${(eventsQ.error as Error).message}`
: `showing ${events.length} ${events.length === 1 ? 'activity' : 'activities'}`}
</p>
<VerticalTimeline>
{events.map((item) => (
<TimelineEntry key={item.id} item={item} />
))}
</VerticalTimeline>
</Col>
</Row>
</Container>
); );
} }

101
ui/src/api/client.ts
View File

@@ -54,15 +54,50 @@ export interface SourceSummary {
latest: string | null; latest: string | null;
} }
export interface ProjectSummary {
repo: string;
source: Source;
host: string;
commit_count: number;
issue_count: number;
pr_count: number;
first_activity: string | null;
last_activity: string | null;
}
export interface DailyCount {
date: string;
count: number;
}
export interface HourlyAvg {
hour: number;
avg: number;
}
export interface LanguageDailyCount {
date: string;
language: string;
color: string | null;
commits: number;
}
export interface EventQuery { export interface EventQuery {
from?: Date; from?: Date;
to?: Date; to?: Date;
sources?: Source[]; sources?: Source[];
repo?: string;
limit?: number; limit?: number;
} }
const API_BASE = '/api/v1'; const API_BASE = '/api/v1';
/** Decode base64 content as UTF-8 (atob only handles Latin-1). */
function decodeBase64Utf8(b64: string): string {
const bytes = Uint8Array.from(atob(b64), (c) => c.charCodeAt(0));
return new TextDecoder().decode(bytes);
}
export async function fetchEvents(q: EventQuery): Promise<TimelineItem[]> { export async function fetchEvents(q: EventQuery): Promise<TimelineItem[]> {
const params = new URLSearchParams(); const params = new URLSearchParams();
if (q.from) params.set('from', q.from.toISOString()); if (q.from) params.set('from', q.from.toISOString());
@@ -70,6 +105,7 @@ export async function fetchEvents(q: EventQuery): Promise<TimelineItem[]> {
if (q.sources && q.sources.length > 0) { if (q.sources && q.sources.length > 0) {
params.set('source', q.sources.join(',')); params.set('source', q.sources.join(','));
} }
if (q.repo) params.set('repo', q.repo);
if (q.limit) params.set('limit', String(q.limit)); if (q.limit) params.set('limit', String(q.limit));
const resp = await fetch(`${API_BASE}/events?${params}`); const resp = await fetch(`${API_BASE}/events?${params}`);
@@ -82,3 +118,68 @@ export async function fetchSources(): Promise<SourceSummary[]> {
if (!resp.ok) throw new Error(`sources: HTTP ${resp.status}`); if (!resp.ok) throw new Error(`sources: HTTP ${resp.status}`);
return resp.json(); return resp.json();
} }
export async function fetchDailyCounts(from: string, to: string): Promise<DailyCount[]> {
const resp = await fetch(`${API_BASE}/activity/daily?from=${from}&to=${to}`);
if (!resp.ok) throw new Error(`daily-counts: HTTP ${resp.status}`);
return resp.json();
}
export async function fetchHourlyAvgs(from: string, to: string, tz: string): Promise<HourlyAvg[]> {
const qs = new URLSearchParams({ from, to, tz });
const resp = await fetch(`${API_BASE}/activity/hourly?${qs}`);
if (!resp.ok) throw new Error(`hourly-avgs: HTTP ${resp.status}`);
return resp.json();
}
export async function fetchLanguageDailyCounts(from: string, to: string): Promise<LanguageDailyCount[]> {
const resp = await fetch(`${API_BASE}/languages/daily?from=${from}&to=${to}`);
if (!resp.ok) throw new Error(`language-daily-counts: HTTP ${resp.status}`);
return resp.json();
}
export interface RepoLanguageEntry {
source: Source;
repo: string;
language: string;
bytes: number;
color: string | null;
}
export async function fetchRepoLanguages(): Promise<RepoLanguageEntry[]> {
const resp = await fetch(`${API_BASE}/languages/repos`);
if (!resp.ok) throw new Error(`repo-languages: HTTP ${resp.status}`);
return resp.json();
}
export async function fetchProjects(): Promise<ProjectSummary[]> {
const resp = await fetch(`${API_BASE}/projects`);
if (!resp.ok) throw new Error(`projects: HTTP ${resp.status}`);
return resp.json();
}
/** Fetch repo README as raw markdown via the forge proxy. */
export async function fetchReadme(source: Source, host: string, repo: string): Promise<string | null> {
if (source === 'github') {
const resp = await fetch(`${API_BASE}/forge/github/repos/${repo}/readme`);
if (!resp.ok) return null;
const data = await resp.json();
if (data.encoding === 'base64' && data.content) {
return decodeBase64Utf8(data.content);
}
return data.content ?? null;
}
if (source === 'gitea') {
for (const name of ['README.md', 'readme.md', 'Readme.md']) {
const resp = await fetch(`${API_BASE}/forge/gitea/repos/${repo}/contents/${name}?host=${encodeURIComponent(host)}`);
if (!resp.ok) continue;
const data = await resp.json();
if (data.encoding === 'base64' && data.content) {
return decodeBase64Utf8(data.content);
}
if (data.content) return data.content;
}
return null;
}
return null;
}

82
ui/src/api/cv.ts Normal file
View File

@@ -0,0 +1,82 @@
// Fetches the CV gist at runtime and returns the parsed config + file
// list. The legacy implementation (cv/src/App.js) hits the same endpoint
// and relies entirely on the inline `content` field — no per-file raw_url
// fetches. We do the same: one request, dedup'd via TanStack Query.
const GIST_OWNER = 'grenade';
const GIST_ID = '8e487477663c8e57c7bf31e8371f454a';
const GIST_API_URL = `https://api.github.com/gists/${GIST_ID}`;
const GIST_RAW_BASE = `https://gist.githubusercontent.com/${GIST_OWNER}/${GIST_ID}/raw`;
export const CV_PHOTO_URL = `${GIST_RAW_BASE}/rob.png`;
const CONFIG_FILENAME = 'cv-config.json';
export type SectionPlacement = 'body' | 'nav';
export type SortDirection = 'ascending' | 'descending';
export interface CvSectionConfig {
name: string;
filename_prefix: string;
order: number;
show_section_name: boolean;
placement: SectionPlacement;
sort?: {
on: 'filename';
direction: SortDirection;
};
}
export interface CvConfig {
sections: CvSectionConfig[];
}
export interface GistFile {
filename: string;
type: string;
language: string | null;
raw_url: string;
size: number;
truncated: boolean;
content: string;
}
interface GistResponse {
files: Record<string, GistFile>;
}
export interface CvData {
config: CvConfig;
files: Record<string, GistFile>;
}
export async function fetchCv(): Promise<CvData> {
const resp = await fetch(GIST_API_URL);
if (!resp.ok) {
throw new Error(`gist: HTTP ${resp.status} ${resp.statusText}`);
}
const gist = (await resp.json()) as GistResponse;
const cfgFile = gist.files[CONFIG_FILENAME];
if (!cfgFile) {
throw new Error(`gist: missing ${CONFIG_FILENAME}`);
}
const config = JSON.parse(cfgFile.content) as CvConfig;
return { config, files: gist.files };
}
// Pick out the gist files whose names start with the given prefix, applying
// the section's sort order. Mirrors the legacy filter at cv/src/App.js:67-68.
export function filesForSection(
data: CvData,
section: CvSectionConfig,
): GistFile[] {
const matches = Object.keys(data.files)
.filter((name) => name.startsWith(section.filename_prefix))
.sort();
if (section.sort?.direction === 'descending') {
matches.reverse();
}
return matches.map((name) => data.files[name]);
}

464
ui/src/components/ContributionGraph.tsx Normal file
View File

@@ -0,0 +1,464 @@
import { useMemo } from "react";
import { useQuery } from "@tanstack/react-query";
import { useNavigate } from "react-router-dom";
import {
fetchDailyCounts,
fetchLanguageDailyCounts,
fetchProjects,
fetchSources,
} from "../api/client";
const CELL_SIZE = 12;
const GAP = 3;
const RADIUS = CELL_SIZE / 2;
const ROWS = 7;
const LEFT_LABEL_WIDTH = 28;
const TOP_LABEL_HEIGHT = 16;
const DAY_LABELS = ["", "mon", "", "wed", "", "fri", ""];
const MONTH_LABELS = [
"jan",
"feb",
"mar",
"apr",
"may",
"jun",
"jul",
"aug",
"sep",
"oct",
"nov",
"dec",
];
const EMPTY_COLOR = "rgba(255,255,255,0.05)";
const FALLBACK_COLOR = "#39d353";
/** Daily contribution graph — last 1 year, one circle per day. */
export function ContributionGraph() {
const to = new Date();
const from = new Date(to);
from.setFullYear(from.getFullYear() - 1);
const fromStr = fmt(from);
const toStr = fmt(to);
const dailyQ = useQuery({
queryKey: ["daily-counts", fromStr, toStr],
queryFn: () => fetchDailyCounts(fromStr, toStr),
staleTime: 5 * 60_000,
});
const langQ = useQuery({
queryKey: ["language-daily", fromStr, toStr],
queryFn: () => fetchLanguageDailyCounts(fromStr, toStr),
staleTime: 5 * 60_000,
});
const projectsQ = useQuery({
queryKey: ["projects"],
queryFn: fetchProjects,
staleTime: 60_000,
});
const repoCount = useMemo(() => {
if (!projectsQ.data) return 0;
const fromMs = from.getTime();
const toMs = to.getTime();
return projectsQ.data.filter((p) => {
const first = p.first_activity
? new Date(p.first_activity).getTime()
: Infinity;
const last = p.last_activity ? new Date(p.last_activity).getTime() : 0;
return last >= fromMs && first <= toMs;
}).length;
}, [projectsQ.data]);
const navigate = useNavigate();
// Build map of date → dominant language color
const dayColorMap = useMemo(() => {
return buildDominantColorMap(langQ.data ?? []);
}, [langQ.data]);
const { weeks, monthMarkers, thresholds, totalCount } = useMemo(() => {
const counts = dailyQ.data ?? [];
const countMap = new Map(counts.map((d) => [d.date, d.count]));
const start = new Date(from);
start.setDate(start.getDate() - start.getDay());
const weeks: { date: string; count: number; col: number; row: number }[][] =
[];
const monthMarkers: { col: number; label: string }[] = [];
let col = 0;
let prevMonth = -1;
const cursor = new Date(start);
while (cursor <= to) {
const week: (typeof weeks)[0] = [];
for (let row = 0; row < ROWS; row++) {
const dateStr = fmt(cursor);
const count = countMap.get(dateStr) ?? 0;
week.push({ date: dateStr, count, col, row });
if (row === 0) {
const m = cursor.getMonth();
if (m !== prevMonth) {
monthMarkers.push({ col, label: MONTH_LABELS[m] });
prevMonth = m;
}
}
cursor.setDate(cursor.getDate() + 1);
}
weeks.push(week);
col++;
}
const nonZero = counts
.map((d) => d.count)
.filter((c) => c > 0)
.sort((a, b) => a - b);
const thresholds = computeThresholds(nonZero);
const totalCount = counts.reduce((sum, d) => sum + d.count, 0);
return { weeks, monthMarkers, thresholds, totalCount };
}, [dailyQ.data]);
const cols = weeks.length;
const svgWidth = LEFT_LABEL_WIDTH + cols * (CELL_SIZE + GAP);
const svgHeight = TOP_LABEL_HEIGHT + ROWS * (CELL_SIZE + GAP);
if (dailyQ.isLoading)
return <p style={{ fontSize: "0.8rem" }}>loading contribution graph...</p>;
if (dailyQ.isError) return null;
return (
<div className="contribution-graph mb-3">
<p style={{ fontSize: "0.8rem", opacity: 0.6 }}>
{new Intl.NumberFormat().format(totalCount)} contributions
{repoCount > 0 && `, across ${repoCount} repositories, `}
in the last year
</p>
<div>
<svg
viewBox={`0 0 ${svgWidth} ${svgHeight}`}
width="100%"
className="d-block"
>
{DAY_LABELS.map((label, i) =>
label ? (
<text
key={i}
x={LEFT_LABEL_WIDTH - 6}
y={TOP_LABEL_HEIGHT + i * (CELL_SIZE + GAP) + CELL_SIZE / 2}
textAnchor="end"
dominantBaseline="central"
className="graph-label"
>
{label}
</text>
) : null,
)}
{monthMarkers.map(({ col, label }, i) => (
<text
key={i}
x={LEFT_LABEL_WIDTH + col * (CELL_SIZE + GAP) + RADIUS}
y={10}
textAnchor="middle"
className="graph-label"
>
{label}
</text>
))}
{weeks.flatMap((week) =>
week.map(({ date, count, col, row }) => (
<circle
key={date}
cx={LEFT_LABEL_WIDTH + col * (CELL_SIZE + GAP) + RADIUS}
cy={TOP_LABEL_HEIGHT + row * (CELL_SIZE + GAP) + RADIUS}
r={RADIUS - 1}
fill={
count === 0
? EMPTY_COLOR
: (dayColorMap.get(date) ?? FALLBACK_COLOR)
}
opacity={count === 0 ? 1 : opacityFor(count, thresholds)}
className="graph-cell"
onClick={() => navigate(`/activity/${date}`)}
>
<title>{`${date}: ${count} ${count === 1 ? "contribution" : "contributions"}`}</title>
</circle>
)),
)}
</svg>
</div>
</div>
);
}
/** All-time monthly contribution graph — years on X axis, months on Y axis. */
export function AllTimeGraph() {
const sourcesQ = useQuery({
queryKey: ["sources"],
queryFn: fetchSources,
staleTime: 60_000,
});
const earliest = useMemo(() => {
if (!sourcesQ.data) return null;
const dates = sourcesQ.data
.map((s) => s.earliest)
.filter((d): d is string => d != null)
.map((d) => new Date(d));
return dates.length > 0
? new Date(Math.min(...dates.map((d) => d.getTime())))
: null;
}, [sourcesQ.data]);
const projectsQ = useQuery({
queryKey: ["projects"],
queryFn: fetchProjects,
staleTime: 60_000,
});
const repoCount = projectsQ.data?.length ?? 0;
const to = new Date();
const from = earliest ?? new Date(to.getFullYear() - 5, 0, 1);
const fromStr = fmt(from);
const toStr = fmt(to);
const dailyQ = useQuery({
queryKey: ["daily-counts-alltime", fromStr, toStr],
queryFn: () => fetchDailyCounts(fromStr, toStr),
enabled: !!earliest,
staleTime: 10 * 60_000,
});
const langQ = useQuery({
queryKey: ["language-daily-alltime", fromStr, toStr],
queryFn: () => fetchLanguageDailyCounts(fromStr, toStr),
enabled: !!earliest,
staleTime: 10 * 60_000,
});
const navigate = useNavigate();
// Aggregate daily language data to month level: pick the language with most commits
const monthColorMap = useMemo(() => {
const entries = langQ.data ?? [];
if (entries.length === 0) return new Map<string, string>();
const map = new Map<
string,
Map<string, { commits: number; color: string }>
>();
for (const e of entries) {
const key = e.date.slice(0, 7); // YYYY-MM
if (!map.has(key)) map.set(key, new Map());
const langMap = map.get(key)!;
const cur = langMap.get(e.language);
if (cur) {
cur.commits += e.commits;
} else {
langMap.set(e.language, {
commits: e.commits,
color: e.color ?? FALLBACK_COLOR,
});
}
}
const result = new Map<string, string>();
for (const [key, langMap] of map) {
let best = { commits: 0, color: FALLBACK_COLOR };
for (const v of langMap.values()) {
if (v.commits > best.commits) best = v;
}
result.set(key, best.color);
}
return result;
}, [langQ.data]);
const { years, monthGrid, thresholds, totalCount } = useMemo(() => {
const counts = dailyQ.data ?? [];
if (counts.length === 0)
return { years: [], monthGrid: [], thresholds: [1, 2, 3], totalCount: 0 };
const countMap = new Map(counts.map((d) => [d.date, d.count]));
const startYear = from.getFullYear();
const endYear = to.getFullYear();
const years: number[] = [];
for (let yr = startYear; yr <= endYear; yr++) years.push(yr);
// Build a 12 x years grid of monthly totals
const monthGrid: {
year: number;
month: number;
count: number;
monthStart: string;
monthEnd: string;
monthKey: string;
}[][] = [];
for (let m = 0; m < 12; m++) {
const row: (typeof monthGrid)[0] = [];
for (const yr of years) {
const monthStart = new Date(yr, m, 1);
const monthEnd = new Date(yr, m + 1, 0); // last day of month
const monthKey = `${yr}-${String(m + 1).padStart(2, "0")}`;
// Don't include months entirely outside our data range
if (monthStart > to || monthEnd < from) {
row.push({
year: yr,
month: m,
count: 0,
monthStart: fmt(monthStart),
monthEnd: fmt(monthEnd),
monthKey,
});
continue;
}
let total = 0;
const cursor = new Date(monthStart);
while (cursor <= monthEnd && cursor <= to) {
total += countMap.get(fmt(cursor)) ?? 0;
cursor.setDate(cursor.getDate() + 1);
}
row.push({
year: yr,
month: m,
count: total,
monthStart: fmt(monthStart),
monthEnd: fmt(monthEnd),
monthKey,
});
}
monthGrid.push(row);
}
const allCounts = monthGrid.flat().map((c) => c.count);
const nonZero = allCounts.filter((c) => c > 0).sort((a, b) => a - b);
const thresholds = computeThresholds(nonZero);
const totalCount = counts.reduce((sum, d) => sum + d.count, 0);
return { years, monthGrid, thresholds, totalCount };
}, [dailyQ.data]);
if (!earliest || dailyQ.isLoading) return null;
if (dailyQ.isError) return null;
if (years.length === 0) return null;
const monthLabelWidth = 28;
const topLabelHeight = 16;
const numCols = years.length;
const svgWidth = monthLabelWidth + numCols * (CELL_SIZE + GAP);
const svgHeight = topLabelHeight + 12 * (CELL_SIZE + GAP);
return (
<div className="contribution-graph mb-4">
<p style={{ fontSize: "0.8rem", opacity: 0.6 }}>
{new Intl.NumberFormat().format(totalCount)} contributions
{repoCount > 0 && `, across ${repoCount} repos, `}
since {fmt(from).split("-")[0]}
</p>
<div>
<svg
viewBox={`0 0 ${svgWidth} ${svgHeight}`}
width="100%"
className="d-block"
>
{/* Year labels along the top */}
{years.map((year, colIdx) => (
<text
key={year}
x={monthLabelWidth + colIdx * (CELL_SIZE + GAP) + RADIUS}
y={10}
textAnchor="middle"
className="graph-label"
>
{String(year).slice(2)}
</text>
))}
{/* Month labels along the left */}
{MONTH_LABELS.map((label, rowIdx) => (
<text
key={rowIdx}
x={monthLabelWidth - 6}
y={topLabelHeight + rowIdx * (CELL_SIZE + GAP) + CELL_SIZE / 2}
textAnchor="end"
dominantBaseline="central"
className="graph-label"
>
{label}
</text>
))}
{/* Monthly contribution circles */}
{monthGrid.map((row, rowIdx) =>
row.map(
({ year, count, monthStart, monthEnd, monthKey }, colIdx) => (
<circle
key={`${year}-${rowIdx}`}
cx={monthLabelWidth + colIdx * (CELL_SIZE + GAP) + RADIUS}
cy={topLabelHeight + rowIdx * (CELL_SIZE + GAP) + RADIUS}
r={RADIUS - 1}
fill={
count === 0
? EMPTY_COLOR
: (monthColorMap.get(monthKey) ?? FALLBACK_COLOR)
}
opacity={count === 0 ? 1 : opacityFor(count, thresholds)}
className="graph-cell"
onClick={() =>
navigate(`/activity/${monthStart}..${monthEnd}`)
}
>
<title>{`${MONTH_LABELS[rowIdx]} ${year}: ${count} ${count === 1 ? "contribution" : "contributions"}`}</title>
</circle>
),
),
)}
</svg>
</div>
</div>
);
}
function fmt(d: Date): string {
return d.toISOString().slice(0, 10);
}
/** Build a map of date → dominant (highest commit count) language color. */
function buildDominantColorMap(
entries: {
date: string;
language: string;
color: string | null;
commits: number;
}[],
): Map<string, string> {
const map = new Map<string, { commits: number; color: string }>();
for (const e of entries) {
const cur = map.get(e.date);
if (!cur || e.commits > cur.commits) {
map.set(e.date, { commits: e.commits, color: e.color ?? FALLBACK_COLOR });
}
}
const result = new Map<string, string>();
for (const [date, { color }] of map) {
result.set(date, color);
}
return result;
}
/** Map count to opacity (0.3 – 1.0) based on quartile thresholds. */
function opacityFor(count: number, thresholds: number[]): number {
if (count <= thresholds[0]) return 0.35;
if (count <= thresholds[1]) return 0.55;
if (count <= thresholds[2]) return 0.75;
return 1;
}
function computeThresholds(sorted: number[]): number[] {
if (sorted.length === 0) return [1, 2, 3];
const p = (pct: number) =>
sorted[Math.min(Math.floor(pct * sorted.length), sorted.length - 1)];
return [p(0.25), p(0.5), p(0.75)];
}

200
ui/src/components/ContributionStats.tsx Normal file
View File

@@ -0,0 +1,200 @@
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { fetchDailyCounts, fetchHourlyAvgs, fetchSources } from '../api/client';
export function ContributionStats() {
const sourcesQ = useQuery({
queryKey: ['sources'],
queryFn: fetchSources,
staleTime: 60_000,
});
const earliest = useMemo(() => {
if (!sourcesQ.data) return null;
const dates = sourcesQ.data
.map((s) => s.earliest)
.filter((d): d is string => d != null)
.map((d) => new Date(d));
return dates.length > 0 ? new Date(Math.min(...dates.map((d) => d.getTime()))) : null;
}, [sourcesQ.data]);
const to = new Date();
const from = earliest ?? new Date(to.getFullYear() - 5, 0, 1);
const fromStr = fmt(from);
const toStr = fmt(to);
const dailyQ = useQuery({
queryKey: ['daily-counts-alltime', fromStr, toStr],
queryFn: () => fetchDailyCounts(fromStr, toStr),
enabled: !!earliest,
staleTime: 10 * 60_000,
});
// Bucket hour-of-day in the user's local timezone so the chart matches
// the clock they see. Browser may report e.g. "Europe/Helsinki"; fall
// back to UTC if the resolver returns something the server won't
// accept (it validates the string before binding).
const tz = useMemo(
() => Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC',
[],
);
const hourlyQ = useQuery({
queryKey: ['hourly-avgs-alltime', fromStr, toStr, tz],
queryFn: () => fetchHourlyAvgs(fromStr, toStr, tz),
enabled: !!earliest,
staleTime: 10 * 60_000,
});
const stats = useMemo(() => {
const counts = dailyQ.data ?? [];
if (counts.length === 0) return null;
// Build a set of dates with contributions
const countMap = new Map(counts.map((d) => [d.date, d.count]));
// Current streak (consecutive days ending today or yesterday with contributions)
let currentStreak = 0;
const cursor = new Date(to);
// Allow today to have 0 (day isn't over yet) — start from yesterday if today is 0
if ((countMap.get(fmt(cursor)) ?? 0) > 0) {
currentStreak = 1;
cursor.setDate(cursor.getDate() - 1);
} else {
cursor.setDate(cursor.getDate() - 1);
if ((countMap.get(fmt(cursor)) ?? 0) > 0) {
currentStreak = 1;
cursor.setDate(cursor.getDate() - 1);
}
}
if (currentStreak > 0) {
while ((countMap.get(fmt(cursor)) ?? 0) > 0) {
currentStreak++;
cursor.setDate(cursor.getDate() - 1);
}
}
// Longest streak
let longestStreak = 0;
let streak = 0;
const sorted = [...counts].sort((a, b) => a.date.localeCompare(b.date));
for (let i = 0; i < sorted.length; i++) {
if (sorted[i].count > 0) {
streak++;
if (streak > longestStreak) longestStreak = streak;
} else {
streak = 0;
}
}
// Busiest day
const busiest = sorted.reduce((best, d) => (d.count > best.count ? d : best), sorted[0]);
// Day-of-week averages
const dayTotals = [0, 0, 0, 0, 0, 0, 0];
const dayCounts = [0, 0, 0, 0, 0, 0, 0];
for (const d of sorted) {
const dow = new Date(d.date + 'T00:00:00').getDay();
dayTotals[dow] += d.count;
dayCounts[dow]++;
}
const dayNames = ['sun', 'mon', 'tue', 'wed', 'thu', 'fri', 'sat'];
const dayAvgs = dayNames.map((name, i) => ({
name,
avg: dayCounts[i] > 0 ? dayTotals[i] / dayCounts[i] : 0,
}));
const maxAvg = Math.max(...dayAvgs.map((d) => d.avg));
// Total active days
const activeDays = sorted.filter((d) => d.count > 0).length;
return { currentStreak, longestStreak, busiest, dayAvgs, maxAvg, activeDays };
}, [dailyQ.data]);
const hourly = useMemo(() => {
const data = hourlyQ.data ?? [];
if (data.length === 0) return null;
const byHour = new Array(24).fill(0);
for (const { hour, avg } of data) {
if (hour >= 0 && hour < 24) byHour[hour] = avg;
}
const max = Math.max(...byHour);
return { hours: byHour, max };
}, [hourlyQ.data]);
if (!stats) return null;
return (
<div>
<p style={{ fontSize: '0.8rem', opacity: 0.6 }}>contribution stats</p>
<div className="d-flex flex-column gap-2" style={{ fontSize: '0.8rem' }}>
<div className="d-flex justify-content-between">
<span style={{ opacity: 0.7 }}>current streak</span>
<span>{stats.currentStreak} {stats.currentStreak === 1 ? 'day' : 'days'}</span>
</div>
<div className="d-flex justify-content-between">
<span style={{ opacity: 0.7 }}>longest streak</span>
<span>{stats.longestStreak} {stats.longestStreak === 1 ? 'day' : 'days'}</span>
</div>
<div className="d-flex justify-content-between">
<span style={{ opacity: 0.7 }}>busiest day</span>
<span>{stats.busiest.count} on {stats.busiest.date}</span>
</div>
<div className="d-flex justify-content-between">
<span style={{ opacity: 0.7 }}>active days</span>
<span>{stats.activeDays.toLocaleString()}</span>
</div>
<div className="mt-1">
<span style={{ opacity: 0.7, fontSize: '0.75rem' }}>avg by weekday</span>
<div className="d-flex align-items-end gap-1 mt-1" style={{ height: 64 }}>
{stats.dayAvgs.map(({ name, avg }) => (
<div key={name} className="d-flex flex-column align-items-center" style={{ flex: 1, height: '100%', justifyContent: 'flex-end' }}>
<span style={{ fontSize: '0.65rem', opacity: 0.6, marginBottom: 2 }}>{avg.toFixed(1)}</span>
<div style={{ width: '100%', maxWidth: 20, borderRadius: 3, background: 'rgba(255,255,255,0.05)', flex: 1, display: 'flex', flexDirection: 'column', justifyContent: 'flex-end' }}>
<div
style={{
height: stats.maxAvg > 0 ? `${(avg / stats.maxAvg) * 100}%` : '0%',
borderRadius: 3,
backgroundColor: '#39d353',
opacity: 0.7,
}}
/>
</div>
<span style={{ fontSize: '0.65rem', opacity: 0.7, marginTop: 2 }}>{name}</span>
</div>
))}
</div>
</div>
{hourly && (
<div className="mt-3">
<span style={{ opacity: 0.7, fontSize: '0.75rem' }}>avg by hour ({tz})</span>
<div className="d-flex align-items-end gap-1 mt-1" style={{ height: 64 }}>
{hourly.hours.map((avg, h) => (
<div key={h} className="d-flex flex-column align-items-center" style={{ flex: 1, height: '100%', justifyContent: 'flex-end' }}>
<div style={{ width: '100%', borderRadius: 2, background: 'rgba(255,255,255,0.05)', flex: 1, display: 'flex', flexDirection: 'column', justifyContent: 'flex-end' }}>
<div
style={{
height: hourly.max > 0 ? `${(avg / hourly.max) * 100}%` : '0%',
borderRadius: 2,
backgroundColor: '#39d353',
opacity: 0.7,
}}
title={`${h.toString().padStart(2, '0')}:00 — ${avg.toFixed(2)}/day`}
/>
</div>
<span style={{ fontSize: '0.6rem', opacity: 0.7, marginTop: 2, minHeight: '0.7rem' }}>
{h % 4 === 0 ? h.toString().padStart(2, '0') : ''}
</span>
</div>
))}
</div>
</div>
)}
</div>
</div>
);
}
function fmt(d: Date): string {
return d.toISOString().slice(0, 10);
}

35
ui/src/components/LanguageBar.tsx Normal file
View File

@@ -0,0 +1,35 @@
export function LanguageBar({ languages, colorMap, compact }: {
languages: Record<string, number>;
colorMap: Record<string, string>;
compact?: boolean;
}) {
const total = Object.values(languages).reduce((a, b) => a + b, 0);
if (total === 0) return null;
const sorted = Object.entries(languages).sort(([, a], [, b]) => b - a);
return (
<div className={compact ? 'mb-1' : 'mt-2'}>
<div className="language-bar">
{sorted.map(([lang, bytes]) => (
<div
key={lang}
className="language-bar-segment"
style={{ width: `${(bytes / total) * 100}%`, backgroundColor: colorMap[lang] ?? '#8b8b8b' }}
title={`${lang} ${((bytes / total) * 100).toFixed(1)}%`}
/>
))}
</div>
{!compact && (
<div className="d-flex flex-wrap gap-2 mt-1" style={{ fontSize: '0.75rem' }}>
{sorted.slice(0, 8).map(([lang, bytes]) => (
<span key={lang}>
<span className="language-dot" style={{ backgroundColor: colorMap[lang] ?? '#8b8b8b' }} />
{lang} {((bytes / total) * 100).toFixed(1)}%
</span>
))}
</div>
)}
</div>
);
}

198
ui/src/components/LanguageStreamGraph.tsx Normal file
View File

@@ -0,0 +1,198 @@
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { fetchLanguageDailyCounts } from '../api/client';
const HEIGHT = 160;
const LABEL_HEIGHT = 16;
/** Language stream graph — stacked area showing language usage over time. */
export function LanguageStreamGraph() {
const to = new Date();
const from = new Date(to);
from.setFullYear(from.getFullYear() - 1);
const fromStr = fmt(from);
const toStr = fmt(to);
const langQ = useQuery({
queryKey: ['language-daily', fromStr, toStr],
queryFn: () => fetchLanguageDailyCounts(fromStr, toStr),
staleTime: 5 * 60_000,
});
const { languages, paths, legendItems } = useMemo(() => {
const raw = langQ.data ?? [];
if (raw.length === 0)
return { weeks: [], languages: [], paths: [], legendItems: [] };
// Aggregate daily counts into weekly buckets
const colorMap = new Map<string, string>();
const weeklyMap = new Map<string, Map<string, number>>();
for (const d of raw) {
if (d.color) colorMap.set(d.language, d.color);
// Bucket to ISO week (Monday-based, keyed by Monday date)
const dt = new Date(d.date + 'T00:00:00Z');
const day = dt.getUTCDay();
const monday = new Date(dt);
monday.setUTCDate(monday.getUTCDate() - ((day + 6) % 7));
const weekKey = monday.toISOString().slice(0, 10);
if (!weeklyMap.has(weekKey)) weeklyMap.set(weekKey, new Map());
const langs = weeklyMap.get(weekKey)!;
langs.set(d.language, (langs.get(d.language) ?? 0) + d.commits);
}
const weeks = [...weeklyMap.keys()].sort();
// Rank languages by total commits to pick top N + "other"
const totals = new Map<string, number>();
for (const langs of weeklyMap.values()) {
for (const [lang, count] of langs) {
totals.set(lang, (totals.get(lang) ?? 0) + count);
}
}
const ranked = [...totals.entries()].sort(([, a], [, b]) => b - a);
const topN = 8;
const topLangs = ranked.slice(0, topN).map(([l]) => l);
const hasOther = ranked.length > topN;
const languages = hasOther ? [...topLangs, 'Other'] : topLangs;
// Build stacked data per week
const stacked: number[][] = weeks.map((wk) => {
const langs = weeklyMap.get(wk)!;
const values = topLangs.map((l) => langs.get(l) ?? 0);
if (hasOther) {
let other = 0;
for (const [l, c] of langs) {
if (!topLangs.includes(l)) other += c;
}
values.push(other);
}
return values;
});
// Compute stream layout (centered baseline)
const maxTotal = Math.max(...stacked.map((row) => row.reduce((a, b) => a + b, 0)), 1);
const chartHeight = HEIGHT - LABEL_HEIGHT;
// For each week, compute y0 (centered) then stack upward
const layerCount = languages.length;
const y0s: number[][] = [];
const y1s: number[][] = [];
for (let w = 0; w < weeks.length; w++) {
const total = stacked[w].reduce((a, b) => a + b, 0);
const scaledTotal = (total / maxTotal) * chartHeight;
let baseline = (chartHeight - scaledTotal) / 2 + LABEL_HEIGHT;
const wy0: number[] = [];
const wy1: number[] = [];
for (let l = 0; l < layerCount; l++) {
const h = (stacked[w][l] / maxTotal) * chartHeight;
wy0.push(baseline);
baseline += h;
wy1.push(baseline);
}
y0s.push(wy0);
y1s.push(wy1);
}
// Build SVG paths for each language layer using smooth curves
const xFor = (w: number) =>
weeks.length > 1 ? (w / (weeks.length - 1)) * 100 : 50;
const paths = languages.map((_, l) => {
if (weeks.length === 0) return '';
const topPts = weeks.map((_, w) => [xFor(w), y0s[w][l]] as [number, number]);
const bottomPts = weeks
.map((_, w) => [xFor(w), y1s[w][l]] as [number, number])
.reverse();
return `M${topPts[0][0]},${topPts[0][1]} ${smoothLine(topPts)} L${bottomPts[0][0]},${bottomPts[0][1]} ${smoothLine(bottomPts)} Z`;
});
// Default colors for "Other" and fallback
const FALLBACK_COLORS = [
'#e34c26', '#563d7c', '#3178c6', '#dea584',
'#f1e05a', '#89e051', '#00ADD8', '#438eff',
];
const legendItems = languages.map((lang, i) => ({
language: lang,
color:
lang === 'Other'
? 'rgba(255,255,255,0.2)'
: colorMap.get(lang) ?? FALLBACK_COLORS[i % FALLBACK_COLORS.length],
total: ranked[i]?.[1] ?? 0,
}));
return { weeks, languages, paths, legendItems };
}, [langQ.data]);
if (langQ.isLoading) return <p style={{ fontSize: '0.8rem' }}>loading language graph...</p>;
if (langQ.isError || languages.length === 0) return null;
return (
<div className="contribution-graph mb-4">
<p style={{ fontSize: '0.8rem', opacity: 0.6 }}>languages by commit activity</p>
<svg viewBox={`0 0 100 ${HEIGHT}`} width="100%" preserveAspectRatio="none" className="d-block" style={{ height: `${HEIGHT}px` }}>
{paths.map((d, i) => (
<path
key={legendItems[i].language}
d={d}
fill={legendItems[i].color}
opacity={0.85}
>
<title>{legendItems[i].language}</title>
</path>
))}
</svg>
<div className="d-flex flex-wrap gap-2 mt-1" style={{ fontSize: '0.75rem' }}>
{legendItems.map(({ language, color }) => (
<span key={language} className="d-flex align-items-center gap-1">
<span
style={{
width: 10,
height: 10,
borderRadius: '50%',
backgroundColor: color,
display: 'inline-block',
}}
/>
{language}
</span>
))}
</div>
</div>
);
}
function fmt(d: Date): string {
return d.toISOString().slice(0, 10);
}
/** Convert a series of points into smooth cubic bezier curve commands.
* Uses Catmull-Rom to Bezier conversion with tension 0.5. */
function smoothLine(pts: [number, number][]): string {
if (pts.length < 2) return '';
if (pts.length === 2)
return `L${pts[1][0]},${pts[1][1]}`;
const commands: string[] = [];
for (let i = 1; i < pts.length; i++) {
const p0 = pts[Math.max(i - 2, 0)];
const p1 = pts[i - 1];
const p2 = pts[i];
const p3 = pts[Math.min(i + 1, pts.length - 1)];
const t = 0.5;
const cp1x = p1[0] + (p2[0] - p0[0]) * t / 3;
const cp1y = p1[1] + (p2[1] - p0[1]) * t / 3;
const cp2x = p2[0] - (p3[0] - p1[0]) * t / 3;
const cp2y = p2[1] - (p3[1] - p1[1]) * t / 3;
commands.push(`C${cp1x},${cp1y} ${cp2x},${cp2y} ${p2[0]},${p2[1]}`);
}
return commands.join(' ');
}

42
ui/src/components/Layout.tsx Normal file
View File

@@ -0,0 +1,42 @@
import { NavLink, Outlet } from 'react-router-dom';
import Container from 'react-bootstrap/Container';
const externalLinks = [
{ url: 'https://linkedin.com/in/thijssen/', label: 'linkedin' },
{ url: 'https://stackoverflow.com/users/68115/grenade', label: 'stackoverflow' },
{ url: 'https://github.com/grenade', label: 'github' },
{ url: 'https://git.lair.cafe/grenade', label: 'gitea' },
];
export function Layout() {
return (
<>
<Container className="py-4">
<header className="site-header d-flex flex-wrap justify-content-between align-items-center mb-4">
<h1 className="mb-0">hi, i'm rob</h1>
<nav className="d-flex flex-wrap gap-3 align-items-center">
<NavLink to="/" end>dash</NavLink>
<NavLink to="/activity">activity</NavLink>
<NavLink to="/cv">cv</NavLink>
<span className="nav-divider">|</span>
{externalLinks.map((el) => (
<a
key={el.url}
href={el.url}
target="_blank"
rel="noopener noreferrer"
>
{el.label}
</a>
))}
</nav>
</header>
<Outlet />
</Container>
<footer className="site-footer">
no cookies are set or read by this site, which is why no consent banner
is shown.
</footer>
</>
);
}

60
ui/src/components/TopLanguages.tsx Normal file
View File

@@ -0,0 +1,60 @@
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { fetchRepoLanguages } from '../api/client';
const MAX_LANGS = 14;
export function TopLanguages() {
const langsQ = useQuery({
queryKey: ['repo-languages'],
queryFn: fetchRepoLanguages,
staleTime: 10 * 60_000,
});
const ranked = useMemo(() => {
if (!langsQ.data) return [];
const totals = new Map<string, { bytes: number; color: string }>();
for (const e of langsQ.data) {
const cur = totals.get(e.language);
if (cur) {
cur.bytes += e.bytes;
} else {
totals.set(e.language, { bytes: e.bytes, color: e.color ?? '#8b8b8b' });
}
}
return [...totals.entries()]
.sort(([, a], [, b]) => b.bytes - a.bytes)
.slice(0, MAX_LANGS);
}, [langsQ.data]);
if (!langsQ.data || ranked.length === 0) return null;
const maxBytes = ranked[0][1].bytes;
return (
<div>
<p style={{ fontSize: '0.8rem', opacity: 0.6 }}>
top languages by code volume
</p>
<div className="d-flex flex-column gap-1">
{ranked.map(([lang, { bytes, color }]) => (
<div key={lang} className="d-flex align-items-center gap-2" style={{ fontSize: '0.75rem' }}>
<span style={{ width: 70, textAlign: 'right', opacity: 0.8, flexShrink: 0 }}>{lang}</span>
<div style={{ flex: 1, height: 10, borderRadius: 3, background: 'rgba(255,255,255,0.05)' }}>
<div
style={{
width: `${(bytes / maxBytes) * 100}%`,
height: '100%',
borderRadius: 3,
backgroundColor: color,
opacity: 0.85,
}}
/>
</div>
</div>
))}
</div>
</div>
);
}

22
ui/src/components/cv/CvHeader.tsx Normal file
View File

@@ -0,0 +1,22 @@
import { Link } from 'react-router-dom';
import Image from 'react-bootstrap/Image';
import { CV_PHOTO_URL } from '../../api/cv';
export function CvHeader() {
return (
<div className="cv-header d-flex flex-column flex-md-row align-items-md-center gap-3 mb-4">
<Image
src={CV_PHOTO_URL}
alt="rob"
roundedCircle
className="cv-photo"
/>
<div className="flex-grow-1">
<h1 className="mb-1">curriculum vitae</h1>
<Link to="/" className="hot-pink">
← timeline
</Link>
</div>
</div>
);
}

66
ui/src/components/cv/CvSection.tsx Normal file
View File

@@ -0,0 +1,66 @@
import ReactMarkdown from 'react-markdown';
import Card from 'react-bootstrap/Card';
import { type CvSectionConfig, type GistFile } from '../../api/cv';
import { entryAnchorId } from '../../lib/cvDates';
interface Props {
section: CvSectionConfig;
files: GistFile[];
}
// Pipe-delimited fields (e.g. "email | phone | github, linkedin" in the
// contact section) become one paragraph per field, so each lands on its own
// line with a paragraph gap. Within each pipe-segment, comma-separated values
// are stacked with a soft line break (markdown ` \n` -> `<br/>`) so multiple
// emails / phones / urls each get their own line at a tighter spacing.
function splitPipes(content: string): string {
return content
.split('\n')
.map((line) => {
if (!line.includes(' | ')) return line;
return line
.split(' | ')
.map((segment) =>
segment.includes(', ') ? segment.split(', ').join(' \n') : segment,
)
.join('\n\n');
})
.join('\n');
}
// Renders a single section. Each .md file becomes its own block. When
// `show_section_name` is true (e.g. experience, education) the entries are
// wrapped in cards and given anchor ids so the timeline sidebar can deep-link
// to them; otherwise (e.g. summary, contact) they render as flat markdown.
export function CvSection({ section, files }: Props) {
return (
<section id={section.name} className="cv-section mb-4">
{section.show_section_name && <h2 className="cv-section-name">{section.name}</h2>}
{files.map((file) => {
const content = section.show_section_name
? file.content
: splitPipes(file.content);
if (section.show_section_name) {
return (
<div
key={file.filename}
id={entryAnchorId(section.name, file.content)}
className="cv-entry mb-3"
>
<Card className="cv-card">
<Card.Body>
<ReactMarkdown>{content}</ReactMarkdown>
</Card.Body>
</Card>
</div>
);
}
return (
<div key={file.filename} className="cv-entry">
<ReactMarkdown>{content}</ReactMarkdown>
</div>
);
})}
</section>
);
}

98
ui/src/components/cv/CvTimeline.tsx Normal file
View File

@@ -0,0 +1,98 @@
import type { ReactNode } from 'react';
import {
VerticalTimeline,
VerticalTimelineElement,
} from 'react-vertical-timeline-component';
import { type CvData, filesForSection } from '../../api/cv';
import { entryAnchorId, parseEntryHeader } from '../../lib/cvDates';
interface Props {
data: CvData;
}
// Tiny inline parser: turns "[text](url)" segments into <a> elements while
// leaving surrounding text alone. Used so the timeline title renders the
// company name as plain text and the linked website as an external link
// (matching how a markdown parser would render the same source).
function renderInlineLinks(text: string): ReactNode[] {
const parts: ReactNode[] = [];
const re = /\[([^\]]+)\]\(([^)]+)\)/g;
let last = 0;
let m: RegExpExecArray | null;
while ((m = re.exec(text)) !== null) {
if (m.index > last) parts.push(text.slice(last, m.index));
parts.push(
<a key={m.index} href={m[2]} target="_blank" rel="noopener noreferrer">
{m[1]}
</a>,
);
last = m.index + m[0].length;
}
if (last < text.length) parts.push(text.slice(last));
return parts;
}
// Sidebar timeline rendered from every body section that has `show_section_name`
// (i.e. timeline-eligible sections — experience and education in the current
// gist). Each element offers a small "→" link to the matching anchor in the
// body; the title and subtitle preserve any inline markdown links so they
// behave as proper external anchors.
export function CvTimeline({ data }: Props) {
const elements = data.config.sections
.filter((s) => s.placement === 'body' && s.show_section_name)
.flatMap((section) =>
filesForSection(data, section).map((file) => ({ section, file })),
);
if (elements.length === 0) return null;
return (
<div className="cv-timeline">
<h2 className="cv-section-name">timeline</h2>
<VerticalTimeline layout="1-column-left" lineColor="#ecf0f1">
{elements.map(({ section, file }) => {
const parsed = parseEntryHeader(file.content);
const anchor = `#${entryAnchorId(section.name, file.content)}`;
return (
<VerticalTimelineElement
key={file.filename}
date={parsed.interval.replace(/\s*\([^)]*\)/g, '')}
iconStyle={
parsed.iconUrl
? { background: '#ffffff', boxShadow: 'none' }
: { background: '#ff4081', color: '#ffffff' }
}
icon={
parsed.iconUrl ? (
<img
src={parsed.iconUrl}
alt={parsed.title}
style={{
width: '100%',
height: '100%',
objectFit: 'contain',
borderRadius: '50%',
padding: 4,
}}
/>
) : undefined
}
>
<h3 className="vertical-timeline-element-title">
{renderInlineLinks(parsed.titleMd)}
</h3>
{parsed.locationRoleMd && (
<h4 className="vertical-timeline-element-subtitle">
{renderInlineLinks(parsed.locationRoleMd)}
</h4>
)}
<a href={anchor} className="cv-timeline-anchor">
→ details
</a>
</VerticalTimelineElement>
);
})}
</VerticalTimeline>
</div>
);
}

91
ui/src/lib/cvDates.ts Normal file
View File

@@ -0,0 +1,91 @@
// Normalizes the date interval line of a CV entry. The legacy implementation
// at cv/src/App.js:139 chained .replace() calls per month name; this collapses
// that into a single regex pass.
const MONTH_ABBREV: Record<string, string> = {
january: 'jan',
february: 'feb',
march: 'mar',
april: 'apr',
// may stays "may"
june: 'jun',
july: 'jul',
august: 'aug',
september: 'sep',
october: 'oct',
november: 'nov',
december: 'dec',
};
const MONTH_RE = new RegExp(
`\\b(${Object.keys(MONTH_ABBREV).join('|')})\\b`,
'gi',
);
// Strip leading/trailing markdown # / whitespace, abbreviate months. Casing
// is left to the body-level `text-transform: lowercase` so a future toggle can
// flip it from a single place.
export function normalizeInterval(line: string): string {
return line
.replace(/^[\s#]+|[\s#]+$/g, '')
.replace(MONTH_RE, (m) => MONTH_ABBREV[m.toLowerCase()] ?? m);
}
// Strip markdown link syntax (e.g. "[text](url)") down to just the text.
export function stripMdLinks(s: string): string {
return s.replace(/\[([^\]]+)\]\([^)]+\)/g, '$1');
}
// Parse the title / location-role / interval triple out of a CV entry's
// markdown content. If the first line embeds a PNG URL (icon-style entry),
// indices shift down by one.
export interface ParsedHeader {
// Plain text, markdown links stripped — for alt= attributes and similar.
title: string;
locationRole: string;
// Markdown source with leading #s/whitespace stripped — for inline rendering
// so [text](url) links render as proper anchors.
titleMd: string;
locationRoleMd: string;
iconUrl: string | null;
interval: string;
}
export function parseEntryHeader(content: string): ParsedHeader {
const lines = content.split('\n');
const firstLine = lines[0] ?? '';
const hasIcon = firstLine.includes('.png');
const iconUrl = hasIcon
? (firstLine.match(/https:[^ )\]]+\.png/)?.[0] ?? null)
: null;
const titleLine = hasIcon ? (lines[1] ?? '') : firstLine;
const locRoleLine = hasIcon ? (lines[2] ?? '') : (lines[1] ?? '');
const intervalLine = hasIcon ? (lines[3] ?? '') : (lines[2] ?? '');
const titleMd = titleLine.replace(/^[\s#]+|[\s#]+$/g, '');
const locationRoleMd = locRoleLine.replace(/^[\s#]+|[\s#]+$/g, '');
return {
title: stripMdLinks(titleMd),
locationRole: stripMdLinks(locationRoleMd),
titleMd,
locationRoleMd,
iconUrl,
interval: normalizeInterval(intervalLine),
};
}
// Anchor id for an entry: combines the section name and a slug of the title
// line. Mirrors the legacy id format at cv/src/App.js:71.
export function entryAnchorId(sectionName: string, content: string): string {
const lines = content.split('\n');
const firstLine = lines[0] ?? '';
const titleLine = firstLine.includes('.png') ? (lines[1] ?? '') : firstLine;
const slug = stripMdLinks(titleLine.replace(/^[\s#]+|[\s#]+$/g, ''))
.toLowerCase()
.replace(/[^a-z0-9\s-]/g, '')
.replace(/\s+/g, '-')
.replace(/-+/g, '-');
return `${sectionName}-${slug}`;
}

5
ui/src/main.tsx
View File

@@ -1,5 +1,6 @@
import { StrictMode } from 'react'; import { StrictMode } from 'react';
import { createRoot } from 'react-dom/client'; import { createRoot } from 'react-dom/client';
import { BrowserRouter } from 'react-router-dom';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import App from './App'; import App from './App';
@@ -15,7 +16,9 @@ const queryClient = new QueryClient({
createRoot(document.getElementById('root')!).render( createRoot(document.getElementById('root')!).render(
<StrictMode> <StrictMode>
<QueryClientProvider client={queryClient}> <QueryClientProvider client={queryClient}>
<App /> <BrowserRouter>
<App />
</BrowserRouter>
</QueryClientProvider> </QueryClientProvider>
</StrictMode>, </StrictMode>,
); );

98
ui/src/pages/CvPage.css Normal file
View File

@@ -0,0 +1,98 @@
.cv-photo {
width: 96px;
height: 96px;
object-fit: cover;
}
.cv-section-name {
margin-bottom: 0.75rem;
border-bottom: 1px solid rgba(236, 241, 241, 0.2);
padding-bottom: 0.25rem;
}
.cv-card {
background-color: #34495e;
color: #ecf0f1;
border: 1px solid rgba(236, 241, 241, 0.1);
}
.cv-card a {
color: #ff80ab;
}
.cv-card a:hover {
color: #ff4081;
}
.cv-card img {
max-width: 96px;
max-height: 48px;
background-color: #ffffff;
padding: 4px;
border-radius: 4px;
margin-bottom: 0.75rem;
}
.cv-card h3 {
font-size: 1.25rem;
}
.cv-card h4 {
font-size: 1.05rem;
opacity: 0.9;
}
.cv-card h5 {
font-size: 0.95rem;
opacity: 0.75;
font-style: italic;
}
.cv-timeline .vertical-timeline-element-content {
background-color: #34495e;
color: #ecf0f1;
box-shadow: 0 3px 0 #ff4081;
}
.cv-timeline .vertical-timeline-element-content a {
color: #ff80ab;
}
.cv-timeline .vertical-timeline-element-content a:hover {
color: #ff4081;
}
.cv-timeline h3.vertical-timeline-element-title {
font-size: 0.95rem;
margin: 0;
}
.cv-timeline h4.vertical-timeline-element-subtitle {
font-size: 0.8rem;
opacity: 0.85;
margin: 0.15rem 0 0.4rem;
font-weight: normal;
}
.cv-timeline .cv-timeline-anchor {
font-size: 0.75rem;
opacity: 0.85;
float: right;
margin-left: 0.75rem;
}
.cv-timeline .vertical-timeline-element-content::before {
border-right-color: #34495e;
border-left-color: #34495e;
}
.cv-timeline .vertical-timeline-element-date {
color: #ecf0f1 !important;
opacity: 0.8;
}
@media (max-width: 991px) {
.cv-timeline {
margin-top: 2rem;
}
}

107
ui/src/pages/CvPage.tsx Normal file
View File

@@ -0,0 +1,107 @@
import { useEffect } from 'react';
import { useLocation } from 'react-router-dom';
import { useQuery } from '@tanstack/react-query';
import Alert from 'react-bootstrap/Alert';
import Col from 'react-bootstrap/Col';
import Row from 'react-bootstrap/Row';
import Spinner from 'react-bootstrap/Spinner';
import { fetchCv, filesForSection } from '../api/cv';
import { CvHeader } from '../components/cv/CvHeader';
import { CvSection } from '../components/cv/CvSection';
import { CvTimeline } from '../components/cv/CvTimeline';
import './CvPage.css';
export function CvPage() {
const { hash } = useLocation();
const cvQ = useQuery({
queryKey: ['cv-gist'],
queryFn: fetchCv,
staleTime: 5 * 60_000,
});
// Scroll to the anchored entry once the gist resolves and the section
// body has rendered its ids. Re-runs if the user changes the hash while
// already on the page.
useEffect(() => {
if (!cvQ.data || !hash) return;
const target = document.getElementById(hash.slice(1));
if (target) {
target.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
}, [cvQ.data, hash]);
if (cvQ.isLoading) {
return (
<>
<CvHeader />
<div className="d-flex align-items-center gap-2">
<Spinner animation="border" role="status" size="sm" />
<span>loading cv…</span>
</div>
</>
);
}
if (cvQ.isError) {
const msg = (cvQ.error as Error).message;
const rateHint = /403|rate limit/i.test(msg)
? ' (github limits unauthenticated requests to 60/hour per ip — try again shortly)'
: '';
return (
<>
<CvHeader />
<Alert variant="danger">
<Alert.Heading>cv unavailable</Alert.Heading>
<p className="mb-2">
{msg}
{rateHint}
</p>
<button className="btn btn-outline-light" onClick={() => cvQ.refetch()}>
retry
</button>
</Alert>
</>
);
}
const data = cvQ.data!;
const bodySections = data.config.sections.filter((s) => s.placement === 'body');
const navSections = data.config.sections.filter((s) => s.placement === 'nav');
if (bodySections.length === 0 && navSections.length === 0) {
return (
<>
<CvHeader />
<Alert variant="warning">cv unavailable: no sections in config</Alert>
</>
);
}
return (
<>
<CvHeader />
<Row>
<Col lg={9} className="cv-body">
{bodySections.map((section) => (
<CvSection
key={section.name}
section={section}
files={filesForSection(data, section)}
/>
))}
</Col>
<Col lg={3} className="cv-sidebar">
{navSections.map((section) => (
<CvSection
key={section.name}
section={section}
files={filesForSection(data, section)}
/>
))}
<CvTimeline data={data} />
</Col>
</Row>
</>
);
}

141
ui/src/pages/DashPage.tsx Normal file
View File

@@ -0,0 +1,141 @@
import { useMemo } from 'react';
import { useQuery } from '@tanstack/react-query';
import { Link } from 'react-router-dom';
import Col from 'react-bootstrap/Col';
import Row from 'react-bootstrap/Row';
import { fetchProjects, fetchRepoLanguages, type ProjectSummary } from '../api/client';
import { LanguageBar } from '../components/LanguageBar';
import { ContributionGraph, AllTimeGraph } from '../components/ContributionGraph';
import { ContributionStats } from '../components/ContributionStats';
import { LanguageStreamGraph } from '../components/LanguageStreamGraph';
import { TopLanguages } from '../components/TopLanguages';
export function DashPage() {
const projectsQ = useQuery({
queryKey: ['projects'],
queryFn: fetchProjects,
refetchInterval: 60_000,
});
const langsQ = useQuery({
queryKey: ['repo-languages'],
queryFn: fetchRepoLanguages,
staleTime: 10 * 60_000,
});
const langsByRepo = useMemo(() => {
const map = new Map<string, Record<string, number>>();
for (const entry of langsQ.data ?? []) {
const key = `${entry.source}:${entry.repo}`;
if (!map.has(key)) map.set(key, {});
map.get(key)![entry.language] = entry.bytes;
}
return map;
}, [langsQ.data]);
const langColors = useMemo(() => {
const map: Record<string, string> = {};
for (const e of langsQ.data ?? []) {
if (e.color && !map[e.language]) map[e.language] = e.color;
}
return map;
}, [langsQ.data]);
const projects = projectsQ.data ?? [];
const ranked = rankProjects(projects);
return (
<>
<Row className="mb-3">
<Col>
<p>
i rarely say anything that warrants capital letters. a peek into the
projects i'm working on is below.
</p>
</Col>
</Row>
<ContributionGraph />
<LanguageStreamGraph />
<Row xs={1} md={2} lg={3} className="g-3 mb-3">
<Col>
<AllTimeGraph />
</Col>
<Col>
<TopLanguages />
</Col>
<Col>
<ContributionStats />
</Col>
</Row>
{projectsQ.isLoading && <p>loading...</p>}
{projectsQ.isError && (
<p>error: {(projectsQ.error as Error).message}</p>
)}
<Row xs={1} md={2} lg={3} className="g-3">
{ranked.map((p) => (
<Col key={`${p.source}:${p.repo}`}>
<ProjectCard project={p} langs={langsByRepo.get(`${p.source}:${p.repo}`) ?? null} colorMap={langColors} />
</Col>
))}
</Row>
</>
);
}
function ProjectCard({ project: p, langs, colorMap }: { project: ProjectSummary; langs: Record<string, number> | null; colorMap: Record<string, string> }) {
return (
<Link to={`/project/${p.source}/${p.repo}`} className="text-decoration-none">
<div className="project-card p-3">
<h5 className="mb-1"><img src={forgeIcon(p.source)} alt={p.source} className="forge-icon" />{p.repo}</h5>
{langs && <LanguageBar languages={langs} colorMap={colorMap} compact />}
<div className="d-flex gap-3" style={{ fontSize: '0.8rem' }}>
{p.commit_count > 0 && <span>{p.commit_count} commits</span>}
{p.issue_count > 0 && <span>{p.issue_count} issues</span>}
{p.pr_count > 0 && <span>{p.pr_count} prs</span>}
</div>
<div style={{ fontSize: '0.75rem', opacity: 0.6, marginTop: '0.25rem' }}>
{formatRange(p.first_activity, p.last_activity)}
</div>
</div>
</Link>
);
}
function forgeIcon(source: string): string {
switch (source) {
case 'github': return '/github.svg';
case 'gitea': return '/gitea.svg';
case 'hg': return '/mozilla.svg';
default: return '/github.svg';
}
}
function formatRange(first: string | null, last: string | null): string {
const fmt = (iso: string) =>
new Date(iso).toLocaleDateString('en-GB', { month: 'short', year: 'numeric' }).toLowerCase();
if (first && last) return `${fmt(first)} — ${fmt(last)}`;
if (last) return fmt(last);
return '';
}
function rankProjects(projects: ProjectSummary[]): ProjectSummary[] {
if (projects.length === 0) return [];
const now = Date.now();
const maxVolume = Math.max(...projects.map((p) => p.commit_count + p.issue_count + p.pr_count));
const oldest = Math.min(
...projects.map((p) => (p.last_activity ? new Date(p.last_activity).getTime() : 0)),
);
const range = now - oldest || 1;
return [...projects].sort((a, b) => score(b) - score(a));
function score(p: ProjectSummary): number {
if (p.commit_count >= 10000) return -1;
const volume = (p.commit_count + p.issue_count + p.pr_count) / (maxVolume || 1);
const recency = p.last_activity
? (new Date(p.last_activity).getTime() - oldest) / range
: 0;
return 0.6 * recency + 0.4 * volume;
}
}

182
ui/src/pages/ProjectPage.tsx Normal file
View File

@@ -0,0 +1,182 @@
import { useMemo } from 'react';
import { useParams } from 'react-router-dom';
import { useQuery } from '@tanstack/react-query';
import Col from 'react-bootstrap/Col';
import Row from 'react-bootstrap/Row';
import ReactMarkdown from 'react-markdown';
import rehypeRaw from 'rehype-raw';
import rehypeSanitize, { defaultSchema } from 'rehype-sanitize';
import remarkGfm from 'remark-gfm';
import { VerticalTimeline } from 'react-vertical-timeline-component';
import { fetchEvents, fetchReadme, fetchRepoLanguages, fetchProjects, type Source } from '../api/client';
import { LanguageBar } from '../components/LanguageBar';
import { TimelineEntry } from '../components/TimelineEntry';
export function ProjectPage() {
const { source, '*': repoPath } = useParams();
const repo = repoPath ?? '';
const projectsQ = useQuery({
queryKey: ['projects'],
queryFn: fetchProjects,
staleTime: 60_000,
});
const project = projectsQ.data?.find(
(p) => p.source === source && p.repo === repo,
);
const host = project?.host ?? '';
const eventsQ = useQuery({
queryKey: ['project-events', source, repo],
queryFn: () =>
fetchEvents({
sources: source ? [source as Source] : undefined,
repo,
limit: 500,
}),
refetchInterval: 60_000,
});
const readmeQ = useQuery({
queryKey: ['readme', source, host, repo],
queryFn: () => fetchReadme(source as Source, host, repo),
enabled: !!host && (source === 'github' || source === 'gitea'),
staleTime: 5 * 60_000,
});
const repoLangsQ = useQuery({
queryKey: ['repo-languages'],
queryFn: fetchRepoLanguages,
staleTime: 10 * 60_000,
});
const langs = useMemo(() => {
if (!repoLangsQ.data || !source) return null;
const entries = repoLangsQ.data.filter(
(e) => e.source === source && e.repo === repo,
);
if (entries.length === 0) return null;
const result: Record<string, number> = {};
for (const e of entries) result[e.language] = e.bytes;
return result;
}, [repoLangsQ.data, source, repo]);
const langColors = useMemo(() => {
const map: Record<string, string> = {};
for (const e of repoLangsQ.data ?? []) {
if (e.color && !map[e.language]) map[e.language] = e.color;
}
return map;
}, [repoLangsQ.data]);
const events = eventsQ.data ?? [];
return (
<>
<Row className="mb-3">
<Col>
<h2><a href={repoUrl(source ?? '', host, repo)} target="_blank" rel="noopener noreferrer"><img src={forgeIcon(source ?? '')} alt={source} className="forge-icon" style={{ width: 24, height: 24 }} /></a>{repo}</h2>
{langs && <LanguageBar languages={langs} colorMap={langColors} />}
</Col>
</Row>
{readmeQ.data && (
<Row className="mb-4">
<Col>
<div className="project-readme">
<ReactMarkdown
remarkPlugins={[remarkGfm]}
rehypePlugins={[rehypeRaw, [rehypeSanitize, readmeSanitizeSchema]]}
>
{readmeQ.data}
</ReactMarkdown>
</div>
</Col>
</Row>
)}
<Row>
<Col>
<p style={{ fontSize: '85%' }}>
{eventsQ.isLoading
? 'loading...'
: eventsQ.isError
? `error: ${(eventsQ.error as Error).message}`
: `${events.length} ${events.length === 1 ? 'activity' : 'activities'}`}
</p>
<VerticalTimeline>
{events.map((item) => (
<TimelineEntry key={item.id} item={item} />
))}
</VerticalTimeline>
</Col>
</Row>
</>
);
}
function repoUrl(source: string, host: string, repo: string): string {
switch (source) {
case 'github': return `https://github.com/${repo}`;
case 'gitea': return `https://${host}/${repo}`;
case 'hg': return `https://${host}/${repo}`;
default: return '#';
}
}
function forgeIcon(source: string): string {
switch (source) {
case 'github': return '/github.svg';
case 'gitea': return '/gitea.svg';
case 'hg': return '/mozilla.svg';
default: return '/github.svg';
}
}
// rehype-sanitize defaults are conservative — README authors lean on raw
// HTML for layout (centered headers, collapsible sections, image
// dimensions). Extend the schema to permit those tags/attributes while
// still blocking script-y or interactive content (iframe, object, etc.).
const readmeSanitizeSchema = {
...defaultSchema,
tagNames: [
...(defaultSchema.tagNames ?? []),
'details',
'summary',
'picture',
'source',
'kbd',
'sub',
'sup',
'mark',
'abbr',
'cite',
'figure',
'figcaption',
'center',
],
attributes: {
...defaultSchema.attributes,
'*': [
...((defaultSchema.attributes && defaultSchema.attributes['*']) || []),
'align',
'style',
],
a: [
...((defaultSchema.attributes && defaultSchema.attributes.a) || []),
'target',
'rel',
],
img: [
...((defaultSchema.attributes && defaultSchema.attributes.img) || []),
'width',
'height',
'align',
'srcset',
],
source: ['srcset', 'media', 'type'],
details: ['open'],
},
};

132
ui/src/pages/TimelineHome.tsx Normal file
View File

@@ -0,0 +1,132 @@
import { useMemo, useState } from 'react';
import { useParams } from 'react-router-dom';
import { useQuery } from '@tanstack/react-query';
import Col from 'react-bootstrap/Col';
import Row from 'react-bootstrap/Row';
import { VerticalTimeline } from 'react-vertical-timeline-component';
import { fetchDailyCounts, fetchEvents, fetchSources, type Source } from '../api/client';
import { Filters } from '../components/Filters';
import { TimelineEntry } from '../components/TimelineEntry';
const RANGE_MIN = new Date('2010-01-01T00:00:00Z').getTime();
const RANGE_MAX = Date.now();
function parseDate(s: string): number {
// Accept YYYY-MM-DD or full ISO datetime
const t = new Date(s.includes('T') ? s : s + 'T00:00:00Z').getTime();
return isNaN(t) ? NaN : t;
}
function endOfDay(s: string): number {
const t = new Date(s.includes('T') ? s : s + 'T23:59:59Z').getTime();
return isNaN(t) ? NaN : t;
}
function parseTimespan(timespan?: string): [number, number] | null {
if (!timespan) return null;
if (timespan.includes('..')) {
const [a, b] = timespan.split('..');
const from = parseDate(a);
const to = endOfDay(b);
if (!isNaN(from) && !isNaN(to)) return [from, to];
} else {
const from = parseDate(timespan);
const to = endOfDay(timespan);
if (!isNaN(from)) return [from, to];
}
return null;
}
export function TimelineHome() {
const { timespan } = useParams();
const [enabledSources, setEnabledSources] = useState<Record<Source, boolean>>({
github: true,
gitea: true,
hg: true,
bugzilla: true,
});
const [rangeValue, setRangeValue] = useState<[number, number]>(() => {
const parsed = parseTimespan(timespan);
if (parsed) return parsed;
const now = Date.now();
const thirtyDaysAgo = now - 30 * 24 * 60 * 60 * 1000;
return [thirtyDaysAgo, now];
});
const [limit, setLimit] = useState<number>(100);
const sourcesQ = useQuery({
queryKey: ['sources'],
queryFn: fetchSources,
refetchInterval: 60_000,
});
const activeSources = useMemo(
() =>
(Object.keys(enabledSources) as Source[]).filter((s) => enabledSources[s]),
[enabledSources],
);
const eventsQ = useQuery({
queryKey: ['events', rangeValue, activeSources, limit],
queryFn: () =>
fetchEvents({
from: new Date(rangeValue[0]),
to: new Date(rangeValue[1]),
sources: activeSources,
limit,
}),
refetchInterval: 60_000,
});
const events = eventsQ.data ?? [];
const fromStr = new Date(rangeValue[0]).toISOString().slice(0, 10);
const toStr = new Date(rangeValue[1]).toISOString().slice(0, 10);
const dailyQ = useQuery({
queryKey: ['daily-counts', fromStr, toStr],
queryFn: () => fetchDailyCounts(fromStr, toStr),
staleTime: 5 * 60_000,
});
const totalCount = useMemo(
() => (dailyQ.data ?? []).reduce((sum, d) => sum + d.count, 0),
[dailyQ.data],
);
const privateCount = totalCount - events.length;
return (
<>
<Filters
enabledSources={enabledSources}
onSourceToggle={(s, on) =>
setEnabledSources((prev) => ({ ...prev, [s]: on }))
}
rangeMin={RANGE_MIN}
rangeMax={RANGE_MAX}
rangeValue={rangeValue}
onRangeChange={setRangeValue}
limit={limit}
onLimitChange={setLimit}
summaries={sourcesQ.data}
/>
<Row>
<Col>
<p className="text-center" style={{ fontSize: '85%' }}>
{eventsQ.isLoading
? 'loading…'
: eventsQ.isError
? `error: ${(eventsQ.error as Error).message}`
: `showing ${events.length} public ${events.length === 1 ? 'activity' : 'activities'}${privateCount > 0 ? `, ${privateCount} private` : ''}`}
</p>
<VerticalTimeline>
{events.map((item) => (
<TimelineEntry key={item.id} item={item} />
))}
</VerticalTimeline>
</Col>
</Row>
</>
);
}