repo visibility changes are never noticed: private repos keep serving their history #6

New Issue

grenade · 2026-08-15T16:41:28Z

grenade commented

2026-08-15 16:41:28 +00:00

events.public is decided once, when a row is ingested, from whatever the forge reported at that moment. Every poller is incremental, so nothing ever revisits a repo:

github events feed — capped at 90 days / 300 events by GitHub
github-search — only re-walks its top-1000 desc window
github-repo — per-branch since cursor, history is never walked again
gitea — max_pages = 1 after the first run, so only the newest page of the feed

The upsert does refresh the flag (public = EXCLUDED.public), but only for events that get re-fetched. So when a repo flips public → private upstream, new activity lands as public = false while everything already ingested stays public = true indefinitely — full commit messages, repo name, issue and PR titles, all still served on /v1/events, /v1/projects and /v1/activity/summary. The reverse flip is equally frozen: history that was ingested while a repo was private stays hidden forever after it goes public.

The "counts yes, details no" split is implemented correctly at the API layer (daily, hourly, sources, languages/daily pass include_private: true; events, projects, activity/summary pass false). It is just keying off a stale flag.

Separate leak, same area

/v1/languages/repos has no visibility gate at all — repo_languages() selects the whole table, and fetch_languages is called with the full repo list including private ones. Private repo names are in that JSON today, and it is prefetched into the prerendered window.__RQ_STATE__, so they are in the static HTML too. The UI only aggregates by language, so nothing renders them, but they are on the wire.

Fix

A reconciliation pass in the worker: after repo discovery (which already holds current visibility for every reachable repo), UPDATE events SET public = ... WHERE repo = .... That needs a repo key to update against — the payload → repo CASE expression already exists, copy-pasted into four separate read queries, and is worth promoting to a stored generated column with an index. The languages response should derive its gate from the same source of truth rather than growing a second visibility column to keep in sync.

`events.public` is decided once, when a row is ingested, from whatever the forge reported at that moment. Every poller is incremental, so nothing ever revisits a repo: - github events feed — capped at 90 days / 300 events by GitHub - github-search — only re-walks its top-1000 desc window - github-repo — per-branch `since` cursor, history is never walked again - gitea — `max_pages = 1` after the first run, so only the newest page of the feed The upsert does refresh the flag (`public = EXCLUDED.public`), but only for events that get re-fetched. So when a repo flips **public → private** upstream, new activity lands as `public = false` while **everything already ingested stays `public = true` indefinitely** — full commit messages, repo name, issue and PR titles, all still served on `/v1/events`, `/v1/projects` and `/v1/activity/summary`. The reverse flip is equally frozen: history that was ingested while a repo was private stays hidden forever after it goes public. The "counts yes, details no" split is implemented correctly at the API layer (`daily`, `hourly`, `sources`, `languages/daily` pass `include_private: true`; `events`, `projects`, `activity/summary` pass `false`). It is just keying off a stale flag. ### Separate leak, same area `/v1/languages/repos` has **no visibility gate at all** — `repo_languages()` selects the whole table, and `fetch_languages` is called with the full repo list including private ones. Private repo names are in that JSON today, and it is prefetched into the prerendered `window.__RQ_STATE__`, so they are in the static HTML too. The UI only aggregates by language, so nothing renders them, but they are on the wire. ### Fix A reconciliation pass in the worker: after repo discovery (which already holds current visibility for every reachable repo), `UPDATE events SET public = ... WHERE repo = ...`. That needs a repo key to update against — the payload → repo `CASE` expression already exists, copy-pasted into four separate read queries, and is worth promoting to a stored generated column with an index. The languages response should derive its gate from the same source of truth rather than growing a second visibility column to keep in sync.

grenade referenced this issue from a commit

2026-08-15 16:41:44 +00:00

fix: reconcile repo visibility instead of trusting the ingest-time flag

grenade closed this issue

2026-08-15 16:41:44 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: grenade/moments#6