feat: scaffold newsfeed — user-controlled news feed
Some checks failed
deploy / build (push) Failing after 7s
deploy / deploy-api (push) Has been skipped
deploy / deploy-web (push) Has been skipped

A self-hosted feed where the user owns the ranking: per-source and signed
per-interest weights, decayed by recency, via a transparent deterministic
scorer. Content is sourced algorithmically (worker RSS/Atom polling) and
agentically (per-user API tokens POSTing candidates to the ingest endpoint).
Single-user today, multi-user by construction (every row keyed on user_id).

Rust cargo workspace + Vite/React/SWC/TS SPA:
- newsfeed-entities: DTOs (ts-rs bindings -> web/src/api/bindings)
- newsfeed-core: ranking, auth primitives, ingest, data-access ports
- newsfeed-data: SQLite adapters (sqlx, runtime queries)
- newsfeed-api: axum REST/JSON daemon
- newsfeed-worker: RSS polling + rescoring loop
- web: responsive, mobile-first SPA (React Query, generated types)

Deploy (Gitea Actions, build static musl + SPA, rsync as gitea_ci):
api+worker -> slartibartfast, SPA -> oolon (nginx serves + proxies /v1).

Deliberate deviations from house conventions (documented in CLAUDE.md/readme):
- SQLite instead of Postgres; api+worker co-locate sharing one DB file.
- Runtime sqlx queries instead of query! macros (SQLite dynamic typing;
  keeps CI database-free).

Verified end-to-end: auth, token ingest, interest-weighted ranking, signals,
pagination (curl + browser); cargo fmt/clippy -D/test and pnpm build/lint pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016fKZzDpvjiJ9eYbPGgJvUP
This commit is contained in:
2026-07-08 11:36:57 +03:00
commit 5ce52fff4d
104 changed files with 10963 additions and 0 deletions

View File

@@ -0,0 +1,34 @@
[package]
name = "newsfeed-api"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
description = "REST/JSON API daemon for newsfeed (axum)."
[[bin]]
name = "newsfeed-api"
path = "src/main.rs"
[dependencies]
newsfeed-entities.workspace = true
newsfeed-core.workspace = true
newsfeed-data.workspace = true
tokio.workspace = true
axum.workspace = true
tower.workspace = true
tower-http.workspace = true
serde.workspace = true
serde_json.workspace = true
chrono.workspace = true
uuid.workspace = true
thiserror.workspace = true
anyhow.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
figment.workspace = true
clap.workspace = true

View File

@@ -0,0 +1,98 @@
//! Authentication extractors and cookie helpers.
//!
//! [`CurrentUser`] authenticates a browser via the `nf_session` cookie; [`ApiPrincipal`]
//! authenticates an agentic/algorithmic producer via a `Bearer` API token. Both resolve
//! through `newsfeed-core::service`, so the crypto and lookup live in one place.
use axum::extract::FromRef;
use axum::extract::FromRequestParts;
use axum::http::header::{AUTHORIZATION, COOKIE};
use axum::http::request::Parts;
use newsfeed_core::service;
use newsfeed_entities::auth::ApiTokenInfo;
use newsfeed_entities::user::User;
use crate::error::ApiError;
use crate::state::AppState;
/// Name of the session cookie.
pub const SESSION_COOKIE: &str = "nf_session";
/// Build a `Set-Cookie` header value for a freshly minted session.
pub fn session_cookie(value: &str, ttl_days: i64, secure: bool) -> String {
let max_age = ttl_days.max(0) * 24 * 60 * 60;
let mut c =
format!("{SESSION_COOKIE}={value}; HttpOnly; SameSite=Lax; Path=/; Max-Age={max_age}");
if secure {
c.push_str("; Secure");
}
c
}
/// Build a `Set-Cookie` header value that clears the session cookie.
pub fn clear_session_cookie(secure: bool) -> String {
let mut c = format!("{SESSION_COOKIE}=; HttpOnly; SameSite=Lax; Path=/; Max-Age=0");
if secure {
c.push_str("; Secure");
}
c
}
/// Extract the value of a named cookie from a `Cookie` header.
fn cookie_value<'a>(parts: &'a Parts, name: &str) -> Option<&'a str> {
let header = parts.headers.get(COOKIE)?.to_str().ok()?;
header.split(';').find_map(|pair| {
let (k, v) = pair.trim().split_once('=')?;
(k == name).then_some(v)
})
}
/// Extract a `Bearer` token from the `Authorization` header.
fn bearer(parts: &Parts) -> Option<&str> {
let header = parts.headers.get(AUTHORIZATION)?.to_str().ok()?;
header.strip_prefix("Bearer ").map(str::trim)
}
/// An authenticated human user, resolved from the session cookie.
pub struct CurrentUser(pub User);
impl<S> FromRequestParts<S> for CurrentUser
where
AppState: FromRef<S>,
S: Send + Sync,
{
type Rejection = ApiError;
async fn from_request_parts(parts: &mut Parts, state: &S) -> Result<Self, Self::Rejection> {
let app = AppState::from_ref(state);
let value = cookie_value(parts, SESSION_COOKIE).ok_or_else(ApiError::unauthorized)?;
let user = service::authenticate_session(app.store.as_ref(), value).await?;
Ok(CurrentUser(user))
}
}
/// An authenticated API-token principal (agentic/algorithmic producer).
pub struct ApiPrincipal(pub ApiTokenInfo);
impl ApiPrincipal {
/// The user the token belongs to.
pub fn user_id(&self) -> uuid::Uuid {
self.0.user_id
}
}
impl<S> FromRequestParts<S> for ApiPrincipal
where
AppState: FromRef<S>,
S: Send + Sync,
{
type Rejection = ApiError;
async fn from_request_parts(parts: &mut Parts, state: &S) -> Result<Self, Self::Rejection> {
let app = AppState::from_ref(state);
let token = bearer(parts).ok_or_else(ApiError::unauthorized)?;
let info = service::authenticate_token(app.store.as_ref(), token).await?;
Ok(ApiPrincipal(info))
}
}

View File

@@ -0,0 +1,59 @@
//! Layered configuration: built-in defaults → optional TOML file → environment
//! (`NEWSFEED_*`). Matches the figment-style layering in architecture `generic.md` §3.
use std::net::SocketAddr;
use std::path::{Path, PathBuf};
use figment::Figment;
use figment::providers::{Env, Format, Serialized, Toml};
use serde::{Deserialize, Serialize};
/// API daemon configuration.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config {
/// Address to bind the HTTP listener. TLS terminates upstream at the site nginx.
pub bind: SocketAddr,
/// Path to the SQLite database file (shared with the worker on the same host).
pub database_path: PathBuf,
/// Session lifetime in days.
pub session_ttl_days: i64,
/// Maximum SQLite connections in the pool.
pub max_db_connections: u32,
/// Allowed CORS origins. Empty means same-origin only (the production shape, where
/// nginx serves the SPA and proxies the API under one origin).
pub cors_origins: Vec<String>,
/// Set `Secure` on the session cookie. True in production (HTTPS via nginx); can be
/// disabled for plain-HTTP local development.
pub cookie_secure: bool,
}
impl Default for Config {
fn default() -> Self {
Self {
bind: "127.0.0.1:8081".parse().expect("valid default bind addr"),
database_path: PathBuf::from("/var/lib/newsfeed/newsfeed.db"),
session_ttl_days: 30,
max_db_connections: 5,
cors_origins: Vec::new(),
cookie_secure: true,
}
}
}
impl Config {
/// Load configuration, layering an optional TOML file and the environment over the
/// defaults. Env vars are `NEWSFEED_` prefixed (e.g. `NEWSFEED_BIND=0.0.0.0:8081`).
pub fn load(file: Option<&Path>) -> anyhow::Result<Self> {
let mut fig = Figment::from(Serialized::defaults(Config::default()));
if let Some(path) = file {
fig = fig.merge(Toml::file(path));
}
let cfg: Config = fig.merge(Env::prefixed("NEWSFEED_")).extract()?;
Ok(cfg)
}
/// Session lifetime as a `chrono::Duration`.
pub fn session_ttl(&self) -> chrono::Duration {
chrono::Duration::days(self.session_ttl_days)
}
}

View File

@@ -0,0 +1,81 @@
//! HTTP error type. Maps [`CoreError`] and domain validation onto status codes and a
//! small JSON error envelope.
use axum::Json;
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use serde::Serialize;
use newsfeed_core::error::CoreError;
/// An API error carrying an HTTP status and a client-safe message.
#[derive(Debug)]
pub struct ApiError {
pub status: StatusCode,
pub message: String,
}
impl ApiError {
pub fn new(status: StatusCode, message: impl Into<String>) -> Self {
Self {
status,
message: message.into(),
}
}
pub fn unauthorized() -> Self {
Self::new(StatusCode::UNAUTHORIZED, "unauthorized")
}
pub fn not_found() -> Self {
Self::new(StatusCode::NOT_FOUND, "not found")
}
pub fn bad_request(message: impl Into<String>) -> Self {
Self::new(StatusCode::BAD_REQUEST, message)
}
}
#[derive(Serialize)]
struct ErrorBody {
error: String,
}
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
// Log server-side faults; client errors are self-explanatory.
if self.status.is_server_error() {
tracing::error!(status = %self.status, message = %self.message, "request failed");
}
(
self.status,
Json(ErrorBody {
error: self.message,
}),
)
.into_response()
}
}
impl From<CoreError> for ApiError {
fn from(e: CoreError) -> Self {
match e {
CoreError::Domain(d) => ApiError::bad_request(d.to_string()),
CoreError::NotFound => ApiError::not_found(),
CoreError::Conflict(m) => ApiError::new(StatusCode::CONFLICT, m),
CoreError::Unauthorized => ApiError::unauthorized(),
CoreError::Crypto(_) | CoreError::Storage(_) => {
ApiError::new(StatusCode::INTERNAL_SERVER_ERROR, "internal error")
}
}
}
}
impl From<newsfeed_entities::Error> for ApiError {
fn from(e: newsfeed_entities::Error) -> Self {
ApiError::bad_request(e.to_string())
}
}
/// Convenient result alias for handlers.
pub type ApiResult<T> = Result<T, ApiError>;

View File

@@ -0,0 +1,100 @@
//! newsfeed REST/JSON API daemon.
//!
//! Thin binary: parse CLI/config, initialise tracing, open the SQLite store (shared with
//! the worker on the same host), build the axum router, and serve until SIGTERM.
mod auth;
mod config;
mod error;
mod routes;
mod state;
use std::path::PathBuf;
use anyhow::Context;
use clap::Parser;
use tokio::net::TcpListener;
use tokio::signal;
use crate::config::Config;
use crate::state::AppState;
#[derive(Parser)]
#[command(
name = "newsfeed-api",
version,
about = "newsfeed REST/JSON API daemon"
)]
struct Cli {
/// Path to a TOML config file. Env (`NEWSFEED_*`) overrides file values.
#[arg(long, default_value = "/etc/newsfeed/config.toml")]
config: PathBuf,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
init_tracing();
let cli = Cli::parse();
let config_file = cli.config.exists().then_some(cli.config.as_path());
let config = Config::load(config_file).context("loading configuration")?;
tracing::info!(bind = %config.bind, db = %config.database_path.display(), "starting newsfeed-api");
if let Some(parent) = config.database_path.parent() {
std::fs::create_dir_all(parent).ok();
}
let store = newsfeed_data::connect(&config.database_path, config.max_db_connections)
.await
.context("opening database")?;
let bind = config.bind;
let state = AppState::new(store, config);
let app = routes::router(state);
let listener = TcpListener::bind(bind)
.await
.with_context(|| format!("binding {bind}"))?;
tracing::info!(%bind, "listening");
axum::serve(listener, app)
.with_graceful_shutdown(shutdown_signal())
.await
.context("server error")?;
tracing::info!("shut down cleanly");
Ok(())
}
/// JSON logs when running under journald (`JOURNAL_STREAM` set), pretty logs on a TTY.
fn init_tracing() {
use tracing_subscriber::{EnvFilter, fmt, prelude::*};
let filter = EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info"));
let registry = tracing_subscriber::registry().with(filter);
if std::env::var_os("JOURNAL_STREAM").is_some() {
registry
.with(fmt::layer().json().flatten_event(true))
.init();
} else {
registry.with(fmt::layer()).init();
}
}
/// Resolve on SIGTERM (systemd) or Ctrl-C.
async fn shutdown_signal() {
let ctrl_c = async {
signal::ctrl_c().await.expect("install Ctrl-C handler");
};
#[cfg(unix)]
let terminate = async {
signal::unix::signal(signal::unix::SignalKind::terminate())
.expect("install SIGTERM handler")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {},
_ = terminate => {},
}
tracing::info!("shutdown signal received");
}

View File

@@ -0,0 +1,83 @@
//! Registration, login, logout, and the current-principal endpoint.
use axum::Json;
use axum::extract::State;
use axum::http::{HeaderMap, StatusCode, header};
use axum::response::IntoResponse;
use newsfeed_core::service;
use newsfeed_entities::auth::Me;
use newsfeed_entities::user::{LoginRequest, RegisterRequest, User};
use crate::auth::{CurrentUser, clear_session_cookie, session_cookie};
use crate::error::ApiResult;
use crate::state::AppState;
/// `POST /v1/auth/register` — create an account. Does not log the user in.
pub async fn register(
State(app): State<AppState>,
Json(req): Json<RegisterRequest>,
) -> ApiResult<impl IntoResponse> {
let user = service::register(app.store.as_ref(), &req).await?;
Ok((StatusCode::CREATED, Json(to_me(user))))
}
/// `POST /v1/auth/login` — authenticate and open a session, setting the cookie.
pub async fn login(
State(app): State<AppState>,
Json(req): Json<LoginRequest>,
) -> ApiResult<impl IntoResponse> {
let outcome = service::login(app.store.as_ref(), &req, app.config.session_ttl()).await?;
let cookie = session_cookie(
&outcome.session_cookie,
app.config.session_ttl_days,
app.config.cookie_secure,
);
let mut headers = HeaderMap::new();
headers.insert(
header::SET_COOKIE,
cookie.parse().expect("valid cookie header"),
);
Ok((headers, Json(to_me(outcome.user))))
}
/// `POST /v1/auth/logout` — end the current session and clear the cookie.
pub async fn logout(
State(app): State<AppState>,
headers: HeaderMap,
) -> ApiResult<impl IntoResponse> {
if let Some(value) = session_cookie_value(&headers) {
service::logout(app.store.as_ref(), &value).await?;
}
let mut out = HeaderMap::new();
out.insert(
header::SET_COOKIE,
clear_session_cookie(app.config.cookie_secure)
.parse()
.expect("valid cookie header"),
);
Ok((StatusCode::NO_CONTENT, out))
}
/// `GET /v1/auth/me` — the authenticated user.
pub async fn me(CurrentUser(user): CurrentUser) -> Json<Me> {
Json(to_me(user))
}
fn to_me(user: User) -> Me {
Me {
id: user.id,
username: user.username,
email: user.email,
}
}
/// Read the raw session cookie value from request headers (logout, before the session is
/// resolved).
fn session_cookie_value(headers: &HeaderMap) -> Option<String> {
let header = headers.get(header::COOKIE)?.to_str().ok()?;
header.split(';').find_map(|pair| {
let (k, v) = pair.trim().split_once('=')?;
(k == crate::auth::SESSION_COOKIE).then(|| v.to_string())
})
}

View File

@@ -0,0 +1,77 @@
//! The feed read endpoint and interaction signals.
use axum::Json;
use axum::extract::{Query, State};
use axum::http::StatusCode;
use newsfeed_core::ports::ItemStore;
use newsfeed_entities::feed::{FeedPage, FeedQuery, Signal, SignalAction};
use newsfeed_entities::item::ItemState;
use crate::auth::CurrentUser;
use crate::error::{ApiError, ApiResult};
use crate::state::AppState;
const DEFAULT_LIMIT: i64 = 50;
const MAX_LIMIT: i64 = 100;
/// `GET /v1/feed` — a ranked, keyset-paginated page of the user's feed.
pub async fn get_feed(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Query(q): Query<FeedQuery>,
) -> ApiResult<Json<FeedPage>> {
let limit = q
.limit
.map(|l| l as i64)
.unwrap_or(DEFAULT_LIMIT)
.clamp(1, MAX_LIMIT);
// Fetch one extra to determine whether a further page exists.
let mut items = app
.store
.feed_page(user.id, limit + 1, q.cursor.as_deref(), q.include_saved)
.await?;
let next_cursor = if items.len() as i64 > limit {
items.truncate(limit as usize);
items.last().map(|it| format!("{}:{}", it.score, it.id))
} else {
None
};
Ok(Json(FeedPage { items, next_cursor }))
}
/// `POST /v1/feed/signals` — record an interaction. Save/Dismiss also move the item's
/// lifecycle state; View/Click are recorded for future ranking feedback.
pub async fn post_signal(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Json(sig): Json<Signal>,
) -> ApiResult<StatusCode> {
// The item must belong to the caller.
if app.store.get_item(user.id, sig.item_id).await?.is_none() {
return Err(ApiError::not_found());
}
app.store
.record_signal(user.id, sig.item_id, sig.action)
.await?;
match sig.action {
SignalAction::Save => {
app.store
.set_item_state(user.id, sig.item_id, ItemState::Saved)
.await?
}
SignalAction::Dismiss => {
app.store
.set_item_state(user.id, sig.item_id, ItemState::Dismissed)
.await?
}
SignalAction::View | SignalAction::Click => {}
}
Ok(StatusCode::NO_CONTENT)
}

View File

@@ -0,0 +1,15 @@
//! Liveness/readiness probe for systemd and the nginx upstream check.
use axum::extract::State;
use axum::http::StatusCode;
use crate::state::AppState;
/// `GET /health` — 200 when the database is reachable, 503 otherwise.
pub async fn health(State(app): State<AppState>) -> StatusCode {
if app.store.ping().await {
StatusCode::OK
} else {
StatusCode::SERVICE_UNAVAILABLE
}
}

View File

@@ -0,0 +1,29 @@
//! The inbound curation endpoint. Agentic and algorithmic producers authenticate with a
//! per-user bearer API token and POST content candidates; each is normalised, scored
//! against the owning user's interests, and promoted into their feed.
use axum::Json;
use axum::extract::State;
use axum::http::StatusCode;
use newsfeed_core::ingest;
use newsfeed_core::ports::InterestStore;
use newsfeed_entities::item::{CandidateSubmission, ContentItem};
use crate::auth::ApiPrincipal;
use crate::error::ApiResult;
use crate::state::AppState;
/// `POST /v1/ingest/candidates` — submit a candidate for curation into the token owner's
/// feed. Re-submitting an already-seen `external_id` returns the stored item unchanged
/// (idempotent), so producers can safely retry.
pub async fn submit(
principal: ApiPrincipal,
State(app): State<AppState>,
Json(sub): Json<CandidateSubmission>,
) -> ApiResult<(StatusCode, Json<ContentItem>)> {
let user_id = principal.user_id();
let interests = app.store.list_interests(user_id).await?;
let item = ingest::ingest_submission(app.store.as_ref(), user_id, &interests, sub).await?;
Ok((StatusCode::ACCEPTED, Json(item)))
}

View File

@@ -0,0 +1,41 @@
//! The user-controlled interest weightings that drive ranking.
use axum::Json;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use uuid::Uuid;
use newsfeed_core::ports::InterestStore;
use newsfeed_entities::interest::{Interest, UpsertInterest};
use crate::auth::CurrentUser;
use crate::error::ApiResult;
use crate::state::AppState;
/// `GET /v1/interests` — the user's interests, strongest first.
pub async fn list(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
) -> ApiResult<Json<Vec<Interest>>> {
Ok(Json(app.store.list_interests(user.id).await?))
}
/// `PUT /v1/interests` — create or update an interest by label.
pub async fn upsert(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Json(body): Json<UpsertInterest>,
) -> ApiResult<Json<Interest>> {
body.validate()?;
Ok(Json(app.store.upsert_interest(user.id, &body).await?))
}
/// `DELETE /v1/interests/{id}` — remove an interest.
pub async fn remove(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Path(id): Path<Uuid>,
) -> ApiResult<StatusCode> {
app.store.delete_interest(user.id, id).await?;
Ok(StatusCode::NO_CONTENT)
}

View File

@@ -0,0 +1,74 @@
//! Router assembly. Public API is versioned under `/v1` from day one; `/health` is
//! unversioned for infrastructure probes.
mod auth_routes;
mod feed;
mod health;
mod ingest;
mod interests;
mod sources;
mod tokens;
use axum::Router;
use axum::http::{Method, header};
use axum::routing::{delete, get, post};
use tower_http::compression::CompressionLayer;
use tower_http::cors::CorsLayer;
use tower_http::trace::TraceLayer;
use crate::state::AppState;
/// Build the full application router.
pub fn router(state: AppState) -> Router {
let cors = build_cors(&state);
let v1 = Router::new()
// auth
.route("/auth/register", post(auth_routes::register))
.route("/auth/login", post(auth_routes::login))
.route("/auth/logout", post(auth_routes::logout))
.route("/auth/me", get(auth_routes::me))
// feed
.route("/feed", get(feed::get_feed))
.route("/feed/signals", post(feed::post_signal))
// sources
.route("/sources", get(sources::list).post(sources::create))
.route("/sources/{id}", delete(sources::remove))
// interests
.route("/interests", get(interests::list).put(interests::upsert))
.route("/interests/{id}", delete(interests::remove))
// api tokens
.route("/tokens", get(tokens::list).post(tokens::create))
.route("/tokens/{id}", delete(tokens::revoke))
// ingest (bearer-token authenticated)
.route("/ingest/candidates", post(ingest::submit));
Router::new()
.route("/health", get(health::health))
.nest("/v1", v1)
.layer(TraceLayer::new_for_http())
.layer(CompressionLayer::new())
.layer(cors)
.with_state(state)
}
/// CORS is only needed in development, where the Vite dev server is a different origin
/// from the API. In production nginx serves the SPA and proxies the API under one origin,
/// so `cors_origins` is empty and no cross-origin headers are emitted.
fn build_cors(state: &AppState) -> CorsLayer {
if state.config.cors_origins.is_empty() {
return CorsLayer::new();
}
let origins = state
.config
.cors_origins
.iter()
.filter_map(|o| o.parse().ok())
.collect::<Vec<_>>();
// Credentialed CORS forbids wildcard method/header lists, so enumerate them.
CorsLayer::new()
.allow_origin(origins)
.allow_methods([Method::GET, Method::POST, Method::PUT, Method::DELETE])
.allow_headers([header::CONTENT_TYPE, header::AUTHORIZATION])
.allow_credentials(true)
}

View File

@@ -0,0 +1,42 @@
//! CRUD for a user's content sources.
use axum::Json;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use uuid::Uuid;
use newsfeed_core::ports::SourceStore;
use newsfeed_entities::source::{NewSource, Source};
use crate::auth::CurrentUser;
use crate::error::ApiResult;
use crate::state::AppState;
/// `GET /v1/sources` — the user's sources.
pub async fn list(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
) -> ApiResult<Json<Vec<Source>>> {
Ok(Json(app.store.list_sources(user.id).await?))
}
/// `POST /v1/sources` — add a source.
pub async fn create(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Json(new): Json<NewSource>,
) -> ApiResult<(StatusCode, Json<Source>)> {
new.validate()?;
let source = app.store.create_source(user.id, &new).await?;
Ok((StatusCode::CREATED, Json(source)))
}
/// `DELETE /v1/sources/{id}` — remove a source.
pub async fn remove(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Path(id): Path<Uuid>,
) -> ApiResult<StatusCode> {
app.store.delete_source(user.id, id).await?;
Ok(StatusCode::NO_CONTENT)
}

View File

@@ -0,0 +1,45 @@
//! Per-user API tokens used by agentic/algorithmic producers to POST candidates.
use axum::Json;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use uuid::Uuid;
use newsfeed_core::ports::TokenStore;
use newsfeed_core::service;
use newsfeed_entities::auth::{ApiTokenInfo, CreateApiToken, CreatedApiToken};
use crate::auth::CurrentUser;
use crate::error::{ApiError, ApiResult};
use crate::state::AppState;
/// `GET /v1/tokens` — list the caller's tokens (metadata only; secrets never returned).
pub async fn list(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
) -> ApiResult<Json<Vec<ApiTokenInfo>>> {
Ok(Json(app.store.list_tokens(user.id).await?))
}
/// `POST /v1/tokens` — mint a token. The full secret is returned exactly once.
pub async fn create(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Json(body): Json<CreateApiToken>,
) -> ApiResult<(StatusCode, Json<CreatedApiToken>)> {
if body.name.trim().is_empty() {
return Err(ApiError::bad_request("token name must not be empty"));
}
let created = service::create_api_token(app.store.as_ref(), user.id, body.name.trim()).await?;
Ok((StatusCode::CREATED, Json(created)))
}
/// `DELETE /v1/tokens/{id}` — revoke a token.
pub async fn revoke(
CurrentUser(user): CurrentUser,
State(app): State<AppState>,
Path(id): Path<Uuid>,
) -> ApiResult<StatusCode> {
app.store.revoke_token(user.id, id).await?;
Ok(StatusCode::NO_CONTENT)
}

View File

@@ -0,0 +1,24 @@
//! Shared application state. The binary wires the concrete SQLite adapter here; handlers
//! reach the ports through it. `AppState` is cheap to clone (Arc-wrapped).
use std::sync::Arc;
use newsfeed_data::SqliteStore;
use crate::config::Config;
/// State shared across all handlers.
#[derive(Clone)]
pub struct AppState {
pub store: Arc<SqliteStore>,
pub config: Arc<Config>,
}
impl AppState {
pub fn new(store: SqliteStore, config: Config) -> Self {
Self {
store: Arc::new(store),
config: Arc::new(config),
}
}
}

View File

@@ -0,0 +1,23 @@
[package]
name = "newsfeed-core"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
description = "Business logic and data-access ports for newsfeed. No direct I/O."
[dependencies]
newsfeed-entities.workspace = true
serde.workspace = true
chrono.workspace = true
uuid.workspace = true
thiserror.workspace = true
async-trait.workspace = true
# pure crypto/compute — no network or DB
argon2.workspace = true
rand.workspace = true
sha2.workspace = true
base64.workspace = true

View File

@@ -0,0 +1,126 @@
//! Credential primitives: password hashing, opaque session tokens, and API tokens.
//!
//! Secrets are generated here and only their hashes are handed to the store. Session
//! cookies are random and hashed with SHA-256 (they are high-entropy, so a fast hash is
//! appropriate); passwords are hashed with Argon2id (slow, salted) because they are
//! low-entropy and attacker-guessable.
use argon2::Argon2;
use argon2::password_hash::{
PasswordHash, PasswordHasher, PasswordVerifier, SaltString, rand_core::OsRng,
};
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use rand::RngCore;
use sha2::{Digest, Sha256};
use crate::error::{CoreError, CoreResult};
/// Hash a plaintext password with Argon2id and a fresh random salt.
pub fn hash_password(plaintext: &str) -> CoreResult<String> {
let salt = SaltString::generate(&mut OsRng);
Argon2::default()
.hash_password(plaintext.as_bytes(), &salt)
.map(|h| h.to_string())
.map_err(|e| CoreError::Crypto(e.to_string()))
}
/// Verify a plaintext password against a stored Argon2 PHC hash.
pub fn verify_password(plaintext: &str, phc_hash: &str) -> CoreResult<bool> {
let parsed = PasswordHash::new(phc_hash).map_err(|e| CoreError::Crypto(e.to_string()))?;
Ok(Argon2::default()
.verify_password(plaintext.as_bytes(), &parsed)
.is_ok())
}
/// A freshly minted secret: the plaintext to hand to the client exactly once, plus the
/// hash to persist.
pub struct MintedSecret {
/// The value the client keeps (cookie value or bearer token).
pub plaintext: String,
/// The SHA-256 hash to store and look up by.
pub hash: String,
}
fn random_token(bytes: usize) -> String {
let mut buf = vec![0u8; bytes];
OsRng.fill_bytes(&mut buf);
URL_SAFE_NO_PAD.encode(buf)
}
/// SHA-256 hash of an opaque high-entropy secret, hex-encoded. Used for both session
/// cookies and API tokens so the store never holds the secret in the clear.
pub fn hash_opaque(secret: &str) -> String {
let digest = Sha256::digest(secret.as_bytes());
hex_encode(&digest)
}
/// Generate a new opaque session token (256 bits of entropy).
pub fn mint_session_token() -> MintedSecret {
let plaintext = random_token(32);
let hash = hash_opaque(&plaintext);
MintedSecret { plaintext, hash }
}
/// A newly minted API token, with its display prefix.
pub struct MintedApiToken {
/// Full bearer token: `nf_<prefix>_<random>`.
pub secret: String,
/// Short non-secret prefix shown to the user (`nf_<prefix>`).
pub prefix: String,
/// SHA-256 hash of the full secret, for storage.
pub hash: String,
}
/// Prefix on every API token. Lets the ingest handler cheaply reject non-tokens.
pub const API_TOKEN_PREFIX: &str = "nf";
/// Generate a new API token of the form `nf_<6hex>_<random>`.
pub fn mint_api_token() -> MintedApiToken {
let mut id = [0u8; 3];
OsRng.fill_bytes(&mut id);
let short = hex_encode(&id); // 6 hex chars
let prefix = format!("{API_TOKEN_PREFIX}_{short}");
let secret = format!("{prefix}_{}", random_token(24));
let hash = hash_opaque(&secret);
MintedApiToken {
secret,
prefix,
hash,
}
}
fn hex_encode(bytes: &[u8]) -> String {
use std::fmt::Write;
let mut s = String::with_capacity(bytes.len() * 2);
for b in bytes {
let _ = write!(s, "{b:02x}");
}
s
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn password_roundtrip() {
let hash = hash_password("correct horse battery staple").unwrap();
assert!(verify_password("correct horse battery staple", &hash).unwrap());
assert!(!verify_password("wrong", &hash).unwrap());
}
#[test]
fn opaque_hash_is_stable_and_matches() {
let minted = mint_session_token();
assert_eq!(minted.hash, hash_opaque(&minted.plaintext));
}
#[test]
fn api_token_has_prefix() {
let t = mint_api_token();
assert!(t.secret.starts_with(&t.prefix));
assert!(t.prefix.starts_with("nf_"));
assert_eq!(t.hash, hash_opaque(&t.secret));
}
}

View File

@@ -0,0 +1,36 @@
//! Error type for the core layer. Wraps domain validation errors and the abstract
//! failure modes a data adapter can surface, without naming any concrete backend.
use thiserror::Error;
/// Errors produced by core business logic and its data ports.
#[derive(Debug, Error)]
pub enum CoreError {
/// A domain value failed validation.
#[error(transparent)]
Domain(#[from] newsfeed_entities::Error),
/// The requested entity does not exist.
#[error("not found")]
NotFound,
/// A uniqueness constraint was violated (duplicate username, email, …).
#[error("conflict: {0}")]
Conflict(String),
/// Authentication failed (bad password, unknown/expired session or token).
#[error("unauthorized")]
Unauthorized,
/// A password/token hashing operation failed.
#[error("crypto error: {0}")]
Crypto(String),
/// An adapter (the `data` crate) failed for an implementation-specific reason.
/// The concrete error is stringified so `core` needn't depend on any backend.
#[error("storage error: {0}")]
Storage(String),
}
/// Result alias for the core layer.
pub type CoreResult<T> = std::result::Result<T, CoreError>;

View File

@@ -0,0 +1,121 @@
//! Candidate normalisation. Turns an inbound [`CandidateSubmission`] (from an agentic
//! producer) or a parsed feed entry into the [`NewCandidate`] shape the store persists.
use chrono::{DateTime, Utc};
use uuid::Uuid;
use newsfeed_entities::interest::Interest;
use newsfeed_entities::item::{CandidateSubmission, ContentItem, ItemState, Media};
use crate::error::CoreResult;
use crate::ports::{ItemStore, NewCandidate, SourceStore};
use crate::ranking::{self, RankingParams};
/// Baseline weight applied to candidates whose named source doesn't (yet) exist as a
/// configured source for the user. Configured sources carry their own weight.
pub const DEFAULT_UNLINKED_WEIGHT: f64 = 0.5;
/// Normalise an agentic submission for a given user and (optional) resolved source.
pub fn candidate_from_submission(
user_id: Uuid,
source_id: Option<Uuid>,
sub: CandidateSubmission,
) -> NewCandidate {
NewCandidate {
user_id,
source_id,
external_id: sub.external_id,
url: sub.url,
title: sub.title,
summary: sub.summary,
author: sub.author,
tags: sub.tags,
media: sub.media,
published_at: sub.published_at,
}
}
/// Fields extracted from a polled feed entry, backend-agnostic so the worker's feed
/// parser (`feed-rs`) doesn't leak into `core`.
pub struct FeedEntry {
pub external_id: String,
pub url: Option<String>,
pub title: String,
pub summary: Option<String>,
pub author: Option<String>,
pub tags: Vec<String>,
pub media: Vec<Media>,
pub published_at: Option<DateTime<Utc>>,
}
/// Normalise a polled feed entry for a user's RSS source.
pub fn candidate_from_feed_entry(user_id: Uuid, source_id: Uuid, entry: FeedEntry) -> NewCandidate {
NewCandidate {
user_id,
source_id: Some(source_id),
external_id: entry.external_id,
url: entry.url,
title: entry.title,
summary: entry.summary,
author: entry.author,
tags: entry.tags,
media: entry.media,
published_at: entry.published_at,
}
}
/// Persist a normalised candidate and, if it is newly created, score it against the
/// user's current interests and promote it into the live feed. Re-submitting an existing
/// `(user, external_id)` is a no-op that returns the stored item unchanged. `interests`
/// is passed in so the caller can load it once per batch.
pub async fn persist_and_rank<S>(
store: &S,
candidate: NewCandidate,
source_weight: f64,
interests: &[Interest],
) -> CoreResult<ContentItem>
where
S: ItemStore,
{
let (mut item, created) = store.upsert_candidate(&candidate).await?;
if created {
let score = ranking::score_item(
&item,
source_weight,
interests,
Utc::now(),
&RankingParams::default(),
);
store.update_score(item.id, score).await?;
store
.set_item_state(item.user_id, item.id, ItemState::Feed)
.await?;
item.score = score;
item.state = ItemState::Feed;
}
Ok(item)
}
/// Ingest an agentic submission: resolve its named source (if any), normalise, then
/// [`persist_and_rank`]. Used by `POST /v1/ingest/candidates`.
pub async fn ingest_submission<S>(
store: &S,
user_id: Uuid,
interests: &[Interest],
sub: CandidateSubmission,
) -> CoreResult<ContentItem>
where
S: SourceStore + ItemStore,
{
let source = match &sub.source {
Some(name) => store.find_source_by_name(user_id, name).await?,
None => None,
};
let source_weight = source
.as_ref()
.map(|s| s.weight)
.unwrap_or(DEFAULT_UNLINKED_WEIGHT);
let source_id = source.as_ref().map(|s| s.id);
let candidate = candidate_from_submission(user_id, source_id, sub);
persist_and_rank(store, candidate, source_weight, interests).await
}

View File

@@ -0,0 +1,15 @@
//! Business logic and data-access ports for newsfeed.
//!
//! This crate consumes [`newsfeed_entities`] and defines the *ports* (traits in
//! [`ports`]) that `newsfeed-data` implements as adapters. It also holds pure compute:
//! password/token hashing ([`auth`]), the feed ranker ([`ranking`]), and candidate
//! normalisation ([`ingest`]). It performs no I/O of its own.
pub mod auth;
pub mod error;
pub mod ingest;
pub mod ports;
pub mod ranking;
pub mod service;
pub use error::{CoreError, CoreResult};

View File

@@ -0,0 +1,156 @@
//! Data-access ports. `newsfeed-data` provides the adapters that implement these; the
//! binaries depend only on the [`Store`] aggregate, never on a concrete backend.
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use uuid::Uuid;
use newsfeed_entities::auth::{ApiTokenInfo, Session};
use newsfeed_entities::interest::{Interest, UpsertInterest};
use newsfeed_entities::item::ContentItem;
use newsfeed_entities::source::{NewSource, Source};
use newsfeed_entities::user::User;
use crate::error::CoreResult;
/// A user together with the password hash needed to authenticate them. Never serialised
/// to a client — it exists only to move the hash from the store to [`crate::auth`].
pub struct StoredCredential {
pub user: User,
pub password_hash: String,
}
/// Fields required to persist a candidate. Normalised from a submission or a polled feed
/// entry by [`crate::ingest`].
pub struct NewCandidate {
pub user_id: Uuid,
pub source_id: Option<Uuid>,
pub external_id: String,
pub url: Option<String>,
pub title: String,
pub summary: Option<String>,
pub author: Option<String>,
pub tags: Vec<String>,
pub media: Vec<newsfeed_entities::item::Media>,
pub published_at: Option<DateTime<Utc>>,
}
/// User accounts.
#[async_trait]
pub trait UserStore: Send + Sync {
async fn create_user(
&self,
username: &str,
email: &str,
password_hash: &str,
) -> CoreResult<User>;
async fn get_user(&self, id: Uuid) -> CoreResult<Option<User>>;
/// Look up by username or email, returning the stored password hash for verification.
async fn find_credential(&self, identifier: &str) -> CoreResult<Option<StoredCredential>>;
}
/// Browser sessions.
#[async_trait]
pub trait SessionStore: Send + Sync {
async fn create_session(
&self,
user_id: Uuid,
token_hash: &str,
expires_at: DateTime<Utc>,
) -> CoreResult<Session>;
/// Return the session for this cookie hash iff it exists and has not expired.
async fn find_session(&self, token_hash: &str) -> CoreResult<Option<Session>>;
async fn delete_session(&self, token_hash: &str) -> CoreResult<()>;
}
/// Per-user API tokens used by agentic/algorithmic ingest producers.
#[async_trait]
pub trait TokenStore: Send + Sync {
async fn create_token(
&self,
user_id: Uuid,
name: &str,
prefix: &str,
token_hash: &str,
) -> CoreResult<ApiTokenInfo>;
async fn list_tokens(&self, user_id: Uuid) -> CoreResult<Vec<ApiTokenInfo>>;
/// Resolve a bearer-token hash to its (non-revoked) owner and record use.
async fn authenticate_token(&self, token_hash: &str) -> CoreResult<Option<ApiTokenInfo>>;
async fn revoke_token(&self, user_id: Uuid, id: Uuid) -> CoreResult<()>;
}
/// Content sources.
#[async_trait]
pub trait SourceStore: Send + Sync {
async fn create_source(&self, user_id: Uuid, new: &NewSource) -> CoreResult<Source>;
async fn list_sources(&self, user_id: Uuid) -> CoreResult<Vec<Source>>;
async fn find_source_by_name(&self, user_id: Uuid, name: &str) -> CoreResult<Option<Source>>;
async fn delete_source(&self, user_id: Uuid, id: Uuid) -> CoreResult<()>;
/// RSS sources whose next poll is due (worker use).
async fn due_rss_sources(
&self,
now: DateTime<Utc>,
min_interval_secs: i64,
limit: i64,
) -> CoreResult<Vec<Source>>;
async fn mark_polled(&self, source_id: Uuid, at: DateTime<Utc>) -> CoreResult<()>;
}
/// User interests / weightings.
#[async_trait]
pub trait InterestStore: Send + Sync {
async fn list_interests(&self, user_id: Uuid) -> CoreResult<Vec<Interest>>;
async fn upsert_interest(&self, user_id: Uuid, upsert: &UpsertInterest)
-> CoreResult<Interest>;
async fn delete_interest(&self, user_id: Uuid, id: Uuid) -> CoreResult<()>;
}
/// Content items and the feed.
#[async_trait]
pub trait ItemStore: Send + Sync {
/// Insert a candidate, or return the existing row if `(user_id, external_id)` already
/// exists. `true` in the tuple means a new row was created.
async fn upsert_candidate(&self, candidate: &NewCandidate) -> CoreResult<(ContentItem, bool)>;
async fn get_item(&self, user_id: Uuid, id: Uuid) -> CoreResult<Option<ContentItem>>;
async fn set_item_state(
&self,
user_id: Uuid,
id: Uuid,
state: newsfeed_entities::item::ItemState,
) -> CoreResult<()>;
async fn update_score(&self, item_id: Uuid, score: f64) -> CoreResult<()>;
/// A ranked page of the user's feed, ordered by score descending.
async fn feed_page(
&self,
user_id: Uuid,
limit: i64,
cursor: Option<&str>,
include_saved: bool,
) -> CoreResult<Vec<ContentItem>>;
/// All feed/candidate items for a user, for (re)scoring by the worker.
async fn items_for_scoring(&self, user_id: Uuid, limit: i64) -> CoreResult<Vec<ContentItem>>;
/// Distinct user ids that own at least one source (worker iteration).
async fn user_ids_with_sources(&self) -> CoreResult<Vec<Uuid>>;
/// Record an interaction signal against an item. This is the feedback the ranker can
/// later weigh (still under the user's control — signals inform, they don't override
/// the user's explicit weights).
async fn record_signal(
&self,
user_id: Uuid,
item_id: Uuid,
action: newsfeed_entities::feed::SignalAction,
) -> CoreResult<()>;
}
/// The aggregate every binary depends on. Any type implementing all the ports is a
/// `Store`, so the concrete adapter satisfies it automatically.
pub trait Store:
UserStore + SessionStore + TokenStore + SourceStore + InterestStore + ItemStore
{
}
impl<T> Store for T where
T: UserStore + SessionStore + TokenStore + SourceStore + InterestStore + ItemStore
{
}

View File

@@ -0,0 +1,156 @@
//! The feed ranker. This is the whole point of newsfeed: the score assigned to an item
//! is a transparent, deterministic function of weights the *user* controls — their
//! per-source baseline weight and their per-interest weights — decayed by age. No
//! opaque engagement model, no third party deciding what surfaces.
use chrono::{DateTime, Utc};
use newsfeed_entities::interest::Interest;
use newsfeed_entities::item::ContentItem;
/// Tunable ranking parameters. Defaults are sensible; a future per-user settings row can
/// override them so the decay curve itself is user-controlled too.
#[derive(Debug, Clone, Copy)]
pub struct RankingParams {
/// Hours after which recency weight halves.
pub half_life_hours: f64,
}
impl Default for RankingParams {
fn default() -> Self {
Self {
half_life_hours: 24.0,
}
}
}
/// Compute the relevance component from the user's interests: the sum of the weights of
/// every interest whose (lower-cased) label occurs in the item's title, summary or tags.
/// Negative-weighted interests subtract, actively burying matching content.
pub fn interest_relevance(item: &ContentItem, interests: &[Interest]) -> f64 {
if interests.is_empty() {
return 0.0;
}
let haystack = {
let mut s = String::new();
s.push_str(&item.title.to_lowercase());
s.push(' ');
if let Some(summary) = &item.summary {
s.push_str(&summary.to_lowercase());
s.push(' ');
}
for tag in &item.tags {
s.push_str(&tag.to_lowercase());
s.push(' ');
}
s
};
interests
.iter()
.filter(|i| {
let label = i.label.trim().to_lowercase();
!label.is_empty() && haystack.contains(&label)
})
.map(|i| i.weight)
.sum()
}
/// Recency multiplier in `(0.0, 1.0]` using exponential decay. Items with no publish
/// date are treated as freshly ingested (multiplier `1.0`).
pub fn recency_multiplier(
published_at: Option<DateTime<Utc>>,
now: DateTime<Utc>,
params: &RankingParams,
) -> f64 {
let Some(published) = published_at else {
return 1.0;
};
let age_hours = (now - published).num_seconds() as f64 / 3600.0;
if age_hours <= 0.0 {
return 1.0;
}
0.5f64.powf(age_hours / params.half_life_hours)
}
/// Score a single item for a user. `source_weight` is the item's source baseline
/// (`0.0..=1.0`); `interests` are the user's weighted interests.
///
/// `score = (source_weight + interest_relevance) * recency_multiplier`
pub fn score_item(
item: &ContentItem,
source_weight: f64,
interests: &[Interest],
now: DateTime<Utc>,
params: &RankingParams,
) -> f64 {
let relevance = source_weight + interest_relevance(item, interests);
relevance * recency_multiplier(item.published_at, now, params)
}
#[cfg(test)]
mod tests {
use super::*;
use chrono::TimeZone;
use uuid::Uuid;
fn item(title: &str, published: Option<DateTime<Utc>>) -> ContentItem {
ContentItem {
id: Uuid::nil(),
user_id: Uuid::nil(),
source_id: None,
external_id: "x".into(),
url: None,
title: title.into(),
summary: None,
author: None,
tags: vec![],
media: vec![],
published_at: published,
ingested_at: Utc::now(),
state: newsfeed_entities::item::ItemState::Feed,
score: 0.0,
}
}
fn interest(label: &str, weight: f64) -> Interest {
Interest {
id: Uuid::nil(),
user_id: Uuid::nil(),
label: label.into(),
weight,
created_at: Utc::now(),
}
}
#[test]
fn positive_interest_raises_score() {
let now = Utc.with_ymd_and_hms(2026, 7, 8, 12, 0, 0).unwrap();
let it = item("Rust 2.0 released", Some(now));
let interests = vec![interest("rust", 0.8)];
let with = score_item(&it, 0.5, &interests, now, &RankingParams::default());
let without = score_item(&it, 0.5, &[], now, &RankingParams::default());
assert!(with > without);
}
#[test]
fn negative_interest_buries() {
let now = Utc.with_ymd_and_hms(2026, 7, 8, 12, 0, 0).unwrap();
let it = item("Celebrity gossip roundup", Some(now));
let interests = vec![interest("gossip", -0.9)];
let score = score_item(&it, 0.5, &interests, now, &RankingParams::default());
assert!(score < 0.0);
}
#[test]
fn older_items_decay() {
let now = Utc.with_ymd_and_hms(2026, 7, 8, 12, 0, 0).unwrap();
let fresh = item("news", Some(now));
let day_old = item("news", Some(now - chrono::Duration::hours(24)));
let p = RankingParams::default();
let fresh_score = score_item(&fresh, 1.0, &[], now, &p);
let old_score = score_item(&day_old, 1.0, &[], now, &p);
assert!(fresh_score > old_score);
// one half-life => ~half the score
assert!((old_score - 0.5).abs() < 0.01);
}
}

View File

@@ -0,0 +1,131 @@
//! Service-layer orchestration that spans the ports: registration, login, and principal
//! resolution. The binaries call these so the auth flow lives in one tested place rather
//! than being re-implemented per handler.
use chrono::{Duration, Utc};
use newsfeed_entities::auth::{ApiTokenInfo, CreatedApiToken};
use newsfeed_entities::user::{LoginRequest, RegisterRequest, User};
use std::collections::HashMap;
use crate::auth;
use crate::error::{CoreError, CoreResult};
use crate::ports::{InterestStore, ItemStore, SessionStore, SourceStore, TokenStore, UserStore};
use crate::ranking::{self, RankingParams};
/// Register a new user. Validates the request, hashes the password, and persists.
pub async fn register(store: &impl UserStore, req: &RegisterRequest) -> CoreResult<User> {
req.validate()?;
let hash = auth::hash_password(&req.password)?;
store
.create_user(req.username.trim(), req.email.trim(), &hash)
.await
}
/// The outcome of a successful login: the user plus the opaque session cookie value to
/// set (returned in the clear exactly once).
pub struct LoggedIn {
pub user: User,
/// The cookie value to hand to the browser.
pub session_cookie: String,
}
/// Authenticate a login request and open a session valid for `ttl`.
pub async fn login<S>(store: &S, req: &LoginRequest, ttl: Duration) -> CoreResult<LoggedIn>
where
S: UserStore + SessionStore,
{
let cred = store
.find_credential(req.identifier.trim())
.await?
.ok_or(CoreError::Unauthorized)?;
if !auth::verify_password(&req.password, &cred.password_hash)? {
return Err(CoreError::Unauthorized);
}
let minted = auth::mint_session_token();
let expires_at = Utc::now() + ttl;
store
.create_session(cred.user.id, &minted.hash, expires_at)
.await?;
Ok(LoggedIn {
user: cred.user,
session_cookie: minted.plaintext,
})
}
/// Resolve a session cookie value to its user, or `Unauthorized`.
pub async fn authenticate_session<S>(store: &S, cookie_value: &str) -> CoreResult<User>
where
S: SessionStore + UserStore,
{
let hash = auth::hash_opaque(cookie_value);
let session = store
.find_session(&hash)
.await?
.ok_or(CoreError::Unauthorized)?;
store
.get_user(session.user_id)
.await?
.ok_or(CoreError::Unauthorized)
}
/// Log out: delete the session backing this cookie value. Idempotent.
pub async fn logout(store: &impl SessionStore, cookie_value: &str) -> CoreResult<()> {
store.delete_session(&auth::hash_opaque(cookie_value)).await
}
/// Resolve a bearer API token to its owning token record, recording use.
pub async fn authenticate_token(store: &impl TokenStore, bearer: &str) -> CoreResult<ApiTokenInfo> {
let hash = auth::hash_opaque(bearer);
store
.authenticate_token(&hash)
.await?
.ok_or(CoreError::Unauthorized)
}
/// Recompute scores for a user's live/candidate items against their current interests
/// and per-source weights. Run periodically by the worker so that changing a weight
/// re-ranks the existing feed, not just future items. Returns the number rescored.
pub async fn rescore_user<S>(store: &S, user_id: uuid::Uuid, limit: i64) -> CoreResult<usize>
where
S: SourceStore + InterestStore + ItemStore,
{
let interests = store.list_interests(user_id).await?;
let weights: HashMap<uuid::Uuid, f64> = store
.list_sources(user_id)
.await?
.into_iter()
.map(|s| (s.id, s.weight))
.collect();
let items = store.items_for_scoring(user_id, limit).await?;
let now = Utc::now();
let params = RankingParams::default();
let mut n = 0;
for item in items {
let source_weight = item
.source_id
.and_then(|id| weights.get(&id).copied())
.unwrap_or(crate::ingest::DEFAULT_UNLINKED_WEIGHT);
let score = ranking::score_item(&item, source_weight, &interests, now, &params);
store.update_score(item.id, score).await?;
n += 1;
}
Ok(n)
}
/// Mint a new API token for a user, returning the secret exactly once.
pub async fn create_api_token(
store: &impl TokenStore,
user_id: uuid::Uuid,
name: &str,
) -> CoreResult<CreatedApiToken> {
let minted = auth::mint_api_token();
let info = store
.create_token(user_id, name, &minted.prefix, &minted.hash)
.await?;
Ok(CreatedApiToken {
info,
secret: minted.secret,
})
}

View File

@@ -0,0 +1,23 @@
[package]
name = "newsfeed-data"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
description = "SQLite data-access adapters implementing the newsfeed-core ports."
[dependencies]
newsfeed-entities.workspace = true
newsfeed-core.workspace = true
sqlx.workspace = true
serde_json.workspace = true
uuid.workspace = true
chrono.workspace = true
async-trait.workspace = true
tracing.workspace = true
anyhow.workspace = true
[dev-dependencies]
tokio.workspace = true

View File

@@ -0,0 +1,92 @@
-- newsfeed initial schema (SQLite).
--
-- Conventions for this datastore:
-- * ids TEXT — hyphenated UUIDv4
-- * timestamps TEXT — RFC3339 / ISO8601 (UTC)
-- * booleans INTEGER — 0/1
-- * json blobs TEXT — serde_json arrays/objects
--
-- Every user-owned row carries user_id and cascades on user deletion, because each
-- user is wholly in control of — and isolated within — their own feed.
CREATE TABLE users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
email TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
created_at TEXT NOT NULL,
expires_at TEXT NOT NULL
);
CREATE INDEX idx_sessions_user ON sessions(user_id);
CREATE TABLE api_tokens (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
prefix TEXT NOT NULL,
token_hash TEXT NOT NULL UNIQUE,
created_at TEXT NOT NULL,
last_used_at TEXT,
revoked_at TEXT
);
CREATE INDEX idx_api_tokens_user ON api_tokens(user_id);
CREATE TABLE sources (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
kind TEXT NOT NULL, -- 'rss' | 'agentic'
name TEXT NOT NULL,
url TEXT,
weight REAL NOT NULL DEFAULT 1.0,
enabled INTEGER NOT NULL DEFAULT 1,
last_polled_at TEXT,
created_at TEXT NOT NULL,
UNIQUE(user_id, name)
);
CREATE INDEX idx_sources_user ON sources(user_id);
CREATE TABLE interests (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
label TEXT NOT NULL,
weight REAL NOT NULL,
created_at TEXT NOT NULL,
UNIQUE(user_id, label)
);
CREATE INDEX idx_interests_user ON interests(user_id);
CREATE TABLE items (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
source_id TEXT REFERENCES sources(id) ON DELETE SET NULL,
external_id TEXT NOT NULL,
url TEXT,
title TEXT NOT NULL,
summary TEXT,
author TEXT,
tags TEXT NOT NULL DEFAULT '[]', -- json array of strings
media TEXT NOT NULL DEFAULT '[]', -- json array of Media
published_at TEXT,
ingested_at TEXT NOT NULL,
state TEXT NOT NULL DEFAULT 'candidate',
score REAL NOT NULL DEFAULT 0.0,
UNIQUE(user_id, external_id)
);
-- Feed reads order by score within a user's live items.
CREATE INDEX idx_items_feed ON items(user_id, state, score DESC, id DESC);
CREATE TABLE signals (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
item_id TEXT NOT NULL REFERENCES items(id) ON DELETE CASCADE,
action TEXT NOT NULL, -- 'view' | 'click' | 'save' | 'dismiss'
created_at TEXT NOT NULL
);
CREATE INDEX idx_signals_item ON signals(item_id);

View File

@@ -0,0 +1,24 @@
//! Maps `sqlx` failures onto the backend-agnostic [`CoreError`] the ports speak.
use newsfeed_core::error::CoreError;
/// Convert a `sqlx::Error` into a [`CoreError`], distinguishing unique-constraint
/// violations (→ [`CoreError::Conflict`]) from everything else (→ storage error).
pub fn map(e: sqlx::Error) -> CoreError {
if let sqlx::Error::Database(db) = &e {
if db.is_unique_violation() {
return CoreError::Conflict(db.message().to_string());
}
}
CoreError::Storage(e.to_string())
}
/// Convert a JSON (de)serialisation failure in the mapping layer into a storage error.
pub fn json(e: serde_json::Error) -> CoreError {
CoreError::Storage(format!("json: {e}"))
}
/// Convert a malformed stored UUID into a storage error.
pub fn uuid(e: uuid::Error) -> CoreError {
CoreError::Storage(format!("uuid: {e}"))
}

View File

@@ -0,0 +1,52 @@
//! SQLite data-access adapters for newsfeed.
//!
//! [`SqliteStore`] implements every port defined in [`newsfeed_core::ports`]; the
//! binaries hold it behind `Arc<dyn Store>`. Ids are TEXT UUIDs, timestamps RFC3339,
//! and small collections JSON — see [`rows`] for the mapping layer.
//!
//! Note: this crate uses runtime `sqlx` queries rather than the compile-time `query!`
//! macros. On SQLite (dynamically typed) the macros add little safety while forcing a
//! live database or a fiddly offline cache into CI; runtime queries keep CI DB-free.
//! This is a deliberate, documented deviation from architecture `generic.md` §5, which
//! is written for Postgres.
mod err;
mod rows;
mod store;
use std::path::Path;
use std::str::FromStr;
use sqlx::sqlite::{
SqliteConnectOptions, SqliteJournalMode, SqlitePool, SqlitePoolOptions, SqliteSynchronous,
};
pub use store::SqliteStore;
/// Embedded migrations, run at startup by [`connect`].
pub static MIGRATOR: sqlx::migrate::Migrator = sqlx::migrate!("./migrations");
/// Open (creating if absent) the SQLite database at `path`, apply migrations, and return
/// a ready [`SqliteStore`]. WAL + `NORMAL` synchronous + `foreign_keys` are enabled;
/// these suit a single-host api+worker sharing one file (the SQLite deployment shape).
pub async fn connect(path: impl AsRef<Path>, max_connections: u32) -> anyhow::Result<SqliteStore> {
let pool = pool(path, max_connections).await?;
MIGRATOR.run(&pool).await?;
Ok(SqliteStore::new(pool))
}
/// Build a connection pool with the newsfeed pragmas applied to every connection.
pub async fn pool(path: impl AsRef<Path>, max_connections: u32) -> anyhow::Result<SqlitePool> {
let url = format!("sqlite://{}", path.as_ref().display());
let opts = SqliteConnectOptions::from_str(&url)?
.create_if_missing(true)
.journal_mode(SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal)
.foreign_keys(true)
.busy_timeout(std::time::Duration::from_secs(5));
let pool = SqlitePoolOptions::new()
.max_connections(max_connections)
.connect_with(opts)
.await?;
Ok(pool)
}

View File

@@ -0,0 +1,197 @@
//! `FromRow` structs mirroring the SQLite tables, plus conversions into the entity
//! types. DB-native types (TEXT ids, RFC3339 timestamps, JSON blobs) are decoded here so
//! the rest of the crate deals only in [`newsfeed_entities`] values.
use chrono::{DateTime, Utc};
use sqlx::FromRow;
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::StoredCredential;
use newsfeed_entities::auth::{ApiTokenInfo, Session};
use newsfeed_entities::interest::Interest;
use newsfeed_entities::item::{ContentItem, ItemState, Media};
use newsfeed_entities::source::{Source, SourceKind};
use newsfeed_entities::user::User;
use crate::err;
fn parse_id(s: &str) -> CoreResult<Uuid> {
Uuid::parse_str(s).map_err(err::uuid)
}
fn parse_state(s: &str) -> ItemState {
match s {
"feed" => ItemState::Feed,
"saved" => ItemState::Saved,
"dismissed" => ItemState::Dismissed,
_ => ItemState::Candidate,
}
}
#[derive(FromRow)]
pub struct UserRow {
pub id: String,
pub username: String,
pub email: String,
pub password_hash: String,
pub created_at: DateTime<Utc>,
}
impl UserRow {
pub fn into_user(self) -> CoreResult<User> {
Ok(User {
id: parse_id(&self.id)?,
username: self.username,
email: self.email,
created_at: self.created_at,
})
}
pub fn into_credential(self) -> CoreResult<StoredCredential> {
let password_hash = self.password_hash.clone();
Ok(StoredCredential {
user: self.into_user()?,
password_hash,
})
}
}
#[derive(FromRow)]
pub struct SessionRow {
pub id: String,
pub user_id: String,
pub created_at: DateTime<Utc>,
pub expires_at: DateTime<Utc>,
}
impl SessionRow {
pub fn into_session(self) -> CoreResult<Session> {
Ok(Session {
id: parse_id(&self.id)?,
user_id: parse_id(&self.user_id)?,
created_at: self.created_at,
expires_at: self.expires_at,
})
}
}
#[derive(FromRow)]
pub struct ApiTokenRow {
pub id: String,
pub user_id: String,
pub name: String,
pub prefix: String,
pub created_at: DateTime<Utc>,
pub last_used_at: Option<DateTime<Utc>>,
pub revoked_at: Option<DateTime<Utc>>,
}
impl ApiTokenRow {
pub fn into_info(self) -> CoreResult<ApiTokenInfo> {
Ok(ApiTokenInfo {
id: parse_id(&self.id)?,
user_id: parse_id(&self.user_id)?,
name: self.name,
prefix: self.prefix,
created_at: self.created_at,
last_used_at: self.last_used_at,
revoked_at: self.revoked_at,
})
}
}
#[derive(FromRow)]
pub struct SourceRow {
pub id: String,
pub user_id: String,
pub kind: String,
pub name: String,
pub url: Option<String>,
pub weight: f64,
pub enabled: bool,
pub last_polled_at: Option<DateTime<Utc>>,
pub created_at: DateTime<Utc>,
}
impl SourceRow {
pub fn into_source(self) -> CoreResult<Source> {
Ok(Source {
id: parse_id(&self.id)?,
user_id: parse_id(&self.user_id)?,
kind: SourceKind::parse(&self.kind).map_err(newsfeed_core::error::CoreError::from)?,
name: self.name,
url: self.url,
weight: self.weight,
enabled: self.enabled,
last_polled_at: self.last_polled_at,
created_at: self.created_at,
})
}
}
#[derive(FromRow)]
pub struct InterestRow {
pub id: String,
pub user_id: String,
pub label: String,
pub weight: f64,
pub created_at: DateTime<Utc>,
}
impl InterestRow {
pub fn into_interest(self) -> CoreResult<Interest> {
Ok(Interest {
id: parse_id(&self.id)?,
user_id: parse_id(&self.user_id)?,
label: self.label,
weight: self.weight,
created_at: self.created_at,
})
}
}
#[derive(FromRow)]
pub struct ItemRow {
pub id: String,
pub user_id: String,
pub source_id: Option<String>,
pub external_id: String,
pub url: Option<String>,
pub title: String,
pub summary: Option<String>,
pub author: Option<String>,
pub tags: String,
pub media: String,
pub published_at: Option<DateTime<Utc>>,
pub ingested_at: DateTime<Utc>,
pub state: String,
pub score: f64,
}
impl ItemRow {
pub fn into_item(self) -> CoreResult<ContentItem> {
let source_id = match self.source_id {
Some(s) => Some(parse_id(&s)?),
None => None,
};
let tags: Vec<String> = serde_json::from_str(&self.tags).map_err(err::json)?;
let media: Vec<Media> = serde_json::from_str(&self.media).map_err(err::json)?;
Ok(ContentItem {
id: parse_id(&self.id)?,
user_id: parse_id(&self.user_id)?,
source_id,
external_id: self.external_id,
url: self.url,
title: self.title,
summary: self.summary,
author: self.author,
tags,
media,
published_at: self.published_at,
ingested_at: self.ingested_at,
state: parse_state(&self.state),
score: self.score,
})
}
}

View File

@@ -0,0 +1,63 @@
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::InterestStore;
use newsfeed_entities::interest::{Interest, UpsertInterest};
use crate::err;
use crate::rows::InterestRow;
use super::SqliteStore;
const SELECT: &str = "SELECT id, user_id, label, weight, created_at FROM interests";
#[async_trait]
impl InterestStore for SqliteStore {
async fn list_interests(&self, user_id: Uuid) -> CoreResult<Vec<Interest>> {
let rows: Vec<InterestRow> = sqlx::query_as(&format!(
"{SELECT} WHERE user_id = ? ORDER BY weight DESC, label ASC"
))
.bind(user_id.to_string())
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
rows.into_iter().map(InterestRow::into_interest).collect()
}
async fn upsert_interest(
&self,
user_id: Uuid,
upsert: &UpsertInterest,
) -> CoreResult<Interest> {
let id = Uuid::new_v4();
let now = Utc::now();
// On (user_id, label) conflict, update the weight in place and return the row.
let row: InterestRow = sqlx::query_as(&format!(
"INSERT INTO interests (id, user_id, label, weight, created_at) VALUES (?, ?, ?, ?, ?) \
ON CONFLICT(user_id, label) DO UPDATE SET weight = excluded.weight \
RETURNING {}",
"id, user_id, label, weight, created_at"
))
.bind(id.to_string())
.bind(user_id.to_string())
.bind(upsert.label.trim())
.bind(upsert.weight)
.bind(now)
.fetch_one(&self.pool)
.await
.map_err(err::map)?;
row.into_interest()
}
async fn delete_interest(&self, user_id: Uuid, id: Uuid) -> CoreResult<()> {
sqlx::query("DELETE FROM interests WHERE id = ? AND user_id = ?")
.bind(id.to_string())
.bind(user_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
}

View File

@@ -0,0 +1,206 @@
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use newsfeed_core::error::{CoreError, CoreResult};
use newsfeed_core::ports::{ItemStore, NewCandidate};
use newsfeed_entities::item::{ContentItem, ItemState};
use crate::err;
use crate::rows::ItemRow;
use super::SqliteStore;
const COLS: &str = "id, user_id, source_id, external_id, url, title, summary, author, tags, media, published_at, ingested_at, state, score";
#[async_trait]
impl ItemStore for SqliteStore {
async fn upsert_candidate(&self, candidate: &NewCandidate) -> CoreResult<(ContentItem, bool)> {
let id = Uuid::new_v4();
let now = Utc::now();
let tags = serde_json::to_string(&candidate.tags).map_err(err::json)?;
let media = serde_json::to_string(&candidate.media).map_err(err::json)?;
// Insert as a candidate; on (user_id, external_id) conflict do nothing. RETURNING
// yields a row only when a new row was actually inserted.
let inserted: Option<ItemRow> = sqlx::query_as(&format!(
"INSERT INTO items ({COLS}) \
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'candidate', 0.0) \
ON CONFLICT(user_id, external_id) DO NOTHING \
RETURNING {COLS}"
))
.bind(id.to_string())
.bind(candidate.user_id.to_string())
.bind(candidate.source_id.map(|s| s.to_string()))
.bind(&candidate.external_id)
.bind(&candidate.url)
.bind(&candidate.title)
.bind(&candidate.summary)
.bind(&candidate.author)
.bind(tags)
.bind(media)
.bind(candidate.published_at)
.bind(now)
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
if let Some(row) = inserted {
return Ok((row.into_item()?, true));
}
// Conflict: return the pre-existing item.
let existing: ItemRow = sqlx::query_as(&format!(
"SELECT {COLS} FROM items WHERE user_id = ? AND external_id = ?"
))
.bind(candidate.user_id.to_string())
.bind(&candidate.external_id)
.fetch_one(&self.pool)
.await
.map_err(err::map)?;
Ok((existing.into_item()?, false))
}
async fn get_item(&self, user_id: Uuid, id: Uuid) -> CoreResult<Option<ContentItem>> {
let row: Option<ItemRow> = sqlx::query_as(&format!(
"SELECT {COLS} FROM items WHERE id = ? AND user_id = ?"
))
.bind(id.to_string())
.bind(user_id.to_string())
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(ItemRow::into_item).transpose()
}
async fn set_item_state(&self, user_id: Uuid, id: Uuid, state: ItemState) -> CoreResult<()> {
let res = sqlx::query("UPDATE items SET state = ? WHERE id = ? AND user_id = ?")
.bind(state.as_str())
.bind(id.to_string())
.bind(user_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
if res.rows_affected() == 0 {
return Err(CoreError::NotFound);
}
Ok(())
}
async fn update_score(&self, item_id: Uuid, score: f64) -> CoreResult<()> {
sqlx::query("UPDATE items SET score = ? WHERE id = ?")
.bind(score)
.bind(item_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
async fn feed_page(
&self,
user_id: Uuid,
limit: i64,
cursor: Option<&str>,
include_saved: bool,
) -> CoreResult<Vec<ContentItem>> {
let states: &[&str] = if include_saved {
&["feed", "saved"]
} else {
&["feed"]
};
let placeholders = states.iter().map(|_| "?").collect::<Vec<_>>().join(", ");
// Keyset pagination over (score DESC, id DESC).
let (cursor_clause, cursor_score, cursor_id) = match cursor {
Some(c) => {
let (score, id) = parse_cursor(c)?;
(
" AND (score < ? OR (score = ? AND id < ?))",
Some(score),
Some(id),
)
}
None => ("", None, None),
};
let sql = format!(
"SELECT {COLS} FROM items WHERE user_id = ? AND state IN ({placeholders}){cursor_clause} \
ORDER BY score DESC, id DESC LIMIT ?"
);
let mut q = sqlx::query_as::<_, ItemRow>(&sql).bind(user_id.to_string());
for s in states {
q = q.bind(*s);
}
if let (Some(score), Some(id)) = (cursor_score, cursor_id) {
q = q.bind(score).bind(score).bind(id);
}
q = q.bind(limit);
let rows = q.fetch_all(&self.pool).await.map_err(err::map)?;
rows.into_iter().map(ItemRow::into_item).collect()
}
async fn items_for_scoring(&self, user_id: Uuid, limit: i64) -> CoreResult<Vec<ContentItem>> {
let rows: Vec<ItemRow> = sqlx::query_as(&format!(
"SELECT {COLS} FROM items WHERE user_id = ? AND state IN ('candidate', 'feed') \
ORDER BY ingested_at DESC LIMIT ?"
))
.bind(user_id.to_string())
.bind(limit)
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
rows.into_iter().map(ItemRow::into_item).collect()
}
async fn user_ids_with_sources(&self) -> CoreResult<Vec<Uuid>> {
let ids: Vec<(String,)> = sqlx::query_as("SELECT DISTINCT user_id FROM sources")
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
ids.into_iter()
.map(|(s,)| Uuid::parse_str(&s).map_err(err::uuid))
.collect()
}
async fn record_signal(
&self,
user_id: Uuid,
item_id: Uuid,
action: newsfeed_entities::feed::SignalAction,
) -> CoreResult<()> {
let action = match action {
newsfeed_entities::feed::SignalAction::View => "view",
newsfeed_entities::feed::SignalAction::Click => "click",
newsfeed_entities::feed::SignalAction::Save => "save",
newsfeed_entities::feed::SignalAction::Dismiss => "dismiss",
};
sqlx::query(
"INSERT INTO signals (id, user_id, item_id, action, created_at) VALUES (?, ?, ?, ?, ?)",
)
.bind(Uuid::new_v4().to_string())
.bind(user_id.to_string())
.bind(item_id.to_string())
.bind(action)
.bind(Utc::now())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
}
/// Cursor is `"<score>:<uuid>"`; the id disambiguates equal scores.
fn parse_cursor(c: &str) -> CoreResult<(f64, String)> {
let (score, id) = c
.split_once(':')
.ok_or_else(|| CoreError::Storage("malformed feed cursor".into()))?;
let score: f64 = score
.parse()
.map_err(|_| CoreError::Storage("malformed feed cursor score".into()))?;
// Validate the id is a uuid but keep the string form for binding.
Uuid::parse_str(id).map_err(err::uuid)?;
Ok((score, id.to_string()))
}

View File

@@ -0,0 +1,33 @@
//! The concrete [`SqliteStore`] and its per-domain port implementations.
mod interests;
mod items;
mod sessions;
mod sources;
mod tokens;
mod users;
use sqlx::sqlite::SqlitePool;
/// SQLite-backed adapter implementing all of the `newsfeed-core` ports.
#[derive(Clone)]
pub struct SqliteStore {
pub(crate) pool: SqlitePool,
}
impl SqliteStore {
/// Wrap an existing pool. Prefer [`crate::connect`] which also runs migrations.
pub fn new(pool: SqlitePool) -> Self {
Self { pool }
}
/// Access the underlying pool (tests, health checks).
pub fn pool(&self) -> &SqlitePool {
&self.pool
}
/// Cheap readiness check: `true` when the database answers a trivial query.
pub async fn ping(&self) -> bool {
sqlx::query("SELECT 1").execute(&self.pool).await.is_ok()
}
}

View File

@@ -0,0 +1,65 @@
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::SessionStore;
use newsfeed_entities::auth::Session;
use crate::err;
use crate::rows::SessionRow;
use super::SqliteStore;
#[async_trait]
impl SessionStore for SqliteStore {
async fn create_session(
&self,
user_id: Uuid,
token_hash: &str,
expires_at: DateTime<Utc>,
) -> CoreResult<Session> {
let id = Uuid::new_v4();
let now = Utc::now();
sqlx::query(
"INSERT INTO sessions (id, user_id, token_hash, created_at, expires_at) VALUES (?, ?, ?, ?, ?)",
)
.bind(id.to_string())
.bind(user_id.to_string())
.bind(token_hash)
.bind(now)
.bind(expires_at)
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(Session {
id,
user_id,
created_at: now,
expires_at,
})
}
async fn find_session(&self, token_hash: &str) -> CoreResult<Option<Session>> {
let row: Option<SessionRow> = sqlx::query_as(
"SELECT id, user_id, created_at, expires_at FROM sessions \
WHERE token_hash = ? AND expires_at > ?",
)
.bind(token_hash)
.bind(Utc::now())
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(SessionRow::into_session).transpose()
}
async fn delete_session(&self, token_hash: &str) -> CoreResult<()> {
sqlx::query("DELETE FROM sessions WHERE token_hash = ?")
.bind(token_hash)
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
}

View File

@@ -0,0 +1,113 @@
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::SourceStore;
use newsfeed_entities::source::{NewSource, Source};
use crate::err;
use crate::rows::SourceRow;
use super::SqliteStore;
const SELECT: &str =
"SELECT id, user_id, kind, name, url, weight, enabled, last_polled_at, created_at FROM sources";
#[async_trait]
impl SourceStore for SqliteStore {
async fn create_source(&self, user_id: Uuid, new: &NewSource) -> CoreResult<Source> {
let id = Uuid::new_v4();
let now = Utc::now();
let weight = new.weight.unwrap_or(1.0);
sqlx::query(
"INSERT INTO sources (id, user_id, kind, name, url, weight, enabled, created_at) \
VALUES (?, ?, ?, ?, ?, ?, 1, ?)",
)
.bind(id.to_string())
.bind(user_id.to_string())
.bind(new.kind.as_str())
.bind(&new.name)
.bind(&new.url)
.bind(weight)
.bind(now)
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(Source {
id,
user_id,
kind: new.kind,
name: new.name.clone(),
url: new.url.clone(),
weight,
enabled: true,
last_polled_at: None,
created_at: now,
})
}
async fn list_sources(&self, user_id: Uuid) -> CoreResult<Vec<Source>> {
let rows: Vec<SourceRow> = sqlx::query_as(&format!(
"{SELECT} WHERE user_id = ? ORDER BY created_at DESC"
))
.bind(user_id.to_string())
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
rows.into_iter().map(SourceRow::into_source).collect()
}
async fn find_source_by_name(&self, user_id: Uuid, name: &str) -> CoreResult<Option<Source>> {
let row: Option<SourceRow> =
sqlx::query_as(&format!("{SELECT} WHERE user_id = ? AND name = ?"))
.bind(user_id.to_string())
.bind(name)
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(SourceRow::into_source).transpose()
}
async fn delete_source(&self, user_id: Uuid, id: Uuid) -> CoreResult<()> {
sqlx::query("DELETE FROM sources WHERE id = ? AND user_id = ?")
.bind(id.to_string())
.bind(user_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
async fn due_rss_sources(
&self,
now: DateTime<Utc>,
min_interval_secs: i64,
limit: i64,
) -> CoreResult<Vec<Source>> {
// Due when never polled, or last poll older than the minimum interval.
let cutoff = now - chrono::Duration::seconds(min_interval_secs);
let rows: Vec<SourceRow> = sqlx::query_as(&format!(
"{SELECT} WHERE kind = 'rss' AND enabled = 1 \
AND (last_polled_at IS NULL OR last_polled_at < ?) \
ORDER BY last_polled_at ASC NULLS FIRST LIMIT ?"
))
.bind(cutoff)
.bind(limit)
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
rows.into_iter().map(SourceRow::into_source).collect()
}
async fn mark_polled(&self, source_id: Uuid, at: DateTime<Utc>) -> CoreResult<()> {
sqlx::query("UPDATE sources SET last_polled_at = ? WHERE id = ?")
.bind(at)
.bind(source_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
}

View File

@@ -0,0 +1,90 @@
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::TokenStore;
use newsfeed_entities::auth::ApiTokenInfo;
use crate::err;
use crate::rows::ApiTokenRow;
use super::SqliteStore;
const SELECT: &str =
"SELECT id, user_id, name, prefix, created_at, last_used_at, revoked_at FROM api_tokens";
#[async_trait]
impl TokenStore for SqliteStore {
async fn create_token(
&self,
user_id: Uuid,
name: &str,
prefix: &str,
token_hash: &str,
) -> CoreResult<ApiTokenInfo> {
let id = Uuid::new_v4();
let now = Utc::now();
sqlx::query(
"INSERT INTO api_tokens (id, user_id, name, prefix, token_hash, created_at) VALUES (?, ?, ?, ?, ?, ?)",
)
.bind(id.to_string())
.bind(user_id.to_string())
.bind(name)
.bind(prefix)
.bind(token_hash)
.bind(now)
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(ApiTokenInfo {
id,
user_id,
name: name.to_string(),
prefix: prefix.to_string(),
created_at: now,
last_used_at: None,
revoked_at: None,
})
}
async fn list_tokens(&self, user_id: Uuid) -> CoreResult<Vec<ApiTokenInfo>> {
let rows: Vec<ApiTokenRow> = sqlx::query_as(&format!(
"{SELECT} WHERE user_id = ? ORDER BY created_at DESC"
))
.bind(user_id.to_string())
.fetch_all(&self.pool)
.await
.map_err(err::map)?;
rows.into_iter().map(ApiTokenRow::into_info).collect()
}
async fn authenticate_token(&self, token_hash: &str) -> CoreResult<Option<ApiTokenInfo>> {
let now = Utc::now();
// Record use and fetch in one round trip via RETURNING (SQLite ≥ 3.35).
let row: Option<ApiTokenRow> = sqlx::query_as(&format!(
"UPDATE api_tokens SET last_used_at = ? \
WHERE token_hash = ? AND revoked_at IS NULL \
RETURNING {}",
"id, user_id, name, prefix, created_at, last_used_at, revoked_at"
))
.bind(now)
.bind(token_hash)
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(ApiTokenRow::into_info).transpose()
}
async fn revoke_token(&self, user_id: Uuid, id: Uuid) -> CoreResult<()> {
sqlx::query("UPDATE api_tokens SET revoked_at = ? WHERE id = ? AND user_id = ? AND revoked_at IS NULL")
.bind(Utc::now())
.bind(id.to_string())
.bind(user_id.to_string())
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(())
}
}

View File

@@ -0,0 +1,67 @@
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use newsfeed_core::error::CoreResult;
use newsfeed_core::ports::{StoredCredential, UserStore};
use newsfeed_entities::user::User;
use crate::err;
use crate::rows::UserRow;
use super::SqliteStore;
#[async_trait]
impl UserStore for SqliteStore {
async fn create_user(
&self,
username: &str,
email: &str,
password_hash: &str,
) -> CoreResult<User> {
let id = Uuid::new_v4();
let now = Utc::now();
sqlx::query(
"INSERT INTO users (id, username, email, password_hash, created_at) VALUES (?, ?, ?, ?, ?)",
)
.bind(id.to_string())
.bind(username)
.bind(email)
.bind(password_hash)
.bind(now)
.execute(&self.pool)
.await
.map_err(err::map)?;
Ok(User {
id,
username: username.to_string(),
email: email.to_string(),
created_at: now,
})
}
async fn get_user(&self, id: Uuid) -> CoreResult<Option<User>> {
let row: Option<UserRow> = sqlx::query_as(
"SELECT id, username, email, password_hash, created_at FROM users WHERE id = ?",
)
.bind(id.to_string())
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(UserRow::into_user).transpose()
}
async fn find_credential(&self, identifier: &str) -> CoreResult<Option<StoredCredential>> {
let row: Option<UserRow> = sqlx::query_as(
"SELECT id, username, email, password_hash, created_at FROM users \
WHERE username = ? OR email = ?",
)
.bind(identifier)
.bind(identifier)
.fetch_optional(&self.pool)
.await
.map_err(err::map)?;
row.map(UserRow::into_credential).transpose()
}
}

View File

@@ -0,0 +1,16 @@
[package]
name = "newsfeed-entities"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
description = "Domain types, DTOs and error enums for newsfeed. No I/O."
[dependencies]
serde.workspace = true
serde_json.workspace = true
uuid.workspace = true
chrono.workspace = true
thiserror.workspace = true
ts-rs.workspace = true

View File

@@ -0,0 +1,61 @@
//! Session and API-token DTOs.
//!
//! Humans authenticate with an opaque session cookie; agentic and algorithmic sources
//! authenticate with per-user bearer API tokens. In both cases only a hash of the
//! secret is ever persisted — see `newsfeed-core::auth`.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
/// An authenticated browser session, keyed by the hash of an opaque cookie value.
#[derive(Debug, Clone)]
pub struct Session {
pub id: Uuid,
pub user_id: Uuid,
pub created_at: DateTime<Utc>,
pub expires_at: DateTime<Utc>,
}
/// Public metadata about an API token. The secret itself is shown exactly once, at
/// creation, and is never retrievable afterwards.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct ApiTokenInfo {
pub id: Uuid,
pub user_id: Uuid,
pub name: String,
/// Short non-secret prefix (e.g. `nf_a1b2c3`) to help the user identify the token.
pub prefix: String,
pub created_at: DateTime<Utc>,
pub last_used_at: Option<DateTime<Utc>>,
pub revoked_at: Option<DateTime<Utc>>,
}
/// Request to mint a new API token.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct CreateApiToken {
pub name: String,
}
/// Response returned once when a token is created. `secret` is the full bearer token
/// (`nf_<prefix>_<random>`) and is never persisted in the clear.
#[derive(Debug, Clone, Serialize, TS)]
#[ts(export)]
pub struct CreatedApiToken {
#[serde(flatten)]
pub info: ApiTokenInfo,
/// The full secret. Present only in this response.
pub secret: String,
}
/// The current authenticated principal, returned by `GET /v1/auth/me`.
#[derive(Debug, Clone, Serialize, TS)]
#[ts(export)]
pub struct Me {
pub id: Uuid,
pub username: String,
pub email: String,
}

View File

@@ -0,0 +1,30 @@
//! Domain-level error type. Validation and invariant failures live here; I/O errors
//! belong to the crates that perform the I/O (`newsfeed-data`, the binaries).
use thiserror::Error;
/// Errors produced while constructing or validating domain values.
#[derive(Debug, Error)]
pub enum Error {
/// A user-supplied value failed validation (empty, malformed, out of range).
#[error("invalid {field}: {reason}")]
Invalid {
/// The field that failed validation.
field: &'static str,
/// Why it failed.
reason: String,
},
}
impl Error {
/// Convenience constructor for [`Error::Invalid`].
pub fn invalid(field: &'static str, reason: impl Into<String>) -> Self {
Self::Invalid {
field,
reason: reason.into(),
}
}
}
/// Result alias for domain operations.
pub type Result<T> = std::result::Result<T, Error>;

View File

@@ -0,0 +1,54 @@
//! Feed query/response DTOs and interaction signals. Signals are the feedback loop:
//! what the user does with an item feeds back into future ranking — but only ever
//! according to weights the user themselves controls.
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
use crate::item::ContentItem;
/// Query parameters for `GET /v1/feed`.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct FeedQuery {
/// Max items to return. Defaults applied server-side.
pub limit: Option<u32>,
/// Opaque cursor for pagination (score+id of the last item seen).
pub cursor: Option<String>,
/// When true, include saved items; otherwise only live feed items.
#[serde(default)]
pub include_saved: bool,
}
/// A page of ranked feed items.
#[derive(Debug, Clone, Serialize, TS)]
#[ts(export)]
pub struct FeedPage {
pub items: Vec<ContentItem>,
/// Cursor to pass back for the next page, or `None` at the end.
pub next_cursor: Option<String>,
}
/// The kinds of interaction the user can have with an item.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, TS)]
#[serde(rename_all = "snake_case")]
#[ts(export)]
pub enum SignalAction {
/// The item scrolled into view.
View,
/// The user opened/clicked through.
Click,
/// The user saved it.
Save,
/// The user dismissed it.
Dismiss,
}
/// A recorded interaction, posted to `POST /v1/feed/signals`.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct Signal {
pub item_id: Uuid,
pub action: SignalAction,
}

View File

@@ -0,0 +1,45 @@
//! Interests: the user-controlled weightings that drive ranking. A user tells the system
//! what they care about and how much; the ranker (see `newsfeed-core::ranking`) uses
//! these weights — and nothing else's opinion — to score their feed.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
use crate::error::{Error, Result};
/// A weighted topic of interest belonging to a single user.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct Interest {
pub id: Uuid,
pub user_id: Uuid,
/// Free-text label matched (case-insensitively) against item title/summary/tags.
pub label: String,
/// How strongly this interest pulls matching items up the feed, `-1.0..=1.0`.
/// Negative weights actively bury matching content.
pub weight: f64,
pub created_at: DateTime<Utc>,
}
/// Create or update an interest.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct UpsertInterest {
pub label: String,
pub weight: f64,
}
impl UpsertInterest {
/// Validate an interest upsert.
pub fn validate(&self) -> Result<()> {
if self.label.trim().is_empty() {
return Err(Error::invalid("label", "must not be empty"));
}
if !(-1.0..=1.0).contains(&self.weight) {
return Err(Error::invalid("weight", "must be within -1.0..=1.0"));
}
Ok(())
}
}

View File

@@ -0,0 +1,89 @@
//! Content items: the units that flow through curation into a user's feed. An item
//! enters as a *candidate* (either polled by the worker or pushed to the ingest
//! endpoint), gets scored, and then lives in the feed until the user acts on it.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
/// Lifecycle state of a content item.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, TS)]
#[serde(rename_all = "snake_case")]
#[ts(export)]
pub enum ItemState {
/// Newly ingested, awaiting scoring/curation.
Candidate,
/// Scored and live in the feed.
Feed,
/// The user explicitly saved/bookmarked it.
Saved,
/// The user dismissed it; kept for negative signal, hidden from the feed.
Dismissed,
}
impl ItemState {
/// Stable string form used in the database and on the wire.
pub fn as_str(self) -> &'static str {
match self {
ItemState::Candidate => "candidate",
ItemState::Feed => "feed",
ItemState::Saved => "saved",
ItemState::Dismissed => "dismissed",
}
}
}
/// A media attachment (image/video/audio) associated with an item.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct Media {
pub kind: String,
pub url: String,
pub width: Option<u32>,
pub height: Option<u32>,
}
/// A content item owned by a user.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct ContentItem {
pub id: Uuid,
pub user_id: Uuid,
pub source_id: Option<Uuid>,
/// De-duplication key from the origin (feed guid, tweet id, URL, …).
pub external_id: String,
pub url: Option<String>,
pub title: String,
pub summary: Option<String>,
pub author: Option<String>,
pub tags: Vec<String>,
pub media: Vec<Media>,
pub published_at: Option<DateTime<Utc>>,
pub ingested_at: DateTime<Utc>,
pub state: ItemState,
/// Cached ranking score; recomputed by the ranker as weights/signals change.
pub score: f64,
}
/// A candidate pushed to `POST /v1/ingest/candidates` by an agentic or algorithmic
/// source. The authenticated API token determines the owning user; `source` names the
/// producer for attribution and per-source weighting.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct CandidateSubmission {
/// Stable de-dup key from the producer. Re-submitting the same key is a no-op.
pub external_id: String,
pub url: Option<String>,
pub title: String,
pub summary: Option<String>,
pub author: Option<String>,
#[serde(default)]
pub tags: Vec<String>,
#[serde(default)]
pub media: Vec<Media>,
pub published_at: Option<DateTime<Utc>>,
/// Optional name of the source that produced this candidate. If it matches an
/// existing agentic source for the user it is linked; otherwise ingested unlinked.
pub source: Option<String>,
}

View File

@@ -0,0 +1,22 @@
//! Domain types, DTOs and error enums shared across the newsfeed workspace.
//!
//! This crate is intentionally free of I/O and async-runtime dependencies. Everything
//! downstream (`newsfeed-core`, `newsfeed-data`, the binaries and — via generated
//! TypeScript bindings — the web frontend) depends on these types.
//!
//! Wire types derive [`ts_rs::TS`] with `#[ts(export)]` so `cargo test -p
//! newsfeed-entities` regenerates the frontend bindings under `web/src/api/bindings/`
//! (path set by `TS_RS_EXPORT_DIR` in `.cargo/config.toml`).
pub mod auth;
pub mod error;
pub mod feed;
pub mod interest;
pub mod item;
pub mod source;
pub mod user;
pub use error::{Error, Result};
/// Newtype over a [`uuid::Uuid`] identifier, re-exported for convenience.
pub type Id = uuid::Uuid;

View File

@@ -0,0 +1,92 @@
//! Content sources. A source is where candidate items come from — an RSS/Atom feed the
//! worker polls, or an agentic/external producer that pushes candidates to the ingest
//! endpoint under a user's API token.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
use crate::error::{Error, Result};
/// How a source produces candidates.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, TS)]
#[serde(rename_all = "snake_case")]
#[ts(export)]
pub enum SourceKind {
/// An RSS/Atom feed the worker polls on a schedule.
Rss,
/// A producer that pushes candidates to `POST /v1/ingest/candidates`
/// (agentic workloads, scrapers, bespoke algorithms).
Agentic,
}
impl SourceKind {
/// Stable string form used in the database and on the wire.
pub fn as_str(self) -> &'static str {
match self {
SourceKind::Rss => "rss",
SourceKind::Agentic => "agentic",
}
}
/// Parse from the database/wire representation.
pub fn parse(s: &str) -> Result<Self> {
match s {
"rss" => Ok(SourceKind::Rss),
"agentic" => Ok(SourceKind::Agentic),
other => Err(Error::invalid(
"kind",
format!("unknown source kind: {other}"),
)),
}
}
}
/// A user's configured content source.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct Source {
pub id: Uuid,
pub user_id: Uuid,
pub kind: SourceKind,
pub name: String,
/// Poll URL for [`SourceKind::Rss`]; unused/optional for agentic sources.
pub url: Option<String>,
/// Baseline weight applied to every item from this source, `0.0..=1.0`.
pub weight: f64,
pub enabled: bool,
/// When the worker last polled this source (rss only).
pub last_polled_at: Option<DateTime<Utc>>,
pub created_at: DateTime<Utc>,
}
/// Request to create a source.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct NewSource {
pub kind: SourceKind,
pub name: String,
pub url: Option<String>,
/// Defaults to `1.0` when omitted.
#[serde(default)]
pub weight: Option<f64>,
}
impl NewSource {
/// Validate a create-source request.
pub fn validate(&self) -> Result<()> {
if self.name.trim().is_empty() {
return Err(Error::invalid("name", "must not be empty"));
}
if self.kind == SourceKind::Rss && self.url.as_deref().unwrap_or("").is_empty() {
return Err(Error::invalid("url", "rss sources require a url"));
}
if let Some(w) = self.weight {
if !(0.0..=1.0).contains(&w) {
return Err(Error::invalid("weight", "must be within 0.0..=1.0"));
}
}
Ok(())
}
}

View File

@@ -0,0 +1,54 @@
//! User accounts. Each user is fully in control of their own feed weightings.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use ts_rs::TS;
use uuid::Uuid;
use crate::error::{Error, Result};
/// A registered user. The password hash never leaves `newsfeed-data`/`newsfeed-core`;
/// it is deliberately absent from this DTO so it can't be serialised to a client.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub struct User {
pub id: Uuid,
pub username: String,
pub email: String,
pub created_at: DateTime<Utc>,
}
/// Registration request from the sign-up form.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct RegisterRequest {
pub username: String,
pub email: String,
pub password: String,
}
impl RegisterRequest {
/// Validate the shape of a registration request before it reaches the store.
/// Uniqueness is enforced by the database, not here.
pub fn validate(&self) -> Result<()> {
if self.username.trim().len() < 3 {
return Err(Error::invalid("username", "must be at least 3 characters"));
}
if !self.email.contains('@') {
return Err(Error::invalid("email", "must contain '@'"));
}
if self.password.len() < 8 {
return Err(Error::invalid("password", "must be at least 8 characters"));
}
Ok(())
}
}
/// Login request.
#[derive(Debug, Clone, Deserialize, TS)]
#[ts(export)]
pub struct LoginRequest {
/// Username or email.
pub identifier: String,
pub password: String,
}

View File

@@ -0,0 +1,31 @@
[package]
name = "newsfeed-worker"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
description = "Algorithmic content-sourcing worker for newsfeed (RSS/Atom polling)."
[[bin]]
name = "newsfeed-worker"
path = "src/main.rs"
[dependencies]
newsfeed-entities.workspace = true
newsfeed-core.workspace = true
newsfeed-data.workspace = true
tokio.workspace = true
reqwest.workspace = true
feed-rs.workspace = true
serde.workspace = true
chrono.workspace = true
uuid.workspace = true
anyhow.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
figment.workspace = true
clap.workspace = true

View File

@@ -0,0 +1,55 @@
//! Worker configuration (figment-layered: defaults → TOML → `NEWSFEED_*` env).
use std::path::{Path, PathBuf};
use figment::Figment;
use figment::providers::{Env, Format, Serialized, Toml};
use serde::{Deserialize, Serialize};
/// Worker configuration.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config {
/// SQLite database file, shared with the api on the same host.
pub database_path: PathBuf,
/// SQLite pool size for the worker.
pub max_db_connections: u32,
/// Seconds to sleep between poll cycles.
pub tick_secs: u64,
/// Minimum seconds between polls of the same RSS source.
pub source_min_interval_secs: i64,
/// Max sources polled per cycle.
pub batch: i64,
/// Max items rescored per user per cycle.
pub rescore_limit: i64,
/// HTTP request timeout when fetching feeds, in seconds.
pub http_timeout_secs: u64,
/// User-Agent sent when fetching feeds.
pub user_agent: String,
}
impl Default for Config {
fn default() -> Self {
Self {
database_path: PathBuf::from("/var/lib/newsfeed/newsfeed.db"),
max_db_connections: 2,
tick_secs: 60,
source_min_interval_secs: 900,
batch: 20,
rescore_limit: 500,
http_timeout_secs: 20,
user_agent: concat!("newsfeed-worker/", env!("CARGO_PKG_VERSION")).to_string(),
}
}
}
impl Config {
/// Load configuration, layering an optional TOML file and the environment over the
/// defaults.
pub fn load(file: Option<&Path>) -> anyhow::Result<Self> {
let mut fig = Figment::from(Serialized::defaults(Config::default()));
if let Some(path) = file {
fig = fig.merge(Toml::file(path));
}
Ok(fig.merge(Env::prefixed("NEWSFEED_")).extract()?)
}
}

View File

@@ -0,0 +1,168 @@
//! newsfeed algorithmic-sourcing worker.
//!
//! Co-located with the api on the same host (they share one SQLite file). Each cycle it
//! polls due RSS/Atom sources, normalises and scores new entries into their owners'
//! feeds, then rescores existing items so weight changes re-rank the feed. Idempotent:
//! re-seeing an item's `external_id` is a no-op.
mod config;
mod sourcing;
use std::collections::HashMap;
use std::path::PathBuf;
use std::time::Duration;
use anyhow::Context;
use chrono::Utc;
use clap::Parser;
use reqwest::Client;
use tokio::signal;
use uuid::Uuid;
use newsfeed_core::ports::{InterestStore, ItemStore, SourceStore};
use newsfeed_core::{ingest, service};
use newsfeed_data::SqliteStore;
use newsfeed_entities::interest::Interest;
use crate::config::Config;
use crate::sourcing::rss;
#[derive(Parser)]
#[command(
name = "newsfeed-worker",
version,
about = "newsfeed content-sourcing worker"
)]
struct Cli {
/// Path to a TOML config file. Env (`NEWSFEED_*`) overrides file values.
#[arg(long, default_value = "/etc/newsfeed/worker.toml")]
config: PathBuf,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
init_tracing();
let cli = Cli::parse();
let config_file = cli.config.exists().then_some(cli.config.as_path());
let cfg = Config::load(config_file).context("loading configuration")?;
tracing::info!(db = %cfg.database_path.display(), tick_secs = cfg.tick_secs, "starting newsfeed-worker");
let store = newsfeed_data::connect(&cfg.database_path, cfg.max_db_connections)
.await
.context("opening database")?;
let client = Client::builder()
.user_agent(&cfg.user_agent)
.timeout(Duration::from_secs(cfg.http_timeout_secs))
.build()
.context("building HTTP client")?;
let mut ticker = tokio::time::interval(Duration::from_secs(cfg.tick_secs));
loop {
tokio::select! {
_ = ticker.tick() => {
if let Err(e) = run_cycle(&store, &client, &cfg).await {
tracing::error!(error = %e, "poll cycle failed");
}
}
_ = shutdown_signal() => {
tracing::info!("shutdown signal received, exiting");
break;
}
}
}
Ok(())
}
/// One poll + rescore cycle.
async fn run_cycle(store: &SqliteStore, client: &Client, cfg: &Config) -> anyhow::Result<()> {
let now = Utc::now();
let due = store
.due_rss_sources(now, cfg.source_min_interval_secs, cfg.batch)
.await?;
tracing::debug!(count = due.len(), "polling due sources");
let mut interests_cache: HashMap<Uuid, Vec<Interest>> = HashMap::new();
for source in due {
let Some(url) = source.url.clone() else {
store.mark_polled(source.id, now).await?;
continue;
};
match rss::fetch(client, &url).await {
Ok(entries) => {
let interests = if let Some(cached) = interests_cache.get(&source.user_id) {
cached
} else {
let loaded = store.list_interests(source.user_id).await?;
interests_cache.entry(source.user_id).or_insert(loaded)
};
let mut added = 0usize;
for entry in entries {
let candidate =
ingest::candidate_from_feed_entry(source.user_id, source.id, entry);
match ingest::persist_and_rank(store, candidate, source.weight, interests).await
{
Ok(item) if item.ingested_at >= now => added += 1,
Ok(_) => {}
Err(e) => {
tracing::warn!(source = %source.name, error = %e, "failed to persist candidate")
}
}
}
tracing::info!(source = %source.name, added, "polled source");
}
Err(e) => {
tracing::warn!(source = %source.name, %url, error = %e, "failed to fetch feed")
}
}
// Mark polled regardless so a persistently-failing feed doesn't monopolise cycles.
store.mark_polled(source.id, now).await?;
}
// Rescore each active user's feed against current weights.
for user_id in store.user_ids_with_sources().await? {
match service::rescore_user(store, user_id, cfg.rescore_limit).await {
Ok(n) => tracing::debug!(%user_id, rescored = n, "rescored feed"),
Err(e) => tracing::warn!(%user_id, error = %e, "rescore failed"),
}
}
Ok(())
}
fn init_tracing() {
use tracing_subscriber::{EnvFilter, fmt, prelude::*};
let filter = EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info"));
let registry = tracing_subscriber::registry().with(filter);
if std::env::var_os("JOURNAL_STREAM").is_some() {
registry
.with(fmt::layer().json().flatten_event(true))
.init();
} else {
registry.with(fmt::layer()).init();
}
}
async fn shutdown_signal() {
let ctrl_c = async {
signal::ctrl_c().await.expect("install Ctrl-C handler");
};
#[cfg(unix)]
let terminate = async {
signal::unix::signal(signal::unix::SignalKind::terminate())
.expect("install SIGTERM handler")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {},
_ = terminate => {},
}
}

View File

@@ -0,0 +1,3 @@
//! Content sourcing. Currently RSS/Atom polling; other algorithmic sources plug in here.
pub mod rss;

View File

@@ -0,0 +1,80 @@
//! Fetch and parse an RSS/Atom feed into backend-agnostic [`FeedEntry`] values.
use anyhow::Context;
use reqwest::Client;
use newsfeed_core::ingest::FeedEntry;
use newsfeed_entities::item::Media;
/// Fetch `url` and parse it, returning one [`FeedEntry`] per feed item.
pub async fn fetch(client: &Client, url: &str) -> anyhow::Result<Vec<FeedEntry>> {
let bytes = client
.get(url)
.send()
.await
.with_context(|| format!("fetching {url}"))?
.error_for_status()
.with_context(|| format!("bad status from {url}"))?
.bytes()
.await
.with_context(|| format!("reading body from {url}"))?;
let feed = feed_rs::parser::parse(&bytes[..]).with_context(|| format!("parsing feed {url}"))?;
Ok(feed.entries.into_iter().map(entry_to_feed_entry).collect())
}
fn entry_to_feed_entry(entry: feed_rs::model::Entry) -> FeedEntry {
let url = entry.links.first().map(|l| l.href.clone());
// Prefer the entry's own id; fall back to the link so de-dup still works.
let external_id = if entry.id.is_empty() {
url.clone().unwrap_or_else(|| {
entry
.title
.as_ref()
.map(|t| t.content.clone())
.unwrap_or_default()
})
} else {
entry.id.clone()
};
let title = entry
.title
.map(|t| t.content)
.unwrap_or_else(|| "(untitled)".to_string());
let summary = entry
.summary
.map(|t| t.content)
.or_else(|| entry.content.and_then(|c| c.body));
let author = entry.authors.first().map(|p| p.name.clone());
let tags = entry.categories.into_iter().map(|c| c.term).collect();
let published_at = entry.published.or(entry.updated);
let media = entry
.media
.into_iter()
.flat_map(|m| m.content)
.filter_map(|c| {
c.url.map(|u| Media {
kind: c
.content_type
.map(|m| m.to_string())
.unwrap_or_else(|| "media".to_string()),
url: u.to_string(),
width: c.width,
height: c.height,
})
})
.collect();
FeedEntry {
external_id,
url,
title,
summary,
author,
tags,
media,
published_at,
}
}