Per Codex: the Next poll column computed only the streak-backoff time, so a
rate-limited source could show an earlier Next poll than the real gate (which
also requires retry_after_at <= now). Take the later of the two in the Python
post-process so the admin table agrees with due_source_rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's spec — a publisher saying "slow down" shouldn't make a feed look
broken, but repeated 429s stay visible via last_success_at / stale-source.
* Schema: sources.retry_after_at (nullable) + migration.
* feeds.parse_retry_after: delta-seconds OR HTTP-date → UTC stamp; ignores
invalid/negative/past; caps at now + MAX_BACKOFF_MINUTES.
* fetch_feed raises RateLimited (carrying the parsed time) on a 429.
* poll_source: on 429 set retry_after_at + last_error, status='rate_limited',
and do NOT increment consecutive_failures; on success clear retry_after_at;
non-429 failures unchanged.
* due_source_rows requires BOTH the streak backoff elapsed AND retry_after_at
passed (i.e. the later of the two).
* Admin: source_health returns retry_after_at; status reads
"rate-limited · rests until …" rather than "failed/resting".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: upsert_sources() wrote `active` but not `status`, so a candidate
promoted inactive (the pipeline default) became active=0 + status='active' —
the exact mirror drift Phase 1 set out to avoid (scheduler won't poll, admin UI
shows "active"). Now derive status from an explicit value or from active, mirror
active off status, and write both columns together (insert + conflict update).
Test: promote_candidate(active=False) → status='paused', active=0.
Also fix stale source_health docstring (now includes retired).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's plan — introduce a lifecycle without a risky "change the source of
truth everywhere" moment.
* Schema: sources.status (active|paused|retired) + content_visible; migration
backfills status from active (active=1→active, else paused), content_visible=1.
* `active` is kept as a SYNCED MIRROR: status active→active=1, paused/retired→0,
so the scheduler/CLI/legacy code keep working unchanged.
* Retire stops polling but keeps articles visible (non-destructive). Hiding is a
separate, reversible lever: content_visible=0 drops a source's articles from
the public feed + brief (read AND build), behind a confirm. Personal saved/
history are untouched.
* API: /sources/{id}/status (validates, mirrors active) + /visibility, replacing
/active. source_health returns status + content_visible.
* Admin: status column (active/paused/retired + "hidden"), Retired filter,
Pause/Resume · Retire/Restore · Hide/Show actions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's optional note: alongside the ingest-wide duplicate_rate, expose
accepted_dup_rate — of what a source got ACCEPTED, how much was a duplicate of
already-served content (accepted_total − served). Nearly free (derived from
existing counts); surfaced as a tooltip on the Dup column so the table stays
uncluttered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the Sources tab into a real management console (per Codex):
* source_health now lists ALL sources (active + paused) with backing metrics:
served / accepted_total / total_articles / duplicates + acceptance & duplicate
rates + review_reason, alongside last success/attempt, next poll, failures.
* Admin endpoints (gated, 404 on missing): POST sources/{id}/active (pause/
resume) and /review (flag/clear with reason).
* Pausing only stops future polling — the feed query has no active filter, so a
paused source's accepted articles stay live.
* Frontend: metric table + Paused filter + per-row Pause/Resume & Flag/Clear
(optimistic, revert on failure). Attention 'resting' now scoped to active.
Retire/Delete intentionally deferred (distinct lifecycle state, later).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the admin Feedback section a real inbox.
* DB: feedback.read_at column (schema + idempotent migration).
* API: feedback list returns read_at; POST /api/admin/feedback/{id}/read
{read} toggles it; DELETE /api/admin/feedback/{id} removes a message
(both admin-gated). admin_stats gains feedback_unread; the Attention strip
and the tab badge now count UNREAD, not total.
* Frontend: unread messages are highlighted with an accent rail + dot; an
Unread filter joins the category chips; each message has Mark read/unread
and Delete (confirm), with optimistic updates that revert on failure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex audit:
* Unknown ?section= values now clamp to Overview, so the page never renders the
tabs with an empty body.
* Summary/image coverage counts join through articles+scores and require
accepted=1 AND duplicate_of IS NULL, so percentages stay ≤100% and honest as
rejected/duplicate rows accrue summaries over time.
* A source that's both resting and flagged now shows "⚠ resting · review"
rather than hiding the review flag behind the resting state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Click a source name on any card → a feed of just that source's articles,
newest-first, still accepted / non-duplicate / boundary-filtered (the calm
promise isn't bypassed). A natural way to follow a publication's feel.
* queries.feed + /api/feed: source_id filter; Article output gains source_id.
* Frontend: source label is a button → transient 'source:<id>' view (like
'tag:<slug>'), rendered in the feed grid with Load more, header = source name.
* Ad-hoc, not a pinned lane. Foundation for a future source page (metadata) +
Follow; shareable /source/<slug> route and source_view analytics come then.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restructure the nav around two permanent lanes, then the reader's chosen ones:
"Highlights" (the curated daily brief — formerly "Today") and "Latest" (the
freshest accepted stories, newest-first). Now that the gate is tight, a
chronological "incoming" feed is safe to expose.
* feed(): new sort="latest" (pure recency) alongside the default best-first
rank; /api/feed exposes sort=ranked|latest (validated). Still accepted-only
and boundary-respecting either way.
* lanes.py: two pinned lanes (Highlights + Latest) instead of one.
* Home: "Latest" view + "Load more" pagination for every feed view (offset-
paged, de-duped). Mobile bottom bar gains a Latest tab.
* LanePicker shows both pinned lanes; nav rail renders them first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Date fix: introduce GOODNEWS_TZ (goodnews/localtime.py) so the brief's "today"
rolls over in a pinned zone (Eastern) instead of UTC — robust to host-clock
resets. The home page now formats the brief's date in each VISITOR's local
timezone (from its UTC freshness stamp), so nobody ever sees "tomorrow."
* Admin "Content served": articles live, fresh (7d), ingested (24h), summaries,
active sources, today's brief size — queries.content_stats().
* Admin "Source health": per active source, the failure streak, last error,
accepted contribution, and computed next-poll time (so backoff / "resting
until" is visible), via queries.source_health() reusing the feeds backoff
math. Failing sources sort to the top; times render in the viewer's zone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Feedback:
- feedback table; POST /api/feedback (anonymous-ok, optional category/email,
honeypot + per-day flood cap) stores + emails the admin; GET /api/admin/feedback.
- Shared feedback store + FeedbackModal; a speech-bubble opens it from the desktop
header, the mobile top bar (logo moves left), the footer, and /account. Feedback
section in /admin.
Stats (additive, same privacy model — no IP/UA/referrer/raw terms):
- Event vocab: summary_viewed (fired on /a load), full_story (card → source),
not_today/less_like_this/hide_topic, replace_used/replace_none, paywall_replace,
paywalled_source_open. Card title/image opens /a (no double-count); history
records via keepalive so it survives the nav.
- Dashboard: Accounts card (counts only), reading funnel (summary→source rate),
emotional-mix & friction, paywall, returning-visitor buckets. (Health metrics
deferred to a future monitoring dashboard.) 131 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make summaries the core reading experience (summary-first, source-forward):
- Cycle pre-warms summaries for Today's 7 (idempotent → only new ones hit the LLM).
- /api/brief items carry their cached summary; Today cards (hero + tiles) show it
inline, so Today reads as a calm briefing.
- Card title/image now open the /a summary page (the canonical artifact), with a
visible "Full story" link straight to the source on every card (the escape hatch).
- /a gains related-grouping chips + a Copy-link/share control.
- Tighten the summary prompt: original, factual, no quotations / no close paraphrase.
Long tail stays lazy+cached. No article bodies stored. 129 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- users.is_admin (+ migration); admin = is_admin OR email in GOODNEWS_ADMIN_EMAILS
(normalized). is_admin exposed on /api/auth/me. Server-authorized GET
/api/admin/stats (403 for non-admins).
- queries.admin_stats: visitors (today/7d/30d), returning vs one-and-done, top
opened articles, popular groupings + topics (derived from article_id at query
time), share breakdown, daily opens/visits trend — all aggregate, no PII.
- /admin page (gated, redirects non-admins): stat cards, CSS bar lists, a daily
trend; "Admin dashboard" link on /account for admins. 129 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- API (auth-required): GET/POST/DELETE /api/saved (+/api/saved/ids), GET/POST
/api/history, POST /api/import — all FK-safe (skip ids that no longer exist).
queries.saved/saved_ids/history reuse the feed article shape.
- Frontend: reactive savedIds store (SvelteSet) + optimistic toggleSave; a Save
control on cards for signed-in users; a "Saved" view (You sheet) with its own
empty state; newly-seen items mirror to account history (cross-device); and a
one-time import folds this device's anonymous history into the account on first
sign-in. Anonymous browsing unchanged. 115 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three-layer organization: primary topic (one per article, for ranking and
brief balance) + grouping tags (1-4 per article from a controlled vocabulary,
the organic "wandering" axis) + tonal flavor.
- taxonomy: add technology + learning topics; 4 calm tag families
(Discovery & Wonder, People & Kindness, Solutions & Progress, Mind & Craft)
defined in code, not the DB; ALLOWED_TAGS union + coerce_tags validation.
- db: article_tags(article_id, tag) join table + tag index.
- llm: tags added to the classifier json_schema (enum-constrained, maxItems 4)
and system prompt; normalize_scores coerces tags; upsert_article_score
replaces a row's tags atomically on every (re)classification.
- queries: feed gains a tag filter and exposes tags via group_concat; tag_counts.
- api: Article.tags, feed tag param, and /api/families with per-tag counts.
- tests: coerce/normalize/upsert/tag-filter/reclassify-replace/tag_counts +
/api/families. 99 passing.
Corpus reclassify (re-tag + new primary topics) runs separately against the
local LLM. Frontend (B2) pairs with this; the live site is unchanged until then.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the agreed model: the brief is server-authoritative and a client Replace is
a soft override that yields when genuinely new data arrives.
- build_daily_brief is now idempotent: if the composed selection is unchanged it
leaves the brief (and its created_at) alone, so the timer's 15-min rebuilds are
no-ops when no new data landed.
- /api/brief exposes generated_at (the brief's created_at = a content-change
stamp). The client pins its view against generated_at and keeps it across plain
refreshes, but drops it and shows the fresh server brief when generated_at
advances. Missed stories remain in the mood feeds.
Tests: idempotent rebuild (no-op vs content change) — 93 total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- New frontend/ SvelteKit static SPA (Svelte 5), served by FastAPI from
frontend/build (falls back to the legacy page if unbuilt).
- Calm design system: cream/sage palette, serif headlines, generous space,
no urgency colors, gentle motion (respects prefers-reduced-motion).
- Home screen: mood-mode nav (Today/Wonder/People Helping/Solutions/Light
Only/Grounded), the daily brief as a hero + remaining four, browsable mood
lanes, an explicit calm end-state, inline Not today / Less like this / Hide
affordances, and device-local Calm Filters mirroring goodnews/filters.py.
- Backend: moods.py + GET /api/moods (single source of truth for the modes);
FilterPrefs gains max_cortisol/max_ragebait ceilings (for Light Only).
- Push categorical filters (include/mute topics+flavors, ceilings) into SQL in
queries.feed so low-ranked-but-matching items (e.g. discovery for Wonder)
are not truncated by ranking; only avoid-terms stay a Python pass.
- PWA manifest + icon (installable; offline deferred per plan).
- Multi-stage Dockerfile builds the site then serves it from the API.
- Tests: queries.feed categorical filters (63 total). README updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- LocalModelClient.embed() calls the OpenAI-compatible /embeddings endpoint
(local nomic model); base_url shared with chat, model via GOODNEWS_EMBED_MODEL.
- New article_embeddings table and articles.duplicate_of column (+ migration).
- dedup module: embeds missing articles, clusters near-identical stories within
a date window by cosine similarity (pure-stdlib, vectors normalised once), and
marks all but the highest-ranked member of each cluster as a duplicate.
- 'dedup' CLI command; cycle now runs poll -> classify -> dedup -> brief.
- Feed and brief queries hide duplicates, so a story carried by multiple
outlets shows once.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- queries.py: shared read-only query helpers (feed, brief, category counts)
returning plain dicts, used by the API and available to the CLI.
- api.py: FastAPI service with Pydantic response models (the companion-app
contract), CORS, and endpoints for categories, feed, brief, and health;
mounts a static site at /.
- static/index.html: minimal dependency-free site rendering the daily five
and topic/flavor category browsing.
- 'goodnews serve' command launches uvicorn (lazy import; core CLI stays
pure-stdlib). Web deps live behind the optional [web] extra.
- Dockerfile + .dockerignore + build-system metadata so the service installs
and deploys cleanly, with the DB mounted as a shared volume.
- README: web/API and deployment docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thejayman77