Per Codex: source-controlled strings (name, feed_url, last_error, review_reason)
could be read as formulas by spreadsheet apps if they start with = + - @. Add
_csv_cell — prefixes such strings with an apostrophe; numbers pass through
untouched (no risk, and avoids mangling negatives). Routed every exported cell
through it. Test: a =HYPERLINK(...) source name is escaped, never bare.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex v1 — boring-in-the-best-way: inspect/archive operational data outside
the app. Admin-gated, Python csv module, text/csv + attachment disposition.
* GET /api/admin/export/sources.csv — current-state snapshot per source: name,
feed/homepage, status, visible, served/accepted/total, acceptance/duplicate/
accepted-dup/image-coverage %, last success/error, retry-after, review.
* GET /api/admin/export/audience.csv?days= — summary block (visitors, returning,
accounts, feedback, shares) + a blank line + the daily visits/opens series;
range applies to audience, sources is a snapshot.
* source_health now also returns feed_url/homepage. Small download links on the
Sources + Audience tabs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: a paused/retired source with a future retry_after_at shouldn't nag
'rate-limited for 12h+' — it's intentionally out of polling. Scope long_rest to
active (matching the other operational items). Test: paused/retired rate-limited
sources stay quiet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex — make the Overview strip diagnostic without making the operator hunt
through tables. Aggregated (one calm line per condition with a count), volume-
gated, conservative thresholds:
* Stale: active+visible source, last success > 10 days ago (warn).
* High rejection: >=20 ingested, acceptance < 25% (info).
* High duplicate: >=10 accepted, accepted-dup > 50% (info).
* Thin images: >=10 served, per-source image coverage < 25% (info).
* Long rate-limit: retry_after_at more than 12h out (info).
source_health gains a per-source images count + image_coverage. _attention takes
an optional now (for tests). Existing site-wide items (global image coverage,
thin brief, unread feedback) unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: the Next poll column computed only the streak-backoff time, so a
rate-limited source could show an earlier Next poll than the real gate (which
also requires retry_after_at <= now). Take the later of the two in the Python
post-process so the admin table agrees with due_source_rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's spec — a publisher saying "slow down" shouldn't make a feed look
broken, but repeated 429s stay visible via last_success_at / stale-source.
* Schema: sources.retry_after_at (nullable) + migration.
* feeds.parse_retry_after: delta-seconds OR HTTP-date → UTC stamp; ignores
invalid/negative/past; caps at now + MAX_BACKOFF_MINUTES.
* fetch_feed raises RateLimited (carrying the parsed time) on a 429.
* poll_source: on 429 set retry_after_at + last_error, status='rate_limited',
and do NOT increment consecutive_failures; on success clear retry_after_at;
non-429 failures unchanged.
* due_source_rows requires BOTH the streak backoff elapsed AND retry_after_at
passed (i.e. the later of the two).
* Admin: source_health returns retry_after_at; status reads
"rate-limited · rests until …" rather than "failed/resting".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the supervised source-candidate flow into Sources (Codex's v1 scope), so
adding feeds no longer needs the CLI.
* feeds.safe_fetch_feed: SSRF-safe fetch for UNTRUSTED (admin-pasted) URLs —
http(s) only, every redirect hop re-validated via enrich._host_is_public,
body size-capped, bounded redirects, no cookies. preview_feed gains a
`fetcher` param; the API path passes safe_fetch_feed (NOT the raw fetch_feed
used for already-vetted polling).
* API (admin-gated): GET /candidates; POST /candidates (suggest+preview, gated
before the outbound fetch, no DB conn held during network); /{id}/preview
(explicit re-preview); /{id}/promote (paused by default, returns the new
source + updated candidate); /{id}/reject. rejected stays on candidates only.
* Admin Sources tab: "Add a source" field + a candidate queue showing the
preview (pass rate, recent count, example headlines) with Promote (as paused,
or Activate immediately) / Re-preview / Reject.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: upsert_sources() wrote `active` but not `status`, so a candidate
promoted inactive (the pipeline default) became active=0 + status='active' —
the exact mirror drift Phase 1 set out to avoid (scheduler won't poll, admin UI
shows "active"). Now derive status from an explicit value or from active, mirror
active off status, and write both columns together (insert + conflict update).
Test: promote_candidate(active=False) → status='paused', active=0.
Also fix stale source_health docstring (now includes retired).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's plan — introduce a lifecycle without a risky "change the source of
truth everywhere" moment.
* Schema: sources.status (active|paused|retired) + content_visible; migration
backfills status from active (active=1→active, else paused), content_visible=1.
* `active` is kept as a SYNCED MIRROR: status active→active=1, paused/retired→0,
so the scheduler/CLI/legacy code keep working unchanged.
* Retire stops polling but keeps articles visible (non-destructive). Hiding is a
separate, reversible lever: content_visible=0 drops a source's articles from
the public feed + brief (read AND build), behind a confirm. Personal saved/
history are untouched.
* API: /sources/{id}/status (validates, mirrors active) + /visibility, replacing
/active. source_health returns status + content_visible.
* Admin: status column (active/paused/retired + "hidden"), Retired filter,
Pause/Resume · Retire/Restore · Hide/Show actions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: slicing the SANITIZED html with [:8000] could cut through a tag or
entity. Cap the RAW editor HTML (20k) before sanitizing instead, and have
sanitize_reply_html auto-close any still-open allowed tags so malformed input
can never leave a dangling/severed tag in message_html or the email body.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the Markdown composer with a small contenteditable WYSIWYG (Codex
greenlit for this narrow, admin-only surface).
* markup.py: render_reply_html → sanitize_reply_html + reply_html_to_text.
Allowlist rebuild via stdlib HTMLParser — keeps strong/em/p/br/ul/ol/li and
span ONLY with a whitelisted font-size (13/15/18/22px); normalizes b→strong,
i→em, div→p, <font size> → safe span; drops links/images/arbitrary styles
(content kept as escaped text) and discards script/style content entirely.
* API: FeedbackReplyBody.html (raw editor HTML); endpoint sanitizes → message_html,
derives plain text → stored message + the email text/plain part. Unchanged:
multipart send, store-on-success, conn released during SMTP, mark-read, 404/400/422.
* Frontend: contenteditable editor + toolbar (Bold/Italic/Size/• List/1. List),
execCommand with styleWithCSS=false for semantic tags, font size wraps the
selection in a fixed-px span, paste intercepted as plain text. No links yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex's optional note: alongside the ingest-wide duplicate_rate, expose
accepted_dup_rate — of what a source got ACCEPTED, how much was a duplicate of
already-served content (accepted_total − served). Nearly free (derived from
existing counts); surfaced as a tooltip on the Dup column so the table stays
uncluttered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Codex: a constrained Markdown-ish composer rather than contenteditable.
* goodnews/markup.render_reply_html — escapes everything first, then introduces
only a tiny whitelist (**bold**, - bullets, #/##/### headings, paragraphs,
line breaks). No links, attributes, inline styles, or raw HTML passthrough.
* feedback_replies.message_html column (+ live migration); replies store both
the Markdown text and the rendered HTML.
* email_send.send_feedback_reply now sends multipart text/plain + text/html
(the sanitized render, wrapped in a trusted email template).
* Frontend: textarea + a small toolbar (Bold / • List / H) that inserts
Markdown; the reply thread renders the server-sanitized HTML.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reply to a reader from the admin inbox instead of a mailto. Per Codex: keep v1
plain text (no rich editor — defers the user's bold/bullets ask as a fast-follow).
* DB: feedback_replies table (feedback_id, user_id, message, sent_to, sent_at),
created on the live DB.
* email_send.send_feedback_reply: plain-text "Re: Your Upbeat Bytes feedback"
with a quoted context block, no analytics/account details.
* API: POST /api/admin/feedback/{id}/reply — admin-gated, requires the feedback
exists (404) and has a contact_email (400), trims+caps the message; sends via
SMTP and only records the reply on success (502 on send failure so the UI keeps
the draft); marks the item read. Feedback list now includes each item's replies.
* Frontend: inline composer (Send/Cancel, sending state, error keeps draft) +
reply thread under the message; Reply only shows when there's an address,
else "No reply address".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/api/admin/stats accepts ?days= (clamped to 7/30/90, default 30) → passed to
admin_stats, which already windows visitors, retention, funnel, sharing, daily
trend, and the top lists by that span. Frontend adds a Window picker on the
analytics tabs (Overview/Content/Audience); changing it refetches and the
windowed labels ("Visitors (Nd)", "Last N days", "Returning visitors (Nd)")
follow. Corpus totals + source health are unaffected (not time-windowed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the Sources tab into a real management console (per Codex):
* source_health now lists ALL sources (active + paused) with backing metrics:
served / accepted_total / total_articles / duplicates + acceptance & duplicate
rates + review_reason, alongside last success/attempt, next poll, failures.
* Admin endpoints (gated, 404 on missing): POST sources/{id}/active (pause/
resume) and /review (flag/clear with reason).
* Pausing only stops future polling — the feed query has no active filter, so a
paused source's accepted articles stay live.
* Frontend: metric table + Paused filter + per-row Pause/Resume & Flag/Clear
(optimistic, revert on failure). Attention 'resting' now scoped to active.
Retire/Delete intentionally deferred (distinct lifecycle state, later).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per audit: read-toggle and delete returned {"ok":true} even for a nonexistent
id. Return 404 when no row is affected, so the optimistic UI can distinguish a
stale/already-deleted row from a real success. (The postJSON/delJSON imports
flagged in the audit were already present — verified in source + built bundle.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the admin Feedback section a real inbox.
* DB: feedback.read_at column (schema + idempotent migration).
* API: feedback list returns read_at; POST /api/admin/feedback/{id}/read
{read} toggles it; DELETE /api/admin/feedback/{id} removes a message
(both admin-gated). admin_stats gains feedback_unread; the Attention strip
and the tab badge now count UNREAD, not total.
* Frontend: unread messages are highlighted with an accent rail + dot; an
Unread filter joins the category chips; each message has Mark read/unread
and Delete (confirm), with optimistic updates that revert on failure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Click a source name on any card → a feed of just that source's articles,
newest-first, still accepted / non-duplicate / boundary-filtered (the calm
promise isn't bypassed). A natural way to follow a publication's feel.
* queries.feed + /api/feed: source_id filter; Article output gains source_id.
* Frontend: source label is a button → transient 'source:<id>' view (like
'tag:<slug>'), rendered in the feed grid with Load more, header = source name.
* Ad-hoc, not a pinned lane. Foundation for a future source page (metadata) +
Follow; shareable /source/<slug> route and source_view analytics come then.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make "no blurry images" sustainable, not a one-off cleanup. RSS feed thumbnails
(~44% were ~90px) were stored at ingest and upscaled to mush, so new articles
would reintroduce them. Now image_url is filled ONLY by the quality-gated
og:image enrichment:
* insert_article no longer stores the feed image (was canonicalize_url(item...)).
* enrich_recent_images(): the cycle fetches a quality og:image for the newest
accepted, imageless articles each run (bounded), keeping Latest photo-rich.
* Brief + on-open enrichment unchanged.
Net: every stored image is a validated, ≥450px og:image; the rest are clean
placeholders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some cards showed blurry photos — feed RSS thumbnails (~90×90, e.g. Phys.org's
/tmb/ path) that load fine but upscale to mush in the banner. Add a header-based
dimension parser (PNG/GIF/JPEG/WebP, stdlib only) and fold a minimum-size gate
(450×250) into the image validation, alongside the existing load check. Images
we can't measure (SVG/AVIF) still pass on content-type. A re-prune clears the
small ones already stored so those cards fall back to the clean placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A stored og:image isn't proof it renders: signed/hotlink-protected URLs (e.g.
the Guardian's i.guim.co.uk) 401 on a direct browser load, so they counted
toward coverage yet always fell back. Now fetch_og_image confirms the image
truly returns 200 + image/* (requested no-referrer, same SSRF-safe redirect
handling) before storing it. Add prune_broken_images() to clear already-stored
URLs that no longer load, so coverage is honest and those cards show the
placeholder cleanly. The browser onerror→placeholder remains the final safety net.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tie image enrichment to attention (per review): when an article earns a summary
(i.e. a reader reached it), best-effort fetch a real og:image if it lacks one —
never blanket-fetch every ingested article. Adds:
* enrich_article_image() — single-article fetch, leaves existing images alone,
retries an imageless article only after 7 days, stamps image_checked_at.
* generate_summary() calls it after caching (wrapped; never breaks summaries).
* enrich_summarized_images() + `goodnews enrich-images` CLI — slow background
backfill of already-summarized, accepted, imageless articles.
* Quality gate: extend the generic-image skip list with data:/tracking-pixel/
spacer markers (on top of the existing logo/placeholder + unbranded-BBC logic).
This is coverage only; display (editorial rhythm, tile treatment) comes next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restructure the nav around two permanent lanes, then the reader's chosen ones:
"Highlights" (the curated daily brief — formerly "Today") and "Latest" (the
freshest accepted stories, newest-first). Now that the gate is tight, a
chronological "incoming" feed is safe to expose.
* feed(): new sort="latest" (pure recency) alongside the default best-first
rank; /api/feed exposes sort=ranked|latest (validated). Still accepted-only
and boundary-respecting either way.
* lanes.py: two pinned lanes (Highlights + Latest) instead of one.
* Home: "Latest" view + "Load more" pagination for every feed view (offset-
paged, de-duped). Mobile bottom bar gains a Latest tab.
* LanePicker shows both pinned lanes; nav rail renders them first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Date fix: introduce GOODNEWS_TZ (goodnews/localtime.py) so the brief's "today"
rolls over in a pinned zone (Eastern) instead of UTC — robust to host-clock
resets. The home page now formats the brief's date in each VISITOR's local
timezone (from its UTC freshness stamp), so nobody ever sees "tomorrow."
* Admin "Content served": articles live, fresh (7d), ingested (24h), summaries,
active sources, today's brief size — queries.content_stats().
* Admin "Source health": per active source, the failure streak, last error,
accepted contribution, and computed next-poll time (so backoff / "resting
until" is visible), via queries.source_health() reusing the feeds backoff
math. Failing sources sort to the top; times render in the viewer's zone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-traffic cleanup from an audit:
* Scheduler: poll_due_sources now keys on the last *attempt* (success or
failure), not the last success, and scales the wait by the consecutive-
failure streak (capped at a day). A failing feed (e.g. Phys.org's HTTP 429s)
used to be retried every cycle because it had no successful run; it now backs
off and recovers on its own. Extracted due_source_rows() + tests.
* FK hygiene: deleting a daily_brief is supposed to cascade to its items, but
SQLite enforces foreign keys per-connection — connect() already sets the
pragma, so the cascade is correct going forward; added a regression test.
(Orphaned items + Phys.org settings were cleaned directly on the live DB.)
* a11y: modal/drawer dialogs are now focusable (tabindex), close on Escape
(window) and on backdrop click via a target check (dropping the inner
stopPropagation handlers). Build is warning-free.
* tests: conftest points any un-mocked LLM client at a closed port with a 1s
timeout, so an accidental real call fails fast instead of hanging the suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Readers can now choose which quick-access lanes sit above the feed; "Today"
stays pinned. The pool (goodnews/lanes.py, served at /api/lanes) is one source
of truth over three lane kinds the feed already renders: moods, primary topics,
and high-volume Discovery tags. Selection lives in the existing prefs blob
(localStorage + /api/prefs sync); the filter parser ignores the new `lanes`
field, so it rides along harmlessly. Default = today's moods, unchanged.
Food/Space stay grouping tags rather than primary topics (per review): `space`
already existed; added `food` to the Mind & Craft family so the classifier
assigns it, and seeded the Food lane by re-tagging the two food sources.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the summary pages discoverable so traffic compounds passively:
- /today: a server-rendered, shareable + indexable digest of today's brief —
each item's title (→ /a summary), our summary, and a source link. OG/Twitter
meta + self-canonical.
- /sitemap.xml: dynamic — home, /today, and every accepted non-duplicate /a page
with lastmod. robots.txt allows all and points to it.
- Home (SPA shell) gains canonical + OG/Twitter tags for cleaner unfurls.
- Caddy routes /today + /sitemap.xml to the API. 133 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Feedback:
- feedback table; POST /api/feedback (anonymous-ok, optional category/email,
honeypot + per-day flood cap) stores + emails the admin; GET /api/admin/feedback.
- Shared feedback store + FeedbackModal; a speech-bubble opens it from the desktop
header, the mobile top bar (logo moves left), the footer, and /account. Feedback
section in /admin.
Stats (additive, same privacy model — no IP/UA/referrer/raw terms):
- Event vocab: summary_viewed (fired on /a load), full_story (card → source),
not_today/less_like_this/hide_topic, replace_used/replace_none, paywall_replace,
paywalled_source_open. Card title/image opens /a (no double-count); history
records via keepalive so it survives the nav.
- Dashboard: Accounts card (counts only), reading funnel (summary→source rate),
emotional-mix & friction, paywall, returning-visitor buckets. (Health metrics
deferred to a future monitoring dashboard.) 131 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
clearSession only reset the device-local history; for a signed-in user the panel
shows the account history (serverHistory), which was never cleared and never sent
to the server — so it looked like nothing happened. Add DELETE /api/history
(clear all) and have clearSession reset serverHistory + call it when signed in.
- users.is_admin (+ migration); admin = is_admin OR email in GOODNEWS_ADMIN_EMAILS
(normalized). is_admin exposed on /api/auth/me. Server-authorized GET
/api/admin/stats (403 for non-admins).
- queries.admin_stats: visitors (today/7d/30d), returning vs one-and-done, top
opened articles, popular groupings + topics (derived from article_id at query
time), share breakdown, daily opens/visits trend — all aggregate, no PII.
- /admin page (gated, redirects non-admins): stat cards, CSS bar lists, a daily
trend; "Admin dashboard" link on /account for admins. 129 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- events table (kind, article_id, visitor_hash, day) with a UNIQUE key that dedups
to one row per visitor-day — caps volume and makes counts mean distinct
visitor-days. NO ip/ua/referrer/url. Groupings derived from article_id at query
time, never stored.
- POST /api/events (public): whitelisted kinds (visit/open/share_ub/copy_source/
native_share/source_click); visitor token hashed server-side (never raw).
- Frontend analytics.js: random localStorage visitor token; track() via sendBeacon;
visit once/day; open on article click; share_ub/copy_source/native_share from the
share menu; /a landing pages fire source_click. 127 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /a/<id> page now carries an original short summary so it stands on its own,
without republishing the publisher's article:
- summarize.py: transient SSRF-guarded fetch of the article text → local LLM
writes a 2-4 sentence ORIGINAL summary (our words). Cached in article_summaries
forever; we store only our summary, never the body. Generated lazily (only for
shared/viewed articles), de-duped so concurrent hits don't double-generate.
- /a serves cached-or-pending; when pending it shows a calm "summary on its way,
read at {source}" note and self-polls /api/summary/<id>, swapping the summary
in the moment it's ready (never blocks the page on the batch-tier LLM).
- Share menu warms generation on open so recipients usually get the rich version.
- Container reaches the arbiter at arbiter:8080 over caddy_web (LLM env added to
the API container). 124 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Server-rendered /a/<id> "pointer" page (FastAPI): OG/Twitter meta from the
article's title/why/image, self-canonical (UB pages are the canonical for
themselves), a prominent "Read the full story at {source}" button and a quiet
"Explore more on Upbeat Bytes". No article body. Unknown/rejected/duplicate/
malformed ids → a calm 404 (no stack traces). Text-card preview when no image.
- Caddy routes /a/* to the API.
- Card Share control → menu: native Share… (where available), Copy link (the UB
card page), Copy source link. Boundary actions now hide-on-hover via a .mute
class so Save/Replace/Share stay visible. 122 tests pass.
(Event tracking for shares/opens lands with the analytics step next.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
find_or_create_user returned early when the identity already existed, so a
returning Google sign-in never refreshed the profile picture (the name had been
set earlier, at link time — which is why name worked but avatar stayed null).
Now profile bits refresh on every sign-in. Also fall back to the OIDC userinfo
endpoint for the picture if the ID token omits it. 119 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Capture the Google profile picture (picture claim) into users.avatar_url; an
Avatar component shows it, falling back to the initial. Used in the desktop
header and the mobile "You" tab (which now shows the user when signed in).
- Move account/settings to its own route /account (robust + scrolls to top),
reached by the desktop avatar and the mobile You tab; drop the inline "You"
sheet. AccountPanel gains a Sign out action; the page links to Saved/History/
Boundaries via home intent params (?view= / ?open=).
- db: users.avatar_url (schema + idempotent migration). 118 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Prefs sync: GET/PUT /api/prefs store Calm Filters/Boundaries on the account.
On sign-in the client adopts the account's prefs if present, else seeds them
from the device; every change PUTs to the account so tuning follows you across
devices. (Login side-effects run under untrack so browsing doesn't re-trigger.)
- Account panel: GET /api/account (email, connected sign-in methods, saved count,
active sessions); Export my data (GET /api/account/export → JSON download);
Sign out everywhere (revoke all sessions); Delete account (cascades to all
account data) with an inline confirm. Reachable from You → Account.
Deferred to a follow-up: link/unlink a provider (OAuth link-mode) and per-session
revoke. 118 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
History was logging every article merely displayed, which made it noise. Split
the two concepts cleanly:
- "displayed" (seenIds) still tracks everything shown, but only to stop Replace
recycling stories — it no longer feeds history.
- "history" now records only deliberate events: articles the user OPENED (card
click) or ones they REPLACED away (recoverable accidental swaps).
Also: per-item removal (× in the History panel; DELETE /api/history/{id}), and
when signed in the panel shows the account (cross-device) history. First-sign-in
import now folds the meaningful history (not everything shown). Copy updated.
115 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- API (auth-required): GET/POST/DELETE /api/saved (+/api/saved/ids), GET/POST
/api/history, POST /api/import — all FK-safe (skip ids that no longer exist).
queries.saved/saved_ids/history reuse the feed article shape.
- Frontend: reactive savedIds store (SvelteSet) + optimistic toggleSave; a Save
control on cards for signed-in users; a "Saved" view (You sheet) with its own
empty state; newly-seen items mirror to account history (cross-device); and a
one-time import folds this device's anonymous history into the account on first
sign-in. Anonymous browsing unchanged. 115 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- oauth_google.py (stdlib): PKCE, auth URL, code exchange, ID-token claim
validation (iss/aud/exp/email_verified — token comes straight from Google's
token endpoint over TLS, so no signature re-verify / JWKS needed).
- API: GET /api/auth/google/start (302 to Google, PKCE + signed state cookie
binding the flow to the browser) and /callback (CSRF-checked state, exchange,
find-or-create by verified email → links to an existing magic-link account,
session cookie, redirect home). Errors land on /auth/verify?error=google.
- SignIn modal: "Continue with Google" + an "or email link" divider.
- 112 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- POST /api/auth/email/start — validate email, rate-limit, email a single-use
magic link (identical reply regardless, so no account enumeration).
- POST /api/auth/email/verify — consume token, find-or-create user, open a
session, set an httpOnly cookie (web) and return a bearer token (app).
- GET /api/auth/me, POST /api/auth/logout.
- Session resolved from cookie OR Authorization: Bearer; cookie is Secure in
prod (https), relaxed for http so tests round-trip. CORS now allows POST.
Live SMTP send verified against the DNSExit relay (587/STARTTLS). 108 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for self-hosted accounts (magic link + Google later), no third parties.
- db: account tables (users, identities, login_tokens, sessions, saved_articles,
user_history, user_prefs); identities link multiple sign-in methods to one user
by verified email. connect() now enables WAL + busy_timeout so the API can write
account data alongside the host ingestion cycle.
- auth.py: users/identities (find-or-create + link), single-use magic-link tokens,
opaque sessions — all secrets stored only as SHA-256 hashes.
- email_send.py: minimal STARTTLS SMTP sender + the magic-link email.
Secrets (SMTP, Google, session) live in the API container's env_file, not git.
API endpoints + sign-in UI come next. 105 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
BBC's og:image comes from the "branded_news" CDN path with a "BBC NEWS" logo
baked into the picture (shows as "…EWS" once the hero crops it). The identical
photo is served under "cpsprodpb" with no logo, so rewrite branded_news →
cpsprodpb. Best of both: full-resolution hero, no burned-in branding. Re-enriched
recent briefs so live images swap over. 99 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
og:image extraction rejected any URL containing "branded_news" as a generic share
image, but that's BBC's normal CDN path for real article photos. So every BBC hero
fell back to the 240px RSS thumbnail (blurry when shown large). Drop that marker;
keep the genuine placeholder markers (facebook-default, og-default, etc.). Updated
the test to assert BBC branded_news paths pass through. 99 tests pass.
(One-time: cleared image_checked_at on the 57 previously-checked articles and
re-enriched recent briefs so existing thumbnails upgrade to og:images.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three-layer organization: primary topic (one per article, for ranking and
brief balance) + grouping tags (1-4 per article from a controlled vocabulary,
the organic "wandering" axis) + tonal flavor.
- taxonomy: add technology + learning topics; 4 calm tag families
(Discovery & Wonder, People & Kindness, Solutions & Progress, Mind & Craft)
defined in code, not the DB; ALLOWED_TAGS union + coerce_tags validation.
- db: article_tags(article_id, tag) join table + tag index.
- llm: tags added to the classifier json_schema (enum-constrained, maxItems 4)
and system prompt; normalize_scores coerces tags; upsert_article_score
replaces a row's tags atomically on every (re)classification.
- queries: feed gains a tag filter and exposes tags via group_concat; tag_counts.
- api: Article.tags, feed tag param, and /api/families with per-tag counts.
- tests: coerce/normalize/upsert/tag-filter/reclassify-replace/tag_counts +
/api/families. 99 passing.
Corpus reclassify (re-tag + new primary topics) runs separately against the
local LLM. Frontend (B2) pairs with this; the live site is unchanged until then.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the agreed model: the brief is server-authoritative and a client Replace is
a soft override that yields when genuinely new data arrives.
- build_daily_brief is now idempotent: if the composed selection is unchanged it
leaves the brief (and its created_at) alone, so the timer's 15-min rebuilds are
no-ops when no new data landed.
- /api/brief exposes generated_at (the brief's created_at = a content-change
stamp). The client pins its view against generated_at and keeps it across plain
refreshes, but drops it and shows the fresh server brief when generated_at
advances. Missed stories remain in the mood feeds.
Tests: idempotent rebuild (no-op vs content change) — 93 total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mood feeds now honor the same dismissed list as the brief: /api/feed accepts an
exclude param (over-fetching to stay full), and the client passes the persisted
dismissed set. Swapping a story away now keeps it gone everywhere — brief and
browse — not just on the home view. Also simplified the feed filter path to the
shared _prefs_sql_kw helper.
Tests: feed exclude (91 total).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A reader who swaps a story away should keep that swap after a refresh; before,
the server re-served the original brief.
- localStorage now persists seen / dismissed / history (loadJSON/saveJSON).
- /api/brief accepts an exclude list; dismissed (replaced-away) ids are dropped
and the highlights refill around them, so swaps stick and stay full.
- Replace records the swap to dismissed+seen and persists; the seen-set
(persisted) keeps Replace from recycling across refreshes too.
- History panel survives refresh and gains 'Clear what I've seen (start fresh)'
so it never feels suffocating. Saved history/favorites still come with sign-in.
Tests: brief exclude + refill (90 total).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a reader's boundary (avoid-term, muted topic/flavor, pause) removes a brief
item, top the highlights back up with other readable, boundary-respecting good
news instead of showing fewer cards — so 'Highlights from Today' stays full and
still honors what they don't want to see. (Reverses the earlier filter-down-only
MVP, now that the count is fixed at seven.)
- /api/brief: after filtering by prefs, refill from the accepted pool (same
categorical SQL filters + avoid-terms) excluding already-shown items.
- Shared _prefs_sql_kw helper for feed/replacement/brief filters.
- Tests: refill stays full and respects mute + avoid-terms (89 total).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thejayman77