Make "no blurry images" sustainable, not a one-off cleanup. RSS feed thumbnails
(~44% were ~90px) were stored at ingest and upscaled to mush, so new articles
would reintroduce them. Now image_url is filled ONLY by the quality-gated
og:image enrichment:
* insert_article no longer stores the feed image (was canonicalize_url(item...)).
* enrich_recent_images(): the cycle fetches a quality og:image for the newest
accepted, imageless articles each run (bounded), keeping Latest photo-rich.
* Brief + on-open enrichment unchanged.
Net: every stored image is a validated, ≥450px og:image; the rest are clean
placeholders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-traffic cleanup from an audit:
* Scheduler: poll_due_sources now keys on the last *attempt* (success or
failure), not the last success, and scales the wait by the consecutive-
failure streak (capped at a day). A failing feed (e.g. Phys.org's HTTP 429s)
used to be retried every cycle because it had no successful run; it now backs
off and recovers on its own. Extracted due_source_rows() + tests.
* FK hygiene: deleting a daily_brief is supposed to cascade to its items, but
SQLite enforces foreign keys per-connection — connect() already sets the
pragma, so the cascade is correct going forward; added a regression test.
(Orphaned items + Phys.org settings were cleaned directly on the live DB.)
* a11y: modal/drawer dialogs are now focusable (tabindex), close on Escape
(window) and on backdrop click via a target check (dropping the inner
stopPropagation handlers). Build is warning-free.
* tests: conftest points any un-mocked LLM client at a closed port with a 1s
timeout, so an accidental real call fails fast instead of hanging the suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the calm north star (images support reading, never become a stimulation
layer; metadata-only stays the posture):
- Image-less cards are now designed, not missing: secondary cards are text-first
(no empty media band), and an image-less hero becomes a fully typographic lead
with a faint topic wordmark behind it (CSS attr(data-topic)). No big empty
image space is ever reserved.
- Opportunistic extraction: parse the first <img src> from a feed's
content/description HTML when present, canonicalized — never fetching the
article page. Applies to new ingests (existing rows keep their current image).
- Held by deliberate choice: og:image page enrichment, stock/AI imagery, and any
image-coverage requirement for sources.
Tests: feed HTML image extraction (72 total).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add source health columns (last_success_at, last_error_at, last_error,
consecutive_failures, review_flag, review_reason) via SCHEMA + migration.
- poll_source maintains them: success resets the failure streak and records the
success time; failure increments it and stores the latest error.
- review_sources() flags active sources that are stale, repeatedly failing,
low-acceptance, duplicate-heavy, or doom-skewed (high cortisol/ragebait) over
a recent window. It is purely advisory: it sets review_flag/review_reason and
never changes the active column (human stays in the loop), clearing the flag
when a source recovers.
- CLI review-sources; cycle runs it as a final step (--no-review to skip);
source-report shows a review line for flagged feeds.
- Tests: healthy/failing/stale/low-acceptance/recovery and never-deactivates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- New source_candidates staging table (status suggested/quarantined/rejected/
promoted, preview_json snapshot) so untrusted/suggested feeds stay out of the
real ingestion path until reviewed.
- sources.py: save_candidate (re-preview never revives a curator's rejection),
list_candidates, reject_candidate, promote_candidate (copies into sources,
inactive by default — active on approval; never automatic).
- CLI: suggest-source / list-candidates / promote-candidate / reject-candidate.
- API: read-only GET /api/candidates (writes stay CLI-only — no unauthenticated
public write surface yet).
- Fix deprecated ElementTree truth-value test in _parse_rss.
- Tests: candidate lifecycle (save/list/promote/reject, status preservation,
name derivation) — 51 total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- feeds.preview_feed(): fetch + score a sample WITHOUT persisting; returns
freshness, acceptance rate, cortisol/ragebait/PR averages, and example
accepted/rejected items. With an LLM client it also returns topic/flavor mix
and the model's (accurate) acceptance view.
- CLI 'preview-source URL [--sample] [--classify]'.
- API 'GET /api/source-preview?url=&sample=&classify=' with an http(s)-only
guard (SSRF note left for go-public hardening).
- Site 'Suggest a source' panel with Quick check (heuristic, instant) and Deep
check (model, accurate), rendered DOM-safely.
- Tests: network-free preview_feed tests via monkeypatched fetch (45 total).
- README documents the command, endpoint, and updated roadmap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- poll_due_sources(): polls only sources whose last successful poll is older
than their poll_interval_minutes (or never polled), finally giving that
config field meaning.
- classify gains only_unclassified to spend the LLM solely on new (heuristic)
articles, so a frequent scheduled run stays cheap.
- 'cycle' command runs poll-due -> classify-new -> rebuild-today's-brief, with
each step non-fatal so a down model endpoint or empty day never aborts it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thejayman77