Bounded hero-image enrichment (og:image for brief items only)

The grid stays typographic; the hero is the one intentional visual slot. At brief-build time we fetch a hero-quality image for the daily five that lack one: - enrich.py reads ONLY a page's <head> og:image/twitter:image and stores just the URL (never the body). - SSRF-guarded: http(s) only, 6s timeout, 300KB cap, <=3 manual redirects each re-validated, and hosts rejected if any resolved address is private, loopback, link-local, multicast, reserved, or unspecified. - image_checked_at column caches success AND failure, so an article is never retried forever. - Wired into build-brief and cycle (brief items only, only if image missing and unchecked). Everything else stays metadata-only. - Verified live: today's five all carry images (feed + enriched). Tests: og:image parser, head-only scope, IP guard across internal ranges, and enrich success + failure-caching (85 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 12:37:41 +00:00
parent 3858380ffe
commit 9e8eddf46d
5 changed files with 230 additions and 1 deletions
@@ -44,6 +44,7 @@ CREATE TABLE IF NOT EXISTS articles (
    url_hash TEXT NOT NULL UNIQUE,
    title_hash TEXT,
    duplicate_of INTEGER REFERENCES articles(id) ON DELETE SET NULL,
+    image_checked_at TEXT,
    FOREIGN KEY (source_id) REFERENCES sources(id)
 );

@@ -152,6 +153,8 @@ def _migrate(conn: sqlite3.Connection) -> None:
        conn.execute(
            "ALTER TABLE articles ADD COLUMN duplicate_of INTEGER REFERENCES articles(id)"
        )
+    if "image_checked_at" not in article_cols:
+        conn.execute("ALTER TABLE articles ADD COLUMN image_checked_at TEXT")
    # Created here (not in SCHEMA) so it runs after the column exists on upgrades.
    conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_duplicate_of ON articles(duplicate_of)")