thejayman77/upbeatBytes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bounded hero-image enrichment (og:image for brief items only)

Browse Source 
The grid stays typographic; the hero is the one intentional visual slot. At
brief-build time we fetch a hero-quality image for the daily five that lack one:
- enrich.py reads ONLY a page's <head> og:image/twitter:image and stores just
  the URL (never the body).
- SSRF-guarded: http(s) only, 6s timeout, 300KB cap, <=3 manual redirects each
  re-validated, and hosts rejected if any resolved address is private, loopback,
  link-local, multicast, reserved, or unspecified.
- image_checked_at column caches success AND failure, so an article is never
  retried forever.
- Wired into build-brief and cycle (brief items only, only if image missing and
  unchecked). Everything else stays metadata-only.
- Verified live: today's five all carry images (feed + enriched).

Tests: og:image parser, head-only scope, IP guard across internal ranges, and
enrich success + failure-caching (85 total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-31 12:37:41 +00:00
parent 3858380ffe
commit 9e8eddf46d
5 changed files with 230 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
goodnews/cli.py
+7 -1
View File
@@ -9,6 +9,7 @@ from pathlib import Path
from .briefs import build_daily_brief, show_brief
from .db import connect, init_db
from .dedup import DEFAULT_THRESHOLD, DEFAULT_WINDOW_DAYS, dedup as run_dedup
from .enrich import enrich_brief_images
from .feeds import (
fetch_feed,
parse_feed,
@@ -298,6 +299,10 @@ def main() -> None:
replace=args.replace,
)
print(f"Built brief {brief_id}")
bdate = args.date or date.today().isoformat()
found = enrich_brief_images(conn, bdate)
if found:
print(f"Enriched {found} hero image(s)")
print_brief(show_brief(conn, brief_date=args.date, limit=args.limit))
elif args.command == "show-brief":
print_brief(show_brief(conn, brief_date=args.date, limit=args.limit))
@@ -442,7 +447,8 @@ def _run_cycle_locked(conn: sqlite3.Connection, args: argparse.Namespace) -> Non
today = date.today().isoformat()
try:
brief_id = build_daily_brief(conn, brief_date=today, limit=5, replace=True)
print(f"brief: rebuilt {today} (id {brief_id})")
found = enrich_brief_images(conn, today)
print(f"brief: rebuilt {today} (id {brief_id}); {found} hero image(s) enriched")
except Exception as exc:
print(f"brief: skipped ({exc})")