images: cache + serve article images from our own origin (bounded, LRU-evicted)
Stop hotlinking news images from third-party CDNs (the source of the "blank until
you refresh a few times" graphic). New goodnews/newsimg.py caches a downscaled WebP
display copy (≤800px) beside the DB, like art_cache:
- GET/HEAD /api/img/{article_id} — resolves id→image_url (allowlisted to our corpus,
not an open proxy), fetch+cache on first miss, serve local after, immutable headers.
- cycle warms display copies for recent accepted-with-image articles (so the FIRST
view is already local) and prunes to a hard size cap (default 1 GB) by LRU eviction.
Frontend now points at /api/img/<id>: the hub lead, every ArticleCard (feed hero +
cards), and the /a/<id> share page's visible image. og:image/twitter:image stay the
source URL so social crawlers fetch the canonical image directly.
Storage is bounded by construction — over the cap, least-recently-used files are
evicted, so it can't grow without limit regardless of ingest rate. Tests cover
fetch/downscale, cache-hit (no refetch), bad-scheme/non-image rejection, fetch
failure, LRU prune, warm, and the endpoint allowlist.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+12
-1
@@ -13,7 +13,7 @@ from .games import generate_daily_puzzles
|
||||
from .localtime import local_today
|
||||
from .dedup import DEFAULT_THRESHOLD, DEFAULT_WINDOW_DAYS, cluster_duplicates, dedup as run_dedup
|
||||
from .geo import tag_articles as tag_geo
|
||||
from . import art, onthisday, quote, wotd
|
||||
from . import art, newsimg, onthisday, quote, wotd
|
||||
from .enrich import enrich_brief_images, enrich_read_times, enrich_recent_images, enrich_summarized_images
|
||||
from .summarize import generate_summary, get_summary
|
||||
from .feeds import (
|
||||
@@ -599,6 +599,17 @@ def _run_cycle_locked(conn: sqlite3.Connection, args: argparse.Namespace) -> Non
|
||||
except Exception as exc:
|
||||
print(f"recent images: skipped ({exc})")
|
||||
|
||||
# Cache downscaled display copies of those images on our own origin (so the
|
||||
# hub/feed serve local files, not flaky third-party hotlinks), then enforce the
|
||||
# size ceiling with LRU eviction. Both bounded + non-fatal.
|
||||
try:
|
||||
warmed = newsimg.warm(conn)
|
||||
pr = newsimg.prune()
|
||||
print(f"images: cached={warmed} size={pr['after'] // 1048576}MB/"
|
||||
f"{pr['cap'] // 1048576}MB evicted={pr['removed']}")
|
||||
except Exception as exc:
|
||||
print(f"images: skipped ({exc})")
|
||||
|
||||
# Full-article read-times: count words for recent accepted articles so the
|
||||
# front door can show "Full story · ~N min" next to our gist (bounded per cycle).
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user