images: fix two fetcher bugs + add source-level image-rights policy (Codex)

Fetcher (the two remaining bugs Codex found):
- Real redirects are now followed. _NoRedirect makes urllib RAISE HTTPError on 3xx, so
  the old status-branch was dead code (mocked tests masked it). Handle 301/302/303/307/308
  HTTPError as redirects (re-validate the destination); classify 4xx≠429 as PERMANENT
  (negative-cached), 429/5xx/network as transient. Real-opener redirect + 404/5xx tests.
- The megapixel ceiling is now enforced: explicit `w*h > _MAX_PIXELS` check BEFORE load()
  (Pillow only warns at MAX_IMAGE_PIXELS). Test with a lowered ceiling.

Image-rights policy (per Codex + owner decision — only cache what's cleared):
- sources.image_policy: 'cache' (re-host a downscaled copy — license/permission/PD only),
  'remote' (hotlink the publisher's image — the conservative DEFAULT), 'none' (no image).
- newsimg.display_url resolves the display URL per policy; applied in Article.from_row so
  feed/brief/history return the right URL, and in share.py (og/twitter still reference the
  publisher's own image, never re-hosted). warm() + /api/img both gated on 'cache'.
- Frontend uses the server-resolved image_url (reverted the hardcoded /api/img); the
  graceful retry covers remote hotlinks too. Admin: per-source image-policy selector +
  POST /api/admin/sources/{id}/image-policy. Default 'remote' → nothing re-hosted until
  a source is explicitly cleared.

445 backend + 36 frontend tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-30 14:01:11 -04:00
parent a55ba185a8
commit 8a7606e20d
10 changed files with 238 additions and 62 deletions
+4 -3
View File
@@ -112,9 +112,10 @@
const q = P.param(prefs.data); // the reader's boundaries (excluded topics, ceilings)
try {
const it = (await getJSON(`/api/brief?limit=1${homeq}${q ? '&' + q : ''}`))?.items?.[0];
// Serve the photo from our own cached/downscaled copy (/api/img/<id>) rather than
// hotlinking the source — keep image_url only as the "has a photo?" signal.
if (it) news = { id: it.id, title: it.title, summary: it.summary || it.description || '', image: it.image_url ? `/api/img/${it.id}` : null, topic: it.topic || null, source_read_minutes: it.source_read_minutes };
// image_url is already resolved server-side per the source's image policy (our
// cached copy for cleared sources, else the publisher URL); the probe below loads
// it with retries either way.
if (it) news = { id: it.id, title: it.title, summary: it.summary || it.description || '', image: it.image_url || null, topic: it.topic || null, source_read_minutes: it.source_read_minutes };
// News images are hotlinked from the source, so a single fetch can transiently
// fail (slow CDN, rate limit, hiccup) and leave a blank plate until you refresh.
// Probe with a couple of retries and reveal the photo only once it's truly loaded