images: fix two fetcher bugs + add source-level image-rights policy (Codex)
Fetcher (the two remaining bugs Codex found):
- Real redirects are now followed. _NoRedirect makes urllib RAISE HTTPError on 3xx, so
the old status-branch was dead code (mocked tests masked it). Handle 301/302/303/307/308
HTTPError as redirects (re-validate the destination); classify 4xx≠429 as PERMANENT
(negative-cached), 429/5xx/network as transient. Real-opener redirect + 404/5xx tests.
- The megapixel ceiling is now enforced: explicit `w*h > _MAX_PIXELS` check BEFORE load()
(Pillow only warns at MAX_IMAGE_PIXELS). Test with a lowered ceiling.
Image-rights policy (per Codex + owner decision — only cache what's cleared):
- sources.image_policy: 'cache' (re-host a downscaled copy — license/permission/PD only),
'remote' (hotlink the publisher's image — the conservative DEFAULT), 'none' (no image).
- newsimg.display_url resolves the display URL per policy; applied in Article.from_row so
feed/brief/history return the right URL, and in share.py (og/twitter still reference the
publisher's own image, never re-hosted). warm() + /api/img both gated on 'cache'.
- Frontend uses the server-resolved image_url (reverted the hardcoded /api/img); the
graceful retry covers remote hotlinks too. Admin: per-source image-policy selector +
POST /api/admin/sources/{id}/image-policy. Default 'remote' → nothing re-hosted until
a source is explicitly cleared.
445 backend + 36 frontend tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+16
-10
@@ -10,6 +10,8 @@ from __future__ import annotations
|
||||
|
||||
from html import escape
|
||||
|
||||
from .newsimg import display_url
|
||||
|
||||
|
||||
def _tag(name: str, content: str | None, attr: str = "property") -> str:
|
||||
if not content:
|
||||
@@ -149,11 +151,17 @@ def render_share_page(article: dict, base_url: str, summary: str | None = None,
|
||||
)
|
||||
src_url = article.get("canonical_url") or base_url
|
||||
image = article.get("image_url")
|
||||
policy = article.get("image_policy")
|
||||
# What WE show, honoring the source's image policy (cache → our copy; remote → the
|
||||
# publisher's URL; none → nothing). og/twitter reference the publisher's own image
|
||||
# (a link, not re-hosting) whenever we'd show anything; 'none' omits it entirely.
|
||||
display = display_url(aid, policy, image)
|
||||
og_image = image if (image and policy != "none") else None
|
||||
page_url = f"{base_url}/a/{aid}"
|
||||
# With an image: a large-image card. Without: a clean text unfurl (title +
|
||||
# why + brand) — tidy, not a broken/muddy preview. (A branded fallback PNG
|
||||
# can replace this later.)
|
||||
twitter_card = "summary_large_image" if image else "summary"
|
||||
twitter_card = "summary_large_image" if display else "summary"
|
||||
|
||||
meta = "\n".join(filter(None, [
|
||||
_tag("og:site_name", "upbeatBytes"),
|
||||
@@ -161,22 +169,20 @@ def render_share_page(article: dict, base_url: str, summary: str | None = None,
|
||||
_tag("og:title", title),
|
||||
_tag("og:description", why),
|
||||
_tag("og:url", page_url),
|
||||
_tag("og:image", image),
|
||||
_tag("og:image", og_image),
|
||||
_tag("twitter:card", twitter_card, attr="name"),
|
||||
_tag("twitter:title", title, attr="name"),
|
||||
_tag("twitter:description", why, attr="name"),
|
||||
_tag("twitter:image", image, attr="name"),
|
||||
_tag("twitter:image", og_image, attr="name"),
|
||||
]))
|
||||
|
||||
# The visible image is served from our cached/downscaled copy (not a hotlink), so a
|
||||
# flaky source CDN can't blank it. og:image/twitter:image above stay the source URL
|
||||
# so social crawlers fetch the canonical image directly.
|
||||
# Served from our cache (/api/img/<id>); if it isn't cached yet / fails, drop the
|
||||
# element so the page degrades to the clean text unfurl rather than a broken icon.
|
||||
# The visible image is whatever the policy resolved to (our cached copy for 'cache'
|
||||
# sources, else the publisher's URL for 'remote'). If it isn't cached yet / fails to
|
||||
# load, drop the element so the page degrades to the clean text unfurl, not a broken icon.
|
||||
media = (
|
||||
f'<img class="media" src="/api/img/{aid}" alt="" referrerpolicy="no-referrer" '
|
||||
f'<img class="media" src="{escape(display)}" alt="" referrerpolicy="no-referrer" '
|
||||
f'onerror="this.remove()">'
|
||||
if image else ""
|
||||
if display else ""
|
||||
)
|
||||
|
||||
raw_tags = (article.get("tags") or "")
|
||||
|
||||
Reference in New Issue
Block a user