Reject branded/generic share images; hero prefers a clean illustrated story

- og:image enrichment now skips branded/generic share images (BBC 'branded_news'
  with its burned-in logo, NPR 'facebook-default', etc.) and keeps the first
  real article image — so no competitor logo lands on our hero. Cleared the few
  already-stored branded URLs so they re-enrich.
- Hero selection now prefers a gentle + readable story that also HAS a (clean)
  image, falling back to gentle-readable, then gentle. The lead is visual when
  possible, typographic otherwise — never branded.

(The '7 cards' report was a stale browser cache: the brief stores 7 and the
built JS requests 7; a hard refresh shows all seven.)

Tests: branded/generic image rejection (87 total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-31 13:03:20 +00:00
parent d8d665ee35
commit 7e1dfd5b3c
3 changed files with 48 additions and 6 deletions
+9 -4
View File
@@ -58,13 +58,18 @@ def get_conn():
def _pick_lead(items: list[dict]) -> list[dict]:
"""Lead with a gentle, readable story when possible.
"""Lead with a gentle, readable, ideally illustrated story.
Tries gentle-and-readable first, then gentle, then leaves the order alone.
Charged or paywalled stories still appear in the set — they just don't lead.
Preference order: gentle + readable + has an image, then gentle + readable,
then gentle, then leave the order alone. Charged/paywalled/imageless stories
still appear in the set — they just don't lead.
"""
def gentle(a: dict) -> bool:
return safe_to_lead(a) and not is_paywalled(a.get("canonical_url"))
for ok in (
lambda a: safe_to_lead(a) and not is_paywalled(a.get("canonical_url")),
lambda a: gentle(a) and bool(a.get("image_url")),
gentle,
safe_to_lead,
):
for i, a in enumerate(items):
+28 -2
View File
@@ -34,6 +34,28 @@ MAX_REDIRECTS = 3
_META_RE = re.compile(rb"<meta\b[^>]*>", re.IGNORECASE)
_HEAD_END_RE = re.compile(rb"</head>", re.IGNORECASE)
# Substrings that mark a branded/generic share image rather than the article's
# own picture (e.g. BBC burns "BBC NEWS" into branded_news; NPR uses a default).
# We'd rather show no image (typographic hero) than a competitor logo.
_GENERIC_IMAGE_MARKERS = (
"branded_news",
"facebook-default",
"default-wide",
"default-fb",
"og-default",
"default-og",
"twitter-default",
"default-image",
"/placeholder",
"share-default",
"social-default",
)
def _is_generic_image(url: str) -> bool:
lowered = url.lower()
return any(marker in lowered for marker in _GENERIC_IMAGE_MARKERS)
def _attr(tag: bytes, name: bytes) -> bytes | None:
m = re.search(name + rb"""\s*=\s*["']([^"']*)["']""", tag, re.IGNORECASE)
@@ -47,8 +69,12 @@ def og_image_from_html(html: bytes) -> str | None:
key = _attr(tag, b"property") or _attr(tag, b"name")
if key and key.lower() in (b"og:image", b"og:image:url", b"twitter:image"):
content = _attr(tag, b"content")
if content:
return canonicalize_url(content.decode("utf-8", "replace"))
if not content:
continue
image = canonicalize_url(content.decode("utf-8", "replace"))
# Skip branded/generic share images; keep scanning for a real one.
if image and not _is_generic_image(image):
return image
return None