Typographic-first imagery + opportunistic feed-HTML image extraction

Per the calm north star (images support reading, never become a stimulation
layer; metadata-only stays the posture):
- Image-less cards are now designed, not missing: secondary cards are text-first
  (no empty media band), and an image-less hero becomes a fully typographic lead
  with a faint topic wordmark behind it (CSS attr(data-topic)). No big empty
  image space is ever reserved.
- Opportunistic extraction: parse the first <img src> from a feed's
  content/description HTML when present, canonicalized — never fetching the
  article page. Applies to new ingests (existing rows keep their current image).
- Held by deliberate choice: og:image page enrichment, stock/AI imagery, and any
  image-coverage requirement for sources.

Tests: feed HTML image extraction (72 total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-30 23:59:36 +00:00
parent b9ecebffde
commit f46fee1197
3 changed files with 89 additions and 31 deletions
+24 -2
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import email.utils
import re
import sqlite3
import urllib.error
import urllib.request
@@ -361,7 +362,7 @@ def _parse_rss(root: ET.Element) -> list[FeedItem]:
description=_first_text(item, "description", "summary", "encoded"),
author=_first_text(item, "author", "creator"),
published_at=_parse_date(_first_text(item, "pubDate", "published", "updated", "date")),
image_url=_find_image_url(item),
image_url=_find_image_url(item) or _html_image(item),
language=language,
raw_guid=guid,
)
@@ -389,7 +390,7 @@ def _parse_atom(root: ET.Element) -> list[FeedItem]:
description=_first_text(entry, "summary", "content"),
author=author,
published_at=_parse_date(_first_text(entry, "published", "updated")),
image_url=_find_image_url(entry),
image_url=_find_image_url(entry) or _html_image(entry),
language=language,
raw_guid=_first_text(entry, "id"),
)
@@ -411,6 +412,27 @@ def _atom_link(entry: ET.Element) -> str | None:
return fallback
_IMG_SRC_RE = re.compile(r"""<img\b[^>]*?\bsrc=["']([^"']+)["']""", re.IGNORECASE)
def _img_from_html(html: str | None) -> str | None:
"""First <img src> in a content/description HTML blob, if any."""
if not html:
return None
match = _IMG_SRC_RE.search(html)
return match.group(1) if match else None
def _html_image(element: ET.Element) -> str | None:
"""Opportunistic image from the feed's content/description HTML.
Only ever reads what the feed already provides — never fetches the article
page. A non-http(s)/relative URL is dropped by canonicalize_url.
"""
html = _first_text(element, "encoded", "content", "description", "summary")
return canonicalize_url(_img_from_html(html))
def _find_image_url(element: ET.Element) -> str | None:
for child in element.iter():
name = _local_name(child.tag)