Paywall awareness (#6) + replace-an-article (#7)

- paywall.py: conservative domain-level paywall detection (New Scientist,
  Nature, and common hard/soft paywalls). Never fetches pages — an honest hint.
- API: Article gains a 'paywalled' flag; the brief now leads with a gentle AND
  readable story (paywalled/charged stories stay in the five, just not first).
- New GET /api/replacement returns the next-best readable, unshown article
  (honors mood+prefs via the merged prefs param; gentle=true for hero swaps).
- UI: paywalled cards show 'May need a subscription'; a Replace / 'Find one I
  can read' action (always visible, while tuning actions stay tucked) swaps the
  card for a readable alternative, with a gentle notice when none remain.
- Tests: paywall detection + replacement behavior (77 total).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-31 00:39:13 +00:00
parent 06c2704ae0
commit bfd612eb9b
7 changed files with 226 additions and 16 deletions
+41
View File
@@ -0,0 +1,41 @@
"""Domain-level paywall hints.
We never fetch article pages, so a paywall can only be inferred from the host.
This is a curated, conservative list of hard/soft paywalls — enough to label a
card "subscription may be required" and to prefer readable stories for the lead
and for replacements. It will never be perfect; it's an honest hint, not a gate.
"""
from __future__ import annotations
from urllib.parse import urlsplit
# Host suffixes considered paywalled. Subdomains match (news.nature.com → nature.com).
PAYWALL_DOMAINS = {
"newscientist.com",
"nature.com",
"nytimes.com",
"wsj.com",
"ft.com",
"economist.com",
"wired.com",
"theatlantic.com",
"washingtonpost.com",
"bloomberg.com",
"technologyreview.com",
"newyorker.com",
"scientificamerican.com",
"nationalgeographic.com",
"thetimes.co.uk",
"telegraph.co.uk",
"foreignpolicy.com",
"hbr.org",
"harpers.org",
}
def is_paywalled(url: str | None) -> bool:
host = urlsplit(url or "").netloc.lower()
if host.startswith("www."):
host = host[4:]
return any(host == d or host.endswith("." + d) for d in PAYWALL_DOMAINS)