Sources: per-source paywall override (3-state) — fix domain-rule mis-flags
The Articles inspector revealed paywall is domain-coarse: nytimes.com is flagged, so NY Times Learning's free Word-of-the-Day inherits 🔒 — and that flag isn't cosmetic, it deprioritizes the content in feed sort + lead selection. Add a per-source override so admins can correct it after inspecting. - sources.paywall_override: NULL (domain rule) | 'free' | 'paywalled'. - paywall.py: keep low-level is_paywalled(url) (domain); add is_paywalled_for_source (url, override) for the EFFECTIVE decision — never patched the domain helper globally (per Codex), so "domain says X" stays distinguishable from "overridden". - Threaded everywhere ranking/UI touches paywall, via src.paywall_override on the shared _ARTICLE_COLUMNS + the source-aware helper: feed sort, /api/since, replace, lead selection, Article badge, brief composition (briefs.py), digest, source_health (table 🔒), the Articles inspector, and the review/attention check — so ranking and UI always agree. - Endpoint POST /api/admin/sources/{id}/paywall {override}; admin UI: a select in the inspector header (Use domain rule / Treat as free / Treat as paywalled) + the basis ("ON (domain)" / "OFF (override)"), optimistic so the panel stays open. Test: domain rule → paywalled in table+inspector+feed badge; 'free' → off in all three; validation 422 + 404. 242 pytest + 11 vitest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+5
-5
@@ -16,7 +16,7 @@ from html import escape
|
||||
|
||||
from . import email_send
|
||||
from .localtime import local_now, local_today
|
||||
from .paywall import is_paywalled
|
||||
from .paywall import is_paywalled, is_paywalled_for_source
|
||||
|
||||
DIGEST_HOUR = int(os.environ.get("GOODNEWS_DIGEST_HOUR", "7"))
|
||||
DIGEST_WINDOW_HOURS = 4 # send between DIGEST_HOUR and +4h, site-local
|
||||
@@ -31,7 +31,7 @@ def digest_items(conn: sqlite3.Connection, brief_date: str, limit: int = 7) -> l
|
||||
"""The brief's items with the bits a calm email needs (visible sources only)."""
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, sc.reason_text,
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, s.paywall_override, sc.reason_text,
|
||||
(SELECT summary FROM article_summaries WHERE article_id = a.id) AS summary
|
||||
FROM daily_briefs b
|
||||
JOIN daily_brief_items bi ON bi.brief_id = b.id
|
||||
@@ -47,7 +47,7 @@ def digest_items(conn: sqlite3.Connection, brief_date: str, limit: int = 7) -> l
|
||||
items = []
|
||||
for r in rows:
|
||||
d = dict(r)
|
||||
d["paywalled"] = is_paywalled(d["canonical_url"])
|
||||
d["paywalled"] = is_paywalled_for_source(d["canonical_url"], d.get("paywall_override"))
|
||||
items.append(d)
|
||||
return items
|
||||
|
||||
@@ -74,7 +74,7 @@ def followed_digest_items(conn: sqlite3.Connection, user_id: int, exclude_ids, l
|
||||
params += ftags
|
||||
rows = conn.execute(
|
||||
f"""
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, a.source_id, sc.reason_text,
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, s.paywall_override, a.source_id, sc.reason_text,
|
||||
(SELECT summary FROM article_summaries WHERE article_id = a.id) AS summary
|
||||
FROM articles a
|
||||
JOIN sources s ON s.id = a.source_id
|
||||
@@ -92,7 +92,7 @@ def followed_digest_items(conn: sqlite3.Connection, user_id: int, exclude_ids, l
|
||||
if d["id"] in exclude or per_source.get(d["source_id"], 0) >= 1:
|
||||
continue
|
||||
per_source[d["source_id"]] = 1
|
||||
d["paywalled"] = is_paywalled(d["canonical_url"])
|
||||
d["paywalled"] = is_paywalled_for_source(d["canonical_url"], d.get("paywall_override"))
|
||||
out.append(d)
|
||||
if len(out) >= limit:
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user