news: close the remaining no-paywall bypass paths (Codex audit)
queries.feed was the main chokepoint, but several discovery paths have their own SQL. Apply the shared source exclusion to all of them so "no paywalls" is truly site-wide: - briefs.build_daily_brief: EXCLUDE paywalled candidates (was: demote) — never stored in a new brief. - queries.brief: stored-brief retrieval (covers /today + /api/brief) filters the paywalled source. - digest.digest_items + followed_digest_items: the morning email + "from what you follow" omit paywalled sources. - sitemap(): paywalled article pages excluded from the sitemap. All reuse queries.paywalled_source_ids (admin override still wins). Regression tests (test_paywall_exclusion.py): never stored in a new brief; /today + digest omit it; followed-source email omits it; Saved retains it; 'free' override restores eligibility. 423 backend tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+10
-5
@@ -17,6 +17,7 @@ from html import escape
|
||||
from . import email_send
|
||||
from .localtime import local_now, local_today
|
||||
from .paywall import is_paywalled, is_paywalled_for_source
|
||||
from .queries import paywalled_source_ids
|
||||
|
||||
DIGEST_HOUR = int(os.environ.get("GOODNEWS_DIGEST_HOUR", "7"))
|
||||
DIGEST_WINDOW_HOURS = 4 # send between DIGEST_HOUR and +4h, site-local
|
||||
@@ -29,8 +30,10 @@ def _base_url() -> str:
|
||||
|
||||
def digest_items(conn: sqlite3.Connection, brief_date: str, limit: int = 7) -> list[dict]:
|
||||
"""The brief's items with the bits a calm email needs (visible sources only)."""
|
||||
pwx = paywalled_source_ids(conn)
|
||||
pw_clause = f" AND a.source_id NOT IN ({','.join('?' * len(pwx))})" if pwx else ""
|
||||
rows = conn.execute(
|
||||
"""
|
||||
f"""
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, s.paywall_override, sc.reason_text,
|
||||
(SELECT summary FROM article_summaries WHERE article_id = a.id) AS summary
|
||||
FROM daily_briefs b
|
||||
@@ -38,11 +41,11 @@ def digest_items(conn: sqlite3.Connection, brief_date: str, limit: int = 7) -> l
|
||||
JOIN articles a ON a.id = bi.article_id
|
||||
JOIN sources s ON s.id = a.source_id
|
||||
LEFT JOIN article_scores sc ON sc.article_id = a.id
|
||||
WHERE b.brief_date = ? AND s.content_visible = 1
|
||||
WHERE b.brief_date = ? AND s.content_visible = 1{pw_clause}
|
||||
ORDER BY bi.rank
|
||||
LIMIT ?
|
||||
""",
|
||||
(brief_date, limit),
|
||||
(brief_date, *pwx, limit),
|
||||
).fetchall()
|
||||
items = []
|
||||
for r in rows:
|
||||
@@ -72,6 +75,8 @@ def followed_digest_items(conn: sqlite3.Connection, user_id: int, exclude_ids, l
|
||||
f"AND at.tag IN ({','.join('?' * len(ftags))}))"
|
||||
)
|
||||
params += ftags
|
||||
pwx = paywalled_source_ids(conn)
|
||||
pw_clause = f" AND a.source_id NOT IN ({','.join('?' * len(pwx))})" if pwx else ""
|
||||
rows = conn.execute(
|
||||
f"""
|
||||
SELECT a.id, a.title, a.canonical_url, s.name AS source, s.paywall_override, a.source_id, sc.reason_text,
|
||||
@@ -79,12 +84,12 @@ def followed_digest_items(conn: sqlite3.Connection, user_id: int, exclude_ids, l
|
||||
FROM articles a
|
||||
JOIN sources s ON s.id = a.source_id
|
||||
JOIN article_scores sc ON sc.article_id = a.id
|
||||
WHERE sc.accepted = 1 AND a.duplicate_of IS NULL AND s.content_visible = 1
|
||||
WHERE sc.accepted = 1 AND a.duplicate_of IS NULL AND s.content_visible = 1{pw_clause}
|
||||
AND ({' OR '.join(ors)})
|
||||
ORDER BY COALESCE(a.published_at, a.discovered_at) DESC
|
||||
LIMIT 30
|
||||
""",
|
||||
params,
|
||||
[*pwx, *params],
|
||||
).fetchall()
|
||||
exclude, per_source, out = set(exclude_ids), {}, []
|
||||
for r in rows:
|
||||
|
||||
Reference in New Issue
Block a user