news: close the remaining no-paywall bypass paths (Codex audit)

queries.feed was the main chokepoint, but several discovery paths have their own
SQL. Apply the shared source exclusion to all of them so "no paywalls" is truly
site-wide:
- briefs.build_daily_brief: EXCLUDE paywalled candidates (was: demote) — never
  stored in a new brief.
- queries.brief: stored-brief retrieval (covers /today + /api/brief) filters the
  paywalled source.
- digest.digest_items + followed_digest_items: the morning email + "from what you
  follow" omit paywalled sources.
- sitemap(): paywalled article pages excluded from the sitemap.
All reuse queries.paywalled_source_ids (admin override still wins).

Regression tests (test_paywall_exclusion.py): never stored in a new brief; /today
+ digest omit it; followed-source email omits it; Saved retains it; 'free'
override restores eligibility. 423 backend tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-28 17:22:52 -04:00
parent 0d21231597
commit c600145ba5
5 changed files with 99 additions and 13 deletions
+5 -2
View File
@@ -1698,11 +1698,14 @@ def create_app() -> FastAPI:
@app.get("/sitemap.xml")
def sitemap() -> Response:
with get_conn() as conn:
pwx = queries.paywalled_source_ids(conn)
pw_clause = f" AND a.source_id NOT IN ({','.join('?' * len(pwx))})" if pwx else ""
rows = conn.execute(
"SELECT a.id, COALESCE(a.published_at, a.discovered_at) AS lm "
"FROM articles a JOIN article_scores s ON s.article_id = a.id "
"WHERE s.accepted = 1 AND a.duplicate_of IS NULL "
"ORDER BY lm DESC LIMIT 5000"
"WHERE s.accepted = 1 AND a.duplicate_of IS NULL" + pw_clause + " "
"ORDER BY lm DESC LIMIT 5000",
pwx,
).fetchall()
base = PUBLIC_BASE_URL
urls = [