news: hard-exclude paywalled sources from the feed + brief (no unreadable news)

Per Jay: don't surface stories people can't read without paying — it's off-brand
("no paywalls") and pointless. Paywalled is source-level (domain rule, admin-
overridable): just 3 sources today (Nature, New Scientist, MIT Tech Review),
~5.4% of accepted articles.

- queries.paywalled_source_ids(conn): live source set (admin override wins).
- queries.feed gains include_paywalled=False (default) → adds `a.source_id NOT IN
  (…)`. One chokepoint covers Latest/tags/sources/moods/topics/search/since AND
  the brief top-up. Source-level + SQL → paging stays exact, no frontend change.
- brief(): filter the cached/home pool by the same rule; replacement already
  avoids paywalled and now rides the feed exclusion too.
- Dropped the now-moot "paywalled below readable" demotion sort.
- Saved/history keep showing items you saved (their own queries, not excluded).
- test_source_paywall_override updated: paywalled source → excluded from the feed
  (was: shown with a badge); 'free' override → returns, no badge. 418 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-28 17:10:00 -04:00
parent 54761f5083
commit 0d21231597
3 changed files with 35 additions and 14 deletions
+10 -7
View File
@@ -532,22 +532,25 @@ def test_source_paywall_override(tmp_path, monkeypatch):
c.commit(); c.close()
tc = _signin(app, api, "boss@x.com")
def feed_badge():
return next(a for a in tc.get("/api/feed?source_id=2").json()["items"] if a["id"] == 2)["paywalled"]
def in_feed():
return any(a["id"] == 2 for a in tc.get("/api/feed?source_id=2").json()["items"])
# domain rule: nytimes.com → paywalled in table, inspector, AND feed badge (all agree)
# domain rule: nytimes.com → paywalled in the source table + inspector, and HARD-EXCLUDED
# from the public feed (we don't surface stories you can't read for free)
assert _src(tc, 2)["paywalled"] is True
assert tc.get("/api/admin/sources/2/articles").json()["summary"]["paywalled"] is True
assert feed_badge() is True
# override 'free' (the NYT Learning fix) → effective OFF everywhere
assert in_feed() is False
# override 'free' (the NYT Learning fix) → effective OFF: it returns to the feed, no badge
assert tc.post("/api/admin/sources/2/paywall", json={"override": "free"}).json()["override"] == "free"
assert _src(tc, 2)["paywalled"] is False
summ = tc.get("/api/admin/sources/2/articles").json()["summary"]
assert summ["paywalled"] is False and summ["paywall_domain"] is True and summ["paywall_override"] == "free"
assert feed_badge() is False # ranking/badge now agree it's free
# back to domain rule, and the 'paywalled' override
assert in_feed() is True
assert next(a for a in tc.get("/api/feed?source_id=2").json()["items"] if a["id"] == 2)["paywalled"] is False
# back to domain rule → excluded again
assert tc.post("/api/admin/sources/2/paywall", json={"override": None}).json()["override"] is None
assert _src(tc, 2)["paywalled"] is True
assert in_feed() is False
# validation + 404
assert tc.post("/api/admin/sources/2/paywall", json={"override": "bogus"}).status_code == 422
assert tc.post("/api/admin/sources/999/paywall", json={"override": "free"}).status_code == 404