images: fix two fetcher bugs + add source-level image-rights policy (Codex)
Fetcher (the two remaining bugs Codex found):
- Real redirects are now followed. _NoRedirect makes urllib RAISE HTTPError on 3xx, so
the old status-branch was dead code (mocked tests masked it). Handle 301/302/303/307/308
HTTPError as redirects (re-validate the destination); classify 4xx≠429 as PERMANENT
(negative-cached), 429/5xx/network as transient. Real-opener redirect + 404/5xx tests.
- The megapixel ceiling is now enforced: explicit `w*h > _MAX_PIXELS` check BEFORE load()
(Pillow only warns at MAX_IMAGE_PIXELS). Test with a lowered ceiling.
Image-rights policy (per Codex + owner decision — only cache what's cleared):
- sources.image_policy: 'cache' (re-host a downscaled copy — license/permission/PD only),
'remote' (hotlink the publisher's image — the conservative DEFAULT), 'none' (no image).
- newsimg.display_url resolves the display URL per policy; applied in Article.from_row so
feed/brief/history return the right URL, and in share.py (og/twitter still reference the
publisher's own image, never re-hosted). warm() + /api/img both gated on 'cache'.
- Frontend uses the server-resolved image_url (reverted the hardcoded /api/img); the
graceful retry covers remote hotlinks too. Admin: per-source image-policy selector +
POST /api/admin/sources/{id}/image-policy. Default 'remote' → nothing re-hosted until
a source is explicitly cleared.
445 backend + 36 frontend tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -585,3 +585,21 @@ def test_source_paywall_override(tmp_path, monkeypatch):
|
||||
# validation + 404
|
||||
assert tc.post("/api/admin/sources/2/paywall", json={"override": "bogus"}).status_code == 422
|
||||
assert tc.post("/api/admin/sources/999/paywall", json={"override": "free"}).status_code == 404
|
||||
|
||||
|
||||
def test_source_image_policy(tmp_path, monkeypatch):
|
||||
import sqlite3, os
|
||||
app, api = _make(tmp_path, monkeypatch, admin_email="boss@x.com")
|
||||
c = sqlite3.connect(os.environ["GOODNEWS_DB"])
|
||||
c.execute("INSERT INTO sources (id,name,feed_url) VALUES (2,'Gov','http://g/f')")
|
||||
c.commit(); c.close()
|
||||
anon = TestClient(app)
|
||||
assert anon.post("/api/admin/sources/2/image-policy", json={"policy": "cache"}).status_code == 401 # gated
|
||||
tc = _signin(app, api, "boss@x.com")
|
||||
assert tc.post("/api/admin/sources/2/image-policy", json={"policy": "cache"}).json()["policy"] == "cache"
|
||||
c = sqlite3.connect(os.environ["GOODNEWS_DB"])
|
||||
assert c.execute("SELECT image_policy FROM sources WHERE id=2").fetchone()[0] == "cache"
|
||||
c.close()
|
||||
assert tc.post("/api/admin/sources/2/image-policy", json={"policy": "remote"}).json()["policy"] == "remote"
|
||||
assert tc.post("/api/admin/sources/2/image-policy", json={"policy": "bogus"}).status_code == 422
|
||||
assert tc.post("/api/admin/sources/999/image-policy", json={"policy": "cache"}).status_code == 404
|
||||
|
||||
Reference in New Issue
Block a user