Hardening pass: scheduler backoff, FK cascade, a11y, test safety net

Pre-traffic cleanup from an audit:

* Scheduler: poll_due_sources now keys on the last *attempt* (success or
  failure), not the last success, and scales the wait by the consecutive-
  failure streak (capped at a day). A failing feed (e.g. Phys.org's HTTP 429s)
  used to be retried every cycle because it had no successful run; it now backs
  off and recovers on its own. Extracted due_source_rows() + tests.

* FK hygiene: deleting a daily_brief is supposed to cascade to its items, but
  SQLite enforces foreign keys per-connection — connect() already sets the
  pragma, so the cascade is correct going forward; added a regression test.
  (Orphaned items + Phys.org settings were cleaned directly on the live DB.)

* a11y: modal/drawer dialogs are now focusable (tabindex), close on Escape
  (window) and on backdrop click via a target check (dropping the inner
  stopPropagation handlers). Build is warning-free.

* tests: conftest points any un-mocked LLM client at a closed port with a 1s
  timeout, so an accidental real call fails fast instead of hanging the suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-06 19:18:18 +00:00
parent 978edc8f4a
commit 452e5a3fe4
8 changed files with 158 additions and 20 deletions
+32 -11
View File
@@ -35,14 +35,32 @@ def poll_all_sources(conn: sqlite3.Connection, limit: int | None = None) -> dict
).fetchall(), limit)
def poll_due_sources(conn: sqlite3.Connection, limit: int | None = None) -> dict:
"""Poll only active sources whose last successful poll is older than their
poll_interval_minutes (or that have never been polled successfully).
# A failing source backs off so it isn't re-hit every scheduler cycle: the
# effective wait grows with the failure streak, capped at a day. This keeps a
# rate-limited feed (HTTP 429) resting instead of hammered, and lets it recover
# on its own once the limit lifts.
MAX_BACKOFF_MINUTES = 1440
This is what makes poll_interval_minutes meaningful and lets a scheduler run
frequently without re-hitting feeds that are not yet due.
def poll_due_sources(conn: sqlite3.Connection, limit: int | None = None) -> dict:
"""Poll only active sources whose last *attempt* (success OR failure) is
older than their effective interval, or that have never been polled.
Keying on the last attempt — not the last success — is what stops a
perpetually-failing feed from being retried every cycle. The effective
interval is poll_interval_minutes scaled up by the consecutive-failure
streak (capped at MAX_BACKOFF_MINUTES), so healthy feeds keep their cadence
while broken ones step down to occasional retries.
"""
rows = conn.execute(
return _poll_rows(conn, due_source_rows(conn), limit)
def due_source_rows(conn: sqlite3.Connection) -> list[sqlite3.Row]:
"""Active sources currently due to poll (see poll_due_sources for the rule).
Split out so the due/backoff decision can be tested without the network.
"""
return conn.execute(
"""
SELECT s.*
FROM sources s
@@ -50,17 +68,20 @@ def poll_due_sources(conn: sqlite3.Connection, limit: int | None = None) -> dict
AND (
NOT EXISTS (
SELECT 1 FROM ingest_runs r
WHERE r.source_id = s.id AND r.status = 'ok'
WHERE r.source_id = s.id AND r.finished_at IS NOT NULL
)
OR (
SELECT MAX(r.finished_at) FROM ingest_runs r
WHERE r.source_id = s.id AND r.status = 'ok'
) <= datetime('now', '-' || s.poll_interval_minutes || ' minutes')
WHERE r.source_id = s.id AND r.finished_at IS NOT NULL
) <= datetime(
'now',
'-' || MIN(?, s.poll_interval_minutes * (1 + s.consecutive_failures)) || ' minutes'
)
)
ORDER BY s.id
"""
""",
(MAX_BACKOFF_MINUTES,),
).fetchall()
return _poll_rows(conn, rows, limit)
def _poll_rows(conn: sqlite3.Connection, rows: list[sqlite3.Row], limit: int | None) -> dict: