Honor Retry-After on HTTP 429 (polite rest, not a failure)

Per Codex's spec — a publisher saying "slow down" shouldn't make a feed look
broken, but repeated 429s stay visible via last_success_at / stale-source.

* Schema: sources.retry_after_at (nullable) + migration.
* feeds.parse_retry_after: delta-seconds OR HTTP-date → UTC stamp; ignores
  invalid/negative/past; caps at now + MAX_BACKOFF_MINUTES.
* fetch_feed raises RateLimited (carrying the parsed time) on a 429.
* poll_source: on 429 set retry_after_at + last_error, status='rate_limited',
  and do NOT increment consecutive_failures; on success clear retry_after_at;
  non-429 failures unchanged.
* due_source_rows requires BOTH the streak backoff elapsed AND retry_after_at
  passed (i.e. the later of the two).
* Admin: source_health returns retry_after_at; status reads
  "rate-limited · rests until …" rather than "failed/resting".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-09 10:47:40 -04:00
parent 373571b476
commit 38abc26ddd
5 changed files with 146 additions and 4 deletions
+3
View File
@@ -25,6 +25,7 @@ CREATE TABLE IF NOT EXISTS sources (
last_error_at TEXT,
last_error TEXT,
consecutive_failures INTEGER NOT NULL DEFAULT 0,
retry_after_at TEXT,
review_flag INTEGER NOT NULL DEFAULT 0,
review_reason TEXT,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
@@ -326,6 +327,8 @@ def _migrate(conn: sqlite3.Connection) -> None:
conn.execute("UPDATE sources SET status = CASE WHEN active = 1 THEN 'active' ELSE 'paused' END")
if "content_visible" not in source_cols:
conn.execute("ALTER TABLE sources ADD COLUMN content_visible INTEGER NOT NULL DEFAULT 1")
if "retry_after_at" not in source_cols:
conn.execute("ALTER TABLE sources ADD COLUMN retry_after_at TEXT")
# feedback.read_at (admin inbox read/unread) added later.
fb_cols = {row["name"] for row in conn.execute("PRAGMA table_info(feedback)")}