Advisory source health: review flags, never auto-deactivate

- Add source health columns (last_success_at, last_error_at, last_error,
  consecutive_failures, review_flag, review_reason) via SCHEMA + migration.
- poll_source maintains them: success resets the failure streak and records the
  success time; failure increments it and stores the latest error.
- review_sources() flags active sources that are stale, repeatedly failing,
  low-acceptance, duplicate-heavy, or doom-skewed (high cortisol/ragebait) over
  a recent window. It is purely advisory: it sets review_flag/review_reason and
  never changes the active column (human stays in the loop), clearing the flag
  when a source recovers.
- CLI review-sources; cycle runs it as a final step (--no-review to skip);
  source-report shows a review line for flagged feeds.
- Tests: healthy/failing/stale/low-acceptance/recovery and never-deactivates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-30 20:28:18 +00:00
parent aa4125ddec
commit 1e190c5e88
6 changed files with 243 additions and 4 deletions
+20
View File
@@ -108,6 +108,15 @@ def poll_source(conn: sqlite3.Connection, source: sqlite3.Row) -> dict:
""",
(seen, inserted, duplicate, run_id),
)
# A clean poll resets the failure streak and records the success time.
conn.execute(
"""
UPDATE sources
SET last_success_at = CURRENT_TIMESTAMP, consecutive_failures = 0
WHERE id = ?
""",
(source["id"],),
)
conn.commit()
return {"status": "ok", "seen": seen, "inserted": inserted, "duplicate": duplicate}
except Exception as exc:
@@ -124,6 +133,17 @@ def poll_source(conn: sqlite3.Connection, source: sqlite3.Row) -> dict:
""",
(seen, inserted, duplicate, str(exc), run_id),
)
# Track the failure streak and latest error for advisory review flags.
conn.execute(
"""
UPDATE sources
SET consecutive_failures = consecutive_failures + 1,
last_error_at = CURRENT_TIMESTAMP,
last_error = ?
WHERE id = ?
""",
(str(exc), source["id"]),
)
conn.commit()
return {
"status": "failed",