Advisory source health: review flags, never auto-deactivate
- Add source health columns (last_success_at, last_error_at, last_error, consecutive_failures, review_flag, review_reason) via SCHEMA + migration. - poll_source maintains them: success resets the failure streak and records the success time; failure increments it and stores the latest error. - review_sources() flags active sources that are stale, repeatedly failing, low-acceptance, duplicate-heavy, or doom-skewed (high cortisol/ragebait) over a recent window. It is purely advisory: it sets review_flag/review_reason and never changes the active column (human stays in the loop), clearing the flag when a source recovers. - CLI review-sources; cycle runs it as a final step (--no-review to skip); source-report shows a review line for flagged feeds. - Tests: healthy/failing/stale/low-acceptance/recovery and never-deactivates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -108,6 +108,15 @@ def poll_source(conn: sqlite3.Connection, source: sqlite3.Row) -> dict:
|
||||
""",
|
||||
(seen, inserted, duplicate, run_id),
|
||||
)
|
||||
# A clean poll resets the failure streak and records the success time.
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE sources
|
||||
SET last_success_at = CURRENT_TIMESTAMP, consecutive_failures = 0
|
||||
WHERE id = ?
|
||||
""",
|
||||
(source["id"],),
|
||||
)
|
||||
conn.commit()
|
||||
return {"status": "ok", "seen": seen, "inserted": inserted, "duplicate": duplicate}
|
||||
except Exception as exc:
|
||||
@@ -124,6 +133,17 @@ def poll_source(conn: sqlite3.Connection, source: sqlite3.Row) -> dict:
|
||||
""",
|
||||
(seen, inserted, duplicate, str(exc), run_id),
|
||||
)
|
||||
# Track the failure streak and latest error for advisory review flags.
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE sources
|
||||
SET consecutive_failures = consecutive_failures + 1,
|
||||
last_error_at = CURRENT_TIMESTAMP,
|
||||
last_error = ?
|
||||
WHERE id = ?
|
||||
""",
|
||||
(str(exc), source["id"]),
|
||||
)
|
||||
conn.commit()
|
||||
return {
|
||||
"status": "failed",
|
||||
|
||||
Reference in New Issue
Block a user