Advisory source health: review flags, never auto-deactivate

- Add source health columns (last_success_at, last_error_at, last_error,
  consecutive_failures, review_flag, review_reason) via SCHEMA + migration.
- poll_source maintains them: success resets the failure streak and records the
  success time; failure increments it and stores the latest error.
- review_sources() flags active sources that are stale, repeatedly failing,
  low-acceptance, duplicate-heavy, or doom-skewed (high cortisol/ragebait) over
  a recent window. It is purely advisory: it sets review_flag/review_reason and
  never changes the active column (human stays in the loop), clearing the flag
  when a source recovers.
- CLI review-sources; cycle runs it as a final step (--no-review to skip);
  source-report shows a review line for flagged feeds.
- Tests: healthy/failing/stale/low-acceptance/recovery and never-deactivates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jay
2026-05-30 20:28:18 +00:00
parent aa4125ddec
commit 1e190c5e88
6 changed files with 243 additions and 4 deletions
+19
View File
@@ -19,6 +19,12 @@ CREATE TABLE IF NOT EXISTS sources (
active INTEGER NOT NULL DEFAULT 1,
poll_interval_minutes INTEGER NOT NULL DEFAULT 60,
notes TEXT,
last_success_at TEXT,
last_error_at TEXT,
last_error TEXT,
consecutive_failures INTEGER NOT NULL DEFAULT 0,
review_flag INTEGER NOT NULL DEFAULT 0,
review_reason TEXT,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
);
@@ -148,3 +154,16 @@ def _migrate(conn: sqlite3.Connection) -> None:
)
# Created here (not in SCHEMA) so it runs after the column exists on upgrades.
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_duplicate_of ON articles(duplicate_of)")
source_cols = {row["name"] for row in conn.execute("PRAGMA table_info(sources)")}
health_columns = {
"last_success_at": "TEXT",
"last_error_at": "TEXT",
"last_error": "TEXT",
"consecutive_failures": "INTEGER NOT NULL DEFAULT 0",
"review_flag": "INTEGER NOT NULL DEFAULT 0",
"review_reason": "TEXT",
}
for column, decl in health_columns.items():
if column not in source_cols:
conn.execute(f"ALTER TABLE sources ADD COLUMN {column} {decl}")