Geo Stage 1-2: subject-geography model + classifier + pipeline wiring

"Closer to Home" foundation (audit greenlit by Codex). Durable geography, kept
decoupled from volatile scoring.

- Schema: article_geo (breadth/confidence/rationale/geo_version) + article_places
  (0..N ISO-coded places), separate from article_scores so re-runs/audits never
  disturb scoring or acceptance. "local" is never stored — it's relative to the
  reader; the UI computes "Near you" later.
- geo.py: LLM proposes place NAMES, code disposes to ISO codes (country alpha-2,
  US state 2-letter); region words like "Europe" can never become a country.
  'global'/placeless is first-class, not failure. Confidence calibrated so 'high'
  needs an explicit location. Geo is its OWN LLM pass, not merged into the scoring
  prompt (durable metadata, re-runnable, keeps the sensitive prompt untouched).
- store_geo replaces places (geo is re-derivable, unlike scores). tag_articles is
  idempotent by geo_version, only touches accepted non-duplicate articles.
- CLI `geo` command (cycle-locked, --limit/--reclassify) for backfill, plus a
  bounded geo step in the cycle (--geo-limit 60, --no-geo). scripts/geo_audit.py
  is the prototype audit tool.

360 tests green; live smoke tagged real articles correctly (Gaza->PS, London->GB,
placeless science->global). No UI / SEO pages yet — ranking/personalization only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jay
2026-06-19 16:56:49 -04:00
parent 59ff48ae90
commit 1c05554a28
7 changed files with 613 additions and 1 deletions
+28
View File
@@ -12,6 +12,7 @@ from .digest import send_due_digests
from .games import generate_daily_puzzles
from .localtime import local_today
from .dedup import DEFAULT_THRESHOLD, DEFAULT_WINDOW_DAYS, cluster_duplicates, dedup as run_dedup
from .geo import tag_articles as tag_geo
from .enrich import enrich_brief_images, enrich_recent_images, enrich_summarized_images
from .summarize import generate_summary, get_summary
from .feeds import (
@@ -132,6 +133,8 @@ def main() -> None:
cycle_parser.add_argument("--classify-limit", type=int, default=40)
cycle_parser.add_argument("--no-classify", action="store_true", help="Skip the LLM classify step")
cycle_parser.add_argument("--no-dedup", action="store_true", help="Skip the embedding dedup step")
cycle_parser.add_argument("--no-geo", action="store_true", help="Skip tagging article subject-geography")
cycle_parser.add_argument("--geo-limit", type=int, default=60, help="Max articles to geo-tag per cycle")
cycle_parser.add_argument("--no-brief", action="store_true", help="Skip rebuilding today's brief")
cycle_parser.add_argument("--no-review", action="store_true", help="Skip recomputing source review flags")
cycle_parser.add_argument("--no-digest", action="store_true", help="Skip sending due daily digests")
@@ -147,6 +150,12 @@ def main() -> None:
)
enrich_images_parser.add_argument("--limit", type=int, default=50, help="Max articles to fetch this batch")
geo_parser = subparsers.add_parser("geo", help="Tag article subject-geography (backfill / manual). Cycle-locked.")
geo_parser.add_argument("--limit", type=int, default=200, help="Max articles to tag this batch")
geo_parser.add_argument("--reclassify", action="store_true", help="Re-tag even rows already at the current geo version")
geo_parser.add_argument("--base-url", help="OpenAI-compatible base URL")
geo_parser.add_argument("--model", help="Local model name")
dedup_parser = subparsers.add_parser("dedup", help="Cluster near-duplicate stories via local embeddings")
dedup_parser.add_argument("--threshold", type=float, default=DEFAULT_THRESHOLD, help="Cosine similarity cutoff")
dedup_parser.add_argument("--window-days", type=int, default=DEFAULT_WINDOW_DAYS)
@@ -298,6 +307,15 @@ def main() -> None:
elif args.command == "enrich-images":
found = enrich_summarized_images(conn, limit=args.limit)
print(f"enrich-images: {found} new image(s) for summarized articles")
elif args.command == "geo":
init_db(conn)
# Cycle-locked so a manual backfill can't contend with the scheduled cycle.
with cycle_lock(args.db) as acquired:
if not acquired:
print("geo: a cycle is already running; try again after it finishes")
return
g = tag_geo(conn, llm_client_from_args(args), limit=args.limit, reclassify=args.reclassify)
print(f"geo: tagged={g['tagged']} errors={g['errors']} (of {g['candidates']} candidates)")
elif args.command == "dedup":
init_db(conn)
if args.force_recluster:
@@ -506,6 +524,16 @@ def _run_cycle_locked(conn: sqlite3.Connection, args: argparse.Namespace) -> Non
except Exception as exc:
print(f"dedup: skipped ({exc})")
# Geo: tag newly-accepted, non-duplicate articles with subject geography (its own
# LLM pass, decoupled from scoring). Bounded per cycle; idempotent (skips rows
# already at the current GEO_VERSION). Non-fatal like every other step.
if not args.no_geo:
try:
g = tag_geo(conn, llm_client_from_args(args), limit=args.geo_limit)
print(f"geo: tagged={g['tagged']} errors={g['errors']} (of {g['candidates']} untagged)")
except Exception as exc:
print(f"geo: skipped ({exc})")
if not args.no_brief:
today = local_today()
try: