Durability pass: tests, clearer diversity/classify behavior, Calm Filters foundation
- Add pytest suite (34 tests) covering scoring thresholds, dedup clustering + representative selection + time window, brief source/category diversity, avoid-term phrase matching, and text canonicalization/truncation. - Rewrite _select_diverse with an explicit, tested contract (best-first, one per source, backfill, then inject a second category by evicting the lowest-ranked pick). - classify_articles now returns attempted/succeeded/skipped (ClassifyReport) so silent model failures are visible in both the cycle and classify output. - Fix clean_text truncation to stay within max_len (ellipsis no longer overshoots). - New filters.py: canonical FilterPrefs shape (include/mute topics+flavors, avoid_terms, pauses) and pure word/phrase-boundary matching engine seeding Calm Filters. Not yet wired into the API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+2
-1
@@ -26,7 +26,8 @@ def clean_text(value: str | None, max_len: int = 1000) -> str | None:
|
||||
text = html.unescape(text)
|
||||
text = WHITESPACE_RE.sub(" ", text).strip()
|
||||
if len(text) > max_len:
|
||||
return text[: max_len - 1].rstrip() + "..."
|
||||
# Keep the ellipsis inside max_len rather than overshooting by 3.
|
||||
return text[: max_len - 3].rstrip() + "..."
|
||||
return text or None
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user