thejayman77
068073423f
Local-first RSS/Atom ingestion pipeline with metadata-only storage, heuristic + local-LLM scoring, and daily brief builder. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
80 lines
2.2 KiB
Markdown
80 lines
2.2 KiB
Markdown
# goodNews
|
|
|
|
Local-first constructive news ingestion prototype.
|
|
|
|
The first milestone is intentionally small: collect public RSS/Atom metadata, dedupe it, store short source-provided snippets, and attach early reason-coded heuristic scores. It does not store full article bodies.
|
|
|
|
## Commands
|
|
|
|
From this directory:
|
|
|
|
```bash
|
|
python3 -m goodnews init-db
|
|
python3 -m goodnews import-sources
|
|
python3 -m goodnews poll --limit 3
|
|
python3 -m goodnews rescore
|
|
python3 -m goodnews check-llm --base-url http://127.0.0.1:1234/v1 --model gpt-oss
|
|
python3 -m goodnews classify --limit 10 --base-url http://127.0.0.1:1234/v1 --model gpt-oss
|
|
python3 -m goodnews build-brief --date 2026-05-27 --replace
|
|
python3 -m goodnews show-brief
|
|
python3 -m goodnews list-recent --limit 10
|
|
python3 -m goodnews list-recent --accepted-only --limit 10
|
|
python3 -m goodnews source-report
|
|
python3 -m goodnews list-runs
|
|
```
|
|
|
|
The SQLite database lives at:
|
|
|
|
```txt
|
|
data/goodnews.sqlite3
|
|
```
|
|
|
|
Sources live at:
|
|
|
|
```txt
|
|
config/sources.toml
|
|
```
|
|
|
|
## Stored Article Data
|
|
|
|
For each article, the database stores:
|
|
|
|
- source
|
|
- canonical URL
|
|
- title
|
|
- short RSS/Atom description or summary
|
|
- author, if present
|
|
- published timestamp, if present
|
|
- image URL, if present
|
|
- language, if present
|
|
- hashes used for dedupe
|
|
- heuristic scores and reason codes
|
|
|
|
## Next Steps
|
|
|
|
1. Run the poller for a few days and inspect which sources produce useful candidates.
|
|
2. Add source-level quality notes and deactivate noisy feeds.
|
|
3. Replace or supplement `heuristic-v0` with a local model classifier.
|
|
4. Add a daily brief builder that selects 5 items using scores and source diversity.
|
|
5. Add a small web/API layer once the ingest data looks trustworthy.
|
|
|
|
## Local Model Configuration
|
|
|
|
The `classify` command expects an OpenAI-compatible local chat-completions server.
|
|
|
|
You can pass settings directly:
|
|
|
|
```bash
|
|
python3 -m goodnews classify --base-url http://127.0.0.1:1234/v1 --model gpt-oss --limit 10
|
|
```
|
|
|
|
Or use environment variables:
|
|
|
|
```bash
|
|
export GOODNEWS_LLM_BASE_URL=http://127.0.0.1:1234/v1
|
|
export GOODNEWS_LLM_MODEL=gpt-oss
|
|
python3 -m goodnews classify --limit 10
|
|
```
|
|
|
|
`classify` rewrites the current score/reason row for selected candidates. `rescore` can restore the fast heuristic scores.
|