Real-time geospatial OSINT intelligence dashboard
5849d72299
Some checks failed
build-and-deploy / build (push) Failing after 4s
The vendored spider only started on urls.txt lines containing '/rss' or '/feed' — but the curated urls.txt holds 200 homepages, so the crawler matched ZERO feeds and silently did nothing (2ms, 0 items). Rewrite the spider with feed autodiscovery: fetch each homepage, find its <link rel="alternate" type="application/rss+xml"> (or /feed|/rss link), parse the feed, then follow each item to extract the article. Bump DEPTH_LIMIT 1->3 (homepage -> feed -> article). Also make the summarizer resilient to booting before the app container has run alembic (docker-compose only guarantees `db` is up): add idempotent ensure_tables() mirroring the scraper's CREATE TABLE IF NOT EXISTS. |
||
|---|---|---|
| .forgejo/workflows | ||
| alembic | ||
| app | ||
| docs | ||
| news | ||
| tests | ||
| .dockerignore | ||
| .env.example | ||
| .gitignore | ||
| alembic.ini | ||
| docker-compose.yml | ||
| Dockerfile | ||
| Dockerfile.pg | ||