sirius/osint-dashboard
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
osint-dashboard/news/scraper/newsScraper/spiders
History
Exact
Sirius DevOps 8643153954 feat: continuous news scrape + 15-min analyst, expand urls.txt 
Scraper loops with NEWS_SCRAPE_INTERVAL_S (default 10s after each
crawl). Summarizer runs every NEWS_SUMMARIZE_INTERVAL_S (default 900)
over the last 15 minutes of articles. Feed list replaced from the
k8s scrapy-urls configmap (334 sources).
2026-08-28 20:53:32 -04:00
..
__init__.py Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
news_spider.py feat: continuous news scrape + 15-min analyst, expand urls.txt 2026-08-28 20:53:32 -04:00