sirius/osint-dashboard
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
osint-dashboard/news/scraper
History
Sirius DevOps 8643153954 feat: continuous news scrape + 15-min analyst, expand urls.txt 
Scraper loops with NEWS_SCRAPE_INTERVAL_S (default 10s after each
crawl). Summarizer runs every NEWS_SUMMARIZE_INTERVAL_S (default 900)
over the last 15 minutes of articles. Feed list replaced from the
k8s scrapy-urls configmap (334 sources).
2026-08-28 20:53:32 -04:00
..
newsScraper feat: continuous news scrape + 15-min analyst, expand urls.txt 2026-08-28 20:53:32 -04:00
.dockerignore Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
.gitignore Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
Dockerfile Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
requirements.txt Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
run_news_scraper.py feat: continuous news scrape + 15-min analyst, expand urls.txt 2026-08-28 20:53:32 -04:00
scrapy.cfg Add news pipeline: hourly scraper + Gemini summarizer in compose 2026-08-24 17:28:46 -04:00
urls.txt feat: continuous news scrape + 15-min analyst, expand urls.txt 2026-08-28 20:53:32 -04:00