Interactive world map panel for the dashboard:
- Vendored Leaflet 1.9.4 under app/static/vendor/leaflet/ (served via
new /static mount on the FastAPI app)
- New Map tab: GIBS WMTS raster basemap (BlueMarble_ShadedRelief_Bathymetry,
VIIRS/MODIS/Aqua CorrectedReflectance_TrueColor, VIIRS_DayNightBand),
layer picker + UTC date selector (Latest/-7d/-30d quick picks)
- /api/map/layers: curated GIBS catalog (tms, format, has_time, max_zoom)
- /api/map/times: per-layer date windows from GIBS Domains XML (6h cache,
graceful 502 on GIBS hiccup; static layers 422)
- Fire hotspots (existing /api/fires) + open cameras (/api/cameras) as
toggleable bbox-scoped overlays; dark UI matching dashboard theme
Vendor the newsPipeline scraper + summarizer into the repo and wire them into
docker-compose against the EXISTING osint-db (no second Postgres), replacing
the upstream k8s CronJobs with in-compose wall-clock loops (:00 scrape, :05
summarize).
- news/scraper: vendored Scrapy project (257 RSS feeds) + hourly loop
scheduler (run_news_scraper.py)
- news/summerizer: vendored Gemini map-reduce summarizer, cleaned:
* fix broken google-genai response handling (_extract_text, defensive)
* fix malformed INSERT/GRANT query in save_summary_to_db
* OSINT-neutral default MAP_PROMPT; futures/markets language gated behind
INCLUDE_FUTURES=0 (yfinance lazy-imported)
* env-configurable model, batch size, lookback window
+ hourly loop scheduler (run_news_summarizer.py, :05)
- alembic 003_news: idempotent articles + article_summaries tables
- API: GET /api/news and GET /api/news/summaries (+ models, schemas)
- tests/test_api_news.py: 5 DB-backed contract tests (all pass vs real PG)
- docs/news.md + .env.example updates
Both services run under the `ingest` compose profile (matching the
ingester/camera-scraper pattern) and build arm64 on the Pi via the existing
Forgejo CI workflow. telebot left out of scope (reserved env only).
Coherent merge of two coordinated features on the shared working tree:
FIRMS fire heatmap (backend, t_6e404c14):
- app/fire_sources.py: fetch FIRMS VIIRS area CSV (free MAP_KEY) -> NATS events.fire
- fires hypertable (TimescaleDB, 1-day chunks) with natural-key PK
(latitude, longitude, acq_time, satellite); idempotent ON CONFLICT DO NOTHING
- alembic/versions/002_fires.py; GET /api/fires?bbox=&since= (JSON only)
- POST /api/ingest/fires; ~15 min poll loop (FIRMS_INTERVAL=900) in ingester
- env-driven config (FIRMS_MAP_KEY/DATASET/BBOX/INTERVAL); docs/firms.md covers
the zero-cost GIBS VIIRS_SNPP_Thermal_Anomalies_375m_All tile alternative
- 18 tests (parser, mapping, idempotency, API contract) verified vs real
TimescaleDB+PostGIS (localhost/osint-dashboard-pg image)
API keys page (frontend, t_4433cff2):
- app/keystore.py: api_keys table (self-creating), FIRMS/GEMINI/TELEGRAM
registry with format validation, ****last4 masking, get_api_key()
- GET/POST/DELETE /api/keys (never returns full values); Keys tab in index.html
DB_NULL_POOL env switch in app/database.py enables a NullPool for tests /
short-lived processes that open a fresh event loop per unit.
