Commit graph

6 commits

Author SHA1 Message Date
Sirius DevOps
5651d251d0
fix(ingest): skip known KEV/FIRMS rows; CI skips unchanged images
CISA KEV republished 1685 NIST URLs every 5 min; FIRMS re-inserted
~325k global hotspots every 15 min. Producer now skips URLs already
in event_dedup and FIRMS CSVs that are unchanged (delta-only persist).

CI rebuilds only images whose paths changed and never pulls/rebuilds
Timescale or bounces osint-db unless Dockerfile.pg changes.
2026-08-29 19:32:00 -04:00
Sirius DevOps
d81ea68dec fix: pipeline reliability — WS, summarizer, ingest, health
Stop the live HUD reconnect storm (nginx WS snippet + backoff), copy
intel/nous_client into the summarizer image, and make event ingest
idempotent on URL. GDELT uses the DOC API; NWS no longer sends bbox;
FIRMS is one ON CONFLICT batch; GET /api/aircraft serves last-known.
Health reports freshness without 503ing docker. EONET + CISA KEV added.
2026-08-28 21:49:05 -04:00
Sirius DevOps
5ea9a4e879 feat: lean-Pi WS fan-out, geofences, DVR playback, fire/aircraft correlation
Phase 1: in-memory ConnectionManager viewport fan-out, 500ms map debounce,
cachetools TTLCache, background masscan/ffmpeg, compose memory caps.

Phase 2: PostGIS geofences + ST_Intersects alerts, Timescale 1-min CAGGs
and timestamp playback, FIRMS/WFIGS x firefighting ADS-B within 20 miles.

No Redis/Kafka/Celery.
2026-08-28 09:35:18 -04:00
sirius0xdev
67fee36c7c
Wire ingestion: long-running ingester (sources->NATS->Postgres), fix nats-py API (durable=/LIMITS), drop UUID source_id on RSS
All checks were successful
build-and-deploy / build (push) Successful in 4m18s
2026-07-07 19:35:28 -04:00
sirius0xdev
3d53ed2c1c
Containerize for Pi: env-var config, compose stack, CI workflow
Some checks failed
build-and-deploy / build (push) Failing after 18s
- Add app/config.py (12-factor, env-driven: DB/NATS/MinIO, URL-encoded pw)
- Route database/ingestor/sources/alembic through config; kill hardcoded
  svc.cluster.local hostnames
- Drop unused redis dependency
- Add docker-compose.yml (timescaledb-postgis pg13 arm64 + optional nats)
- Add .env.example, .dockerignore, .forgejo/workflows/build.yml (arm64 build+SSH deploy)
2026-07-07 17:50:51 -04:00
sirius0xdev
93b8b89639
first commit 2026-06-04 20:30:04 -04:00