Commit graph

8 commits

Author SHA1 Message Date
Sirius DevOps
627990efde Add NASA FIRMS active-fire ingest + /api/fires; API keys management page
Coherent merge of two coordinated features on the shared working tree:

FIRMS fire heatmap (backend, t_6e404c14):
- app/fire_sources.py: fetch FIRMS VIIRS area CSV (free MAP_KEY) -> NATS events.fire
- fires hypertable (TimescaleDB, 1-day chunks) with natural-key PK
  (latitude, longitude, acq_time, satellite); idempotent ON CONFLICT DO NOTHING
- alembic/versions/002_fires.py; GET /api/fires?bbox=&since= (JSON only)
- POST /api/ingest/fires; ~15 min poll loop (FIRMS_INTERVAL=900) in ingester
- env-driven config (FIRMS_MAP_KEY/DATASET/BBOX/INTERVAL); docs/firms.md covers
  the zero-cost GIBS VIIRS_SNPP_Thermal_Anomalies_375m_All tile alternative
- 18 tests (parser, mapping, idempotency, API contract) verified vs real
  TimescaleDB+PostGIS (localhost/osint-dashboard-pg image)

API keys page (frontend, t_4433cff2):
- app/keystore.py: api_keys table (self-creating), FIRMS/GEMINI/TELEGRAM
  registry with format validation, ****last4 masking, get_api_key()
- GET/POST/DELETE /api/keys (never returns full values); Keys tab in index.html

DB_NULL_POOL env switch in app/database.py enables a NullPool for tests /
short-lived processes that open a fresh event loop per unit.
2026-08-24 15:37:42 -04:00
sirius0xdev
8c7f17f645
Harden ingest_event: coerce non-UUID source_id to NULL (tolerates legacy messages)
All checks were successful
build-and-deploy / build (push) Successful in 4m15s
2026-07-07 19:44:13 -04:00
sirius0xdev
67fee36c7c
Wire ingestion: long-running ingester (sources->NATS->Postgres), fix nats-py API (durable=/LIMITS), drop UUID source_id on RSS
All checks were successful
build-and-deploy / build (push) Successful in 4m18s
2026-07-07 19:35:28 -04:00
sirius0xdev
5496023285
Add long-running ingester (sources->NATS->Postgres) as compose 'ingest' profile service
Some checks failed
build-and-deploy / build (push) Failing after 2m58s
2026-07-07 19:18:37 -04:00
sirius0xdev
8eabe3b5e8
Fix SQLAlchemy 2.0 count().filter(); add search_vector trigger for full-text search
Some checks failed
build-and-deploy / build (push) Failing after 15s
2026-07-07 18:32:02 -04:00
sirius0xdev
0a6da2f9c1
Run migrations via entrypoint (alembic upgrade head) instead of nested asyncio.run in startup hook
Some checks failed
build-and-deploy / build (push) Failing after 16s
2026-07-07 18:24:01 -04:00
sirius0xdev
3d53ed2c1c
Containerize for Pi: env-var config, compose stack, CI workflow
Some checks failed
build-and-deploy / build (push) Failing after 18s
- Add app/config.py (12-factor, env-driven: DB/NATS/MinIO, URL-encoded pw)
- Route database/ingestor/sources/alembic through config; kill hardcoded
  svc.cluster.local hostnames
- Drop unused redis dependency
- Add docker-compose.yml (timescaledb-postgis pg13 arm64 + optional nats)
- Add .env.example, .dockerignore, .forgejo/workflows/build.yml (arm64 build+SSH deploy)
2026-07-07 17:50:51 -04:00
sirius0xdev
93b8b89639
first commit 2026-06-04 20:30:04 -04:00