Commit graph

8 commits

Author SHA1 Message Date
Sirius DevOps
d09009e0bf merge master: keep VesselAPI env and self-hosted TiTiler in compose
PR #20 conflicted on docker-compose.yml after #19 landed. App service now
passes both VESSELAPI_* and TITILER_* ; titiler stays on 127.0.0.1:8001.
2026-08-29 08:25:51 -04:00
Sirius DevOps
5212aaa5cd feat(sentinel1): Sentinel-1 SAR STAC -> self-hosted TiTiler tile template
Add GET /api/map/sentinel1?bbox=... which queries Planetary Computer STAC
(sentinel-1-grd, last 7d, most recent), signs the vv/hh COG with a SAS token,
and returns a same-origin /titiler/... XYZ tile template. Cache keyed on
quantized bbox + UTC day (20 min TTL). 429 -> 429 (Retry-After), no imagery
-> 404, other upstream errors -> 502.

Self-host TiTiler on the Pi (ghcr.io/developmentseed/titiler, arm64, 1G cap,
host loopback 8001 -> container 8000) instead of titiler.xyz. nginx
/titiler/ proxy snippet routes browser tiles to it; TITILER_PUBLIC_BASE and
TITILER_INTERNAL_URL are env-driven (no hardcoded hostnames).
2026-08-29 01:01:20 -04:00
Sirius DevOps
1f23083351 feat(vessels): VesselAPI quota-capped AIS poller for Middle East blind spot
Add a server-side REST poller for VesselAPI (free tier 150 calls/mo) that
upserts last-known positions into the shared vessel_last_known store when
AISStream is unset. Default box is the Strait of Hormuz (span 3.6 <= 4 deg),
never CONUS/NC (AISStream owns US coasts).

- app/vesselapi.py: worker loop, 4deg span validator, position->marker
  transform (skip suspected_glitch), durable Postgres daily-quota table.
- Local hard cap 5 successful 2xx/UTC day (VESSELAPI_MAX_CALLS_PER_DAY),
  monthly floor from X-RateLimit-Remaining, single request limit=50, no
  nextToken, no filter.sat, no retry-storm.
- keystore VESSELAPI_API_KEY registry entry; config + compose env passthrough
  (app + ingest); wired next to AISStream in main.py lifespan + run_ingester.
- GET /api/vessels docstring notes AISStream and/or VesselAPI cache.
- tests/test_vesselapi.py: 20 unit tests (no network/DB).
2026-08-29 00:18:54 -04:00
Sirius DevOps
f779b1f225 feat: toggleable live map feeds (ADS-B, trains, AIS, radar, WFIGS, NWS)
Wire the free data streams from docs/free-data-streams.md into the
dashboard as layer-panel toggles. Third-party APIs are proxied/cached
in FastAPI; raster tiles (IEM, RainViewer, GIBS) stay in the browser.

- Aircraft via ADSB.lol viewport poll (bbox required, radius ≤ 150 nm)
- Amtraker trains, NHC storms, WFIGS incidents/perimeters
- NWS + IEM SBW as /api/weather-alerts (does not collide with /api/alerts)
- AISStream worker is server-side only and idles without AISSTREAM_API_KEY
- Caltrans CWWP2 D1–D12 camera parser; FIRMS dual-write NOAA-20/21
2026-08-27 19:08:30 -04:00
Sirius DevOps
16436a8c08 fires: default FIRMS_DAYS to 2 (NRT latency leaves day=1 empty)
Some checks failed
build-and-deploy / build (push) Failing after 1m31s
At some hours FIRMS NRT returns zero detections for day range 1;
day range 2 returns thousands. Bump default to 2.
2026-08-24 22:07:51 -04:00
Sirius DevOps
1f0b831835 fires: append FIRMS day-range param; API 400s without it
All checks were successful
build-and-deploy / build (push) Successful in 1m54s
FIRMS area-CSV requires an explicit [1..5] day range — requests without
it return 'Invalid day range. Expects [1..5].' as a 400. New
FIRMS_DAYS env (default 1).
2026-08-24 21:32:55 -04:00
Sirius DevOps
627990efde Add NASA FIRMS active-fire ingest + /api/fires; API keys management page
Coherent merge of two coordinated features on the shared working tree:

FIRMS fire heatmap (backend, t_6e404c14):
- app/fire_sources.py: fetch FIRMS VIIRS area CSV (free MAP_KEY) -> NATS events.fire
- fires hypertable (TimescaleDB, 1-day chunks) with natural-key PK
  (latitude, longitude, acq_time, satellite); idempotent ON CONFLICT DO NOTHING
- alembic/versions/002_fires.py; GET /api/fires?bbox=&since= (JSON only)
- POST /api/ingest/fires; ~15 min poll loop (FIRMS_INTERVAL=900) in ingester
- env-driven config (FIRMS_MAP_KEY/DATASET/BBOX/INTERVAL); docs/firms.md covers
  the zero-cost GIBS VIIRS_SNPP_Thermal_Anomalies_375m_All tile alternative
- 18 tests (parser, mapping, idempotency, API contract) verified vs real
  TimescaleDB+PostGIS (localhost/osint-dashboard-pg image)

API keys page (frontend, t_4433cff2):
- app/keystore.py: api_keys table (self-creating), FIRMS/GEMINI/TELEGRAM
  registry with format validation, ****last4 masking, get_api_key()
- GET/POST/DELETE /api/keys (never returns full values); Keys tab in index.html

DB_NULL_POOL env switch in app/database.py enables a NullPool for tests /
short-lived processes that open a fresh event loop per unit.
2026-08-24 15:37:42 -04:00
sirius0xdev
3d53ed2c1c
Containerize for Pi: env-var config, compose stack, CI workflow
Some checks failed
build-and-deploy / build (push) Failing after 18s
- Add app/config.py (12-factor, env-driven: DB/NATS/MinIO, URL-encoded pw)
- Route database/ingestor/sources/alembic through config; kill hardcoded
  svc.cluster.local hostnames
- Drop unused redis dependency
- Add docker-compose.yml (timescaledb-postgis pg13 arm64 + optional nats)
- Add .env.example, .dockerignore, .forgejo/workflows/build.yml (arm64 build+SSH deploy)
2026-07-07 17:50:51 -04:00