Commit graph

21 commits

Author SHA1 Message Date
240b9e70b3 Merge pull request 'feat(api): GET /api/stats HUD counters (counts only, never 500)' (#42) from osint-dashboard/t_75d2037f-osint-get-api-stats-hud-counters into master
All checks were successful
build-and-deploy / build-push-deploy (push) Successful in 20s
Reviewed-on: #42
2026-08-31 22:12:55 -04:00
Sirius DevOps
92dd9c5803 feat(api): GET /api/stats HUD counters (counts only, never 500) 2026-08-31 21:41:45 -04:00
Sirius DevOps
a82b62a011 feat(conflicts): GET /api/conflicts catalog + news/GDELT event counts
Curated static catalogue of 13 conflict theatres (Ukraine, Gaza, Sudan,
Myanmar, DRC, Yemen, Syria, Lebanon, Sahel, Somalia, Red Sea, Taiwan
Strait, Korean DMZ) with war|high|elevated severity and short factual
descriptions. GET /api/conflicts returns { zones: [{id,label,severity,
lat,lon,description,eventCount,lastUpdated}], timestamp } where
eventCount rolls up pre-existing geocoded news/GDELT//api/news/map rows
inside each zone bbox (0 when the tables are empty). Adds the conflicts
entry to overlay_catalog(). No upstream scraping, no jittered coords.
2026-08-31 21:30:23 -04:00
Sirius DevOps
c48788d4b6
fix(map): geofence delete, vessel snapshots, sentinel cache, news briefs
Geofences could be drawn but not removed. VesselAPI Hormuz dots vanished
on restart and DVR skipped between the 5 daily polls. Sentinel-1 re-hit
STAC on every pan and often painted a neighbouring swath. Executive
briefs truncated; ticker stayed empty unless something was critical.

- Layer-panel list + polygon popup DELETE /api/geofences/{id}
- Persist VesselAPI polls to vessels (UTC-day purge, DVR as-of, boot hydrate)
- Cache Sentinel-1 by 2° cell; pick covering scene; clip Leaflet tiles
- Retry truncated LLM JSON; ticker falls back to medium/low; 3-min HUD poll
2026-08-29 20:40:27 -04:00
0406eb6b7b Merge pull request 'feat(gpsjam): GPS interference hex overlay API (GPSJAM/ADS-B Exchange)' (#29) from feat/gpsjam-interference-overlay into master
Some checks failed
build-and-deploy / build-push-deploy (push) Failing after 1h14m25s
Reviewed-on: #29
2026-08-29 17:54:01 -04:00
Sirius DevOps
42ca6295f0 feat(gpsjam): GPS interference hex overlay API (GPSJAM/ADS-B Exchange)
GET /api/map/gpsjam?date=YYYY-MM-DD (default yesterday UTC) fetches the
daily gpsjam.org H3 resolution-4 CSV, converts hexes to GeoJSON polygons,
and tags each with level low|medium|high (0-2% / 2-10% / >10%) via GPSJAM's
denoise formula. Whole world once, 1h TTL, graceful unavailable fallback.

- overlay_catalog: add kind=geojson gpsjam stub
- tests: mocked-HTTP unit + API contract (9 new)
2026-08-29 14:17:52 -04:00
Sirius DevOps
ef8b877d21 feat(map): chokepoint preset catalog + vessels ?src= filter
- GET /api/map/chokepoints: static five-strait fly-to catalog (Hormuz, Bab
  el-Mandeb, Suez, Malacca, Taiwan) with bbox/center/zoom/vesselapi. No
  VesselAPI calls; only Hormuz flags vesselapi=true.
- GET /api/vessels?src=aisstream|vesselapi|all (default all) filters the
  union store by provider so a Hormuz view skips ~5k CONUS AISStream rows.
- Tests: span validation of all five boxes, catalog 200 shape, src filter.
2026-08-29 14:03:15 -04:00
Sirius DevOps
86b412dd37 fix(vessels): protect Hormuz VesselAPI rows from AISStream eviction
AISStream's live US-coast feed crowds vessel_last_known past _MAX_VESSELS
(6000) and the 'evict oldest seen_at' pass reaps the Hormuz VesselAPI rows,
which are only refreshed every VESSELAPI_INTERVAL (~4.8h). Result: the poller
was fetching 50 Hormuz vessels per 2xx but /api/vessels?bbox=Hormuz stayed
empty. Eviction now skips src=vesselapi rows (falls back to evictable-only,
clamped) so the blind-spot box stays resident between polls.
2026-08-29 13:05:57 -04:00
Sirius DevOps
68e1b63421 fix(titiler): drop @1x suffix + append SAS token top-level
TiTiler /tiles/{z}/{x}/{y} takes a strict int for y; the @1x scale-suffix
template 422s on every SAR tile request (int_parsing on '21@1x').

sign_cog_url wrapped the Planetary Computer SAS token under a single
token= param, which Azure rejects (403→409); the token is a pre-encoded
query string and must be appended top-level (st=…&se=…&sig=…).

Proven live: correct-form URL renders a 200 image/png 256px tile.
2026-08-29 11:42:03 -04:00
Sirius DevOps
5212aaa5cd feat(sentinel1): Sentinel-1 SAR STAC -> self-hosted TiTiler tile template
Add GET /api/map/sentinel1?bbox=... which queries Planetary Computer STAC
(sentinel-1-grd, last 7d, most recent), signs the vv/hh COG with a SAS token,
and returns a same-origin /titiler/... XYZ tile template. Cache keyed on
quantized bbox + UTC day (20 min TTL). 429 -> 429 (Retry-After), no imagery
-> 404, other upstream errors -> 502.

Self-host TiTiler on the Pi (ghcr.io/developmentseed/titiler, arm64, 1G cap,
host loopback 8001 -> container 8000) instead of titiler.xyz. nginx
/titiler/ proxy snippet routes browser tiles to it; TITILER_PUBLIC_BASE and
TITILER_INTERNAL_URL are env-driven (no hardcoded hostnames).
2026-08-29 01:01:20 -04:00
Sirius DevOps
20a266563c fix(aircraft): show planespotters photo in overlay popup
Planespotters 403s server clients whose User-Agent has no contact
email; compose default UA lacked one. Leaflet popup.update() also
re-ran the bindPopup factory and wiped the thumbnail after load.

- Send a contact UA on planespotters lookups; align compose default.
- Resize the popup without re-rendering content.
- Keep hex/src on live firefighter upserts so they get a photo too.
2026-08-28 23:28:28 -04:00
Sirius DevOps
665aaec22f feat(aircraft): planespotters.net photo in aircraft popup
Add planespotters.net latest-photo lookup for ADS-B aircraft.

- app/live_layers.py: fetch_planespotters_photo() (hex preferred, reg
  fallback) + _normalize_planespotter_photo(); 24h TTL cache (their ToS cap).
- app/main.py: GET /api/aircraft/photo?hex=...|reg=... (422/404/502).
- app/static/index.html: .ps-photo block in ADS-B popup; lazy load on
  popupopen; thumbnail links to photo page + photographer credit.

Server-side proxy, not browser fetch(): planespotters 403s any request
carrying an Origin header (every browser fetch() sends one). The thumbnail
binary is loaded by the browser straight from their CDN, never re-hosted.

Tests: 4 unit + 4 API contract (38 pass in the two files).
2026-08-28 22:58:04 -04:00
Sirius DevOps
d81ea68dec fix: pipeline reliability — WS, summarizer, ingest, health
Stop the live HUD reconnect storm (nginx WS snippet + backoff), copy
intel/nous_client into the summarizer image, and make event ingest
idempotent on URL. GDELT uses the DOC API; NWS no longer sends bbox;
FIRMS is one ON CONFLICT batch; GET /api/aircraft serves last-known.
Health reports freshness without 503ing docker. EONET + CISA KEV added.
2026-08-28 21:49:05 -04:00
Sirius DevOps
ca15885e7f feat: classify aircraft/vessels as military vs civilian in popups
Aircraft: readsb dbFlags bit0, then ICAO combat types / US-DoD hex /
callsign. Popup shows type, operator, alt, vs, squawk, emitter class.
Vessels: ITU ship type 35 = military, 50/51/55 = government; cargo/
tanker/passenger/etc otherwise. Popup shows kind, flag, dest, nav,
IMO, size. Magenta glyphs = military, gold = law/gov.
2026-08-28 21:00:20 -04:00
Sirius DevOps
5ea9a4e879 feat: lean-Pi WS fan-out, geofences, DVR playback, fire/aircraft correlation
Phase 1: in-memory ConnectionManager viewport fan-out, 500ms map debounce,
cachetools TTLCache, background masscan/ffmpeg, compose memory caps.

Phase 2: PostGIS geofences + ST_Intersects alerts, Timescale 1-min CAGGs
and timestamp playback, FIRMS/WFIGS x firefighting ADS-B within 20 miles.

No Redis/Kafka/Celery.
2026-08-28 09:35:18 -04:00
Sirius DevOps
32320635dc feat: AIS stream follows the map viewport
All checks were successful
build-and-deploy / build (push) Successful in 2m32s
The Vessels layer now retunes the server-side AISStream subscription to the
client viewport instead of a static AISSTREAM_BBOX. The frontend POSTs its
quantized viewport box to /api/vessels/subscribe on moveend; the ais_stream
worker coalesces and applies it at the service's 1 subscription/s cap, then
last-known positions for the new area arrive within a couple of seconds (the
frontend does one follow-up fetch after retuning). Bounds the in-memory
vessel store across regions. Key stays server-side.
2026-08-27 21:50:38 -04:00
Sirius DevOps
435f63e473 perf: shared httpx client, gzip, overlay cache-control
Reuse one TLS pool for overlay upstreams (8s/3s timeouts). Gzip JSON
over 1 KB. Aircraft/vessels Cache-Control max-age=5, alerts/perimeters 30.
Lifespan replaces deprecated on_event startup.
2026-08-27 21:21:19 -04:00
Sirius DevOps
9c14b899ee perf: simplify WFIGS perimeter geometry at query time
Ask ArcGIS for 500 features, 5-decimal precision, and ~250m offset so
fire rings are outlines instead of tens of thousands of vertices.
2026-08-27 21:16:53 -04:00
Sirius DevOps
fdd59052c5 perf: cache IEM SBW globally, clip alerts to viewport, slim properties
National storm-based warnings are fetched once (45s TTL) and clipped to
the quantized bbox. Popup fields only — NWS descriptions stay off the wire.
2026-08-27 21:16:16 -04:00
Sirius DevOps
84532d505a perf: per-key overlay cache locks and quantized bbox keys
Nearby pans share a 0.25° cache cell. Slow NWS/WFIGS factories no longer
hold a process-wide lock that stalls trains/aircraft/radar fills.
2026-08-27 21:14:49 -04:00
Sirius DevOps
f779b1f225 feat: toggleable live map feeds (ADS-B, trains, AIS, radar, WFIGS, NWS)
Wire the free data streams from docs/free-data-streams.md into the
dashboard as layer-panel toggles. Third-party APIs are proxied/cached
in FastAPI; raster tiles (IEM, RainViewer, GIBS) stay in the browser.

- Aircraft via ADSB.lol viewport poll (bbox required, radius ≤ 150 nm)
- Amtraker trains, NHC storms, WFIGS incidents/perimeters
- NWS + IEM SBW as /api/weather-alerts (does not collide with /api/alerts)
- AISStream worker is server-side only and idles without AISSTREAM_API_KEY
- Caltrans CWWP2 D1–D12 camera parser; FIRMS dual-write NOAA-20/21
2026-08-27 19:08:30 -04:00