map: fix cameras vanishing after opening a camera popup and zooming

Root cause: Leaflet 1.9.4's Map.closePopup() never nulls map._popup (only unbindPopup does), so the moveend guard 'if (map._popup) return' skipped every overlay reload forever after the first popup was ever opened. Opening one camera popup froze the marker layer: zoom out and the map kept showing the stale zoomed-in subset (or nothing) until a full page refresh. Track real popup state via popupopen/popupclose events instead, and close the popup on user zoom/drag so the moveend reload always runs after navigation; the autopan skip still protects the popup when it pans itself into view.
2026-08-27 17:38:38 -04:00 · 2026-08-27 17:38:38 -04:00 · bd4197cb6c
commit bd4197cb6c
parent 5cea0a5940
4 changed files with 630 additions and 6 deletions
--- a/.worktrees/frontend-overhaul
+++ b/.worktrees/frontend-overhaul
@ -0,0 +1 @@
+Subproject commit 6f0fc8dbe3fdd5d0a8c4abb444dd751acd4f9186
--- a/IDEA.md
+++ b/IDEA.md
@ -0,0 +1 @@
+redesign the osing dashboard front end
--- a/app/static/index.html
+++ b/app/static/index.html
@ -833,13 +833,22 @@ async function initMap() {
            if (activeHls) { try { activeHls.destroy(); } catch (_) {} activeHls = null; }
        });
        updateHeatLegend();
-        // Reload overlays when the user pans/zooms. Skip when a popup is open:
-        // opening a camera popup auto-pans the map to fit it, and that moveend
-        // must NOT rebuild the marker group under the open popup (which would
-        // close it). Leaflet closes popups on manual drag, so a live popup here
-        // means the move was autopan — safe to skip.
+        // Reload overlays when the user pans/zooms. A live popup makes the
+        // rebuild skip (so the popup's autopan doesn't destroy it), but user
+        // zoom/drag must close the popup FIRST — otherwise zooming out with a
+        // camera popup open leaves the stale, zoomed-in marker group on the
+        // map and the cameras "vanish" until a refresh.
+        //
+        // NOTE: `map._popup` is NOT a reliable open-check in Leaflet 1.9.4 —
+        // Map.closePopup() never nulls it, so once any popup has been opened
+        // it stays truthy forever. Track the real state via popupopen/close.
+        let camPopupOpen = false;
+        map.on('popupopen', () => { camPopupOpen = true; });
+        map.on('popupclose', () => { camPopupOpen = false; });
+        map.on('zoomstart', () => { if (camPopupOpen) map.closePopup(); });
+        map.on('dragstart', () => { if (camPopupOpen) map.closePopup(); });
        map.on('moveend', () => {
-            if (map._popup) return;
+            if (camPopupOpen) return; // only the popup's own autopan now
            if (firesOn) loadFires();
            if (camsOn) loadCams();
        });
--- a/docs/free-data-streams.md
+++ b/docs/free-data-streams.md
@ -0,0 +1,613 @@
+# Free data streams for the OSINT dashboard
+
+Builder brief for backend + frontend. Researched 2026-08-27. Every endpoint below was either live-probed from this machine or taken from the provider’s current docs. Prefer **free, no-key, CORS-open** sources first. Keys are called out explicitly.
+
+This is **not** a camera-discovery / masscan change. Existing camera rules still apply: never emit `rtsp://` hrefs; masscan pins go through `/api/cameras/{id}/snapshot`; HTTP directory cams use `/stream` MJPEG.
+
+---
+
+## 0. What is already in the repo (do not rebuild)
+
+| Layer | Status | Notes |
+|---|---|---|
+| NASA FIRMS VIIRS hotspots | Ingested (`app/fire_sources.py` → NATS `events.fire` → `fires` hypertable → `GET /api/fires`) | Needs free `FIRMS_MAP_KEY`. See `docs/firms.md`. |
+| NASA GIBS basemaps | Frontend tiles via `app/gibs_map.py` | No key. CORS `*`. |
+| GIBS VIIRS thermal tiles | Documented, not wired as overlay | Same GIBS stack; no key. |
+| Cameras | Scraper + masscan → `cameras` table | Defaults already include ALERTWest JPEGs + Live-Environment-Streams HLS/YouTube GeoJSON. |
+| News / RSS / GDELT / USGS quakes | Ingest | Out of scope for this brief. |
+
+**Action for existing fire ingest:** NASA will stop Suomi NPP product delivery on **2026-11-01**. Switch `FIRMS_DATASET` from `VIIRS_SNPP_NRT` to `VIIRS_NOAA20_NRT` and/or `VIIRS_NOAA21_NRT` before then.[20]
+
+---
+
+## 1. Architecture rules (backend + frontend)
+
+Moving objects (aircraft, vessels, trains) and alerts are **vectors**. Radar / satellite / GIBS fire tiles are **rasters**.
+
+| Kind | Where it runs | Persist? | API shape |
+|---|---|---|---|
+| Radar, GOES, GIBS, OpenRailwayMap tiles | **Browser** `L.tileLayer` (CORS `*` on the recommended ones) | No | Tile URL template |
+| Aircraft / trains / last-known ships | **Backend poll** → last-known row → `GET /api/{layer}?bbox=` | Last-known only (Timescale, 1-day chunks, drop after 24–72 h) | JSON array of points |
+| AIS live stream | **Backend WebSocket** (aisstream forbids browsers)[5] | Last-known + optional 15-min trail | Same `/api/vessels` |
+| NWS / SPC polygons | Backend poll **or** browser (NWS CORS `*`)[17] | Optional 24 h | GeoJSON FeatureCollection |
+| Official JPEG/HLS cameras | Existing `cameras` scraper | Yes, same table | Existing `/stream` + HLS |
+
+**Do not** hit third-party APIs from the Leaflet client when: (a) an API key would leak, (b) CORS is closed, (c) the provider rate-limits by IP and many map users would share one Pi egress. Proxy those through FastAPI.
+
+**Pi / home-uplink constraints:** viewport bbox only. Never poll global ADS-B or global AIS. Cap aircraft poll to 5–10 s, trains 15–30 s, FIRMS already 900 s. Do not store 1 Hz AIS history.
+
+Suggested NATS subjects (match `events.fire`): `events.aircraft`, `events.vessel`, `events.train`, `events.alert`.
+
+Suggested last-known tables (natural keys):
+
+```
+aircraft(icao24 PK, callsign, lat, lon, alt_baro_m, gs_kt, track_deg, squawk, src, seen_at)
+vessels(mmsi PK, name, lat, lon, sog_kt, cog_deg, navstat, dest, src, seen_at)
+trains(train_id PK, route, lat, lon, heading, late_min, src, seen_at)
+```
+
+---
+
+## 2. Build order (what to ship first)
+
+### Phase A — frontend-only, no new tables (1–2 days)
+
+1. RainViewer radar overlay (global, no key, CORS `*`).[13][14]
+2. IEM NEXRAD CONUS overlay as US-quality alternative (no key, CORS `*`).[15]
+3. GIBS `VIIRS_SNPP_Thermal_Anomalies_375m_All` fire-pixel overlay (already documented in `docs/firms.md`).
+4. NWS active alerts GeoJSON overlay (no key, User-Agent required, CORS `*`).[17]
+5. IEM current storm-based warnings GeoJSON.[16]
+
+### Phase B — backend last-known overlays
+
+6. ADSB.lol viewport aircraft (no key today).[3]
+7. OpenSky bbox as fallback / research-grade (anonymous works; OAuth recommended).[1]
+8. Amtraker trains (no key; **must** send identifying `User-Agent`).[9]
+9. WFIGS incident points + current perimeters (no key).[21]
+10. Caltrans CWWP2 cameras into existing `cameras` scraper (JPEG + HLS).[27]
+
+### Phase C — free keys, streams
+
+11. AISStream WebSocket (free GitHub-login key; **server-side only**).[5]
+12. Optional OpenSky OAuth client for 4,000 credits/day vs 400 anonymous.[1]
+13. Optional Transitland key later for multi-agency GTFS-RT (10k REST/month free).[12]
+
+Skip paid: MarineTraffic, ADS-B Exchange, FlightAware, TrafficLand, OpenWeatherMap One Call paid tiers.
+
+---
+
+## 3. Air traffic
+
+### 3.1 ADSB.lol — **recommended primary for the map**
+
+| | |
+|---|---|
+| Cost / key | Free, no key today. Docs say a feeder key *may* be required in the future; contact them before treating this as a production SLA.[3] |
+| License | ODbL 1.0 (same family as OSM; share-alike on the database).[4] |
+| CORS | Treat as **backend-only**. Do not assume browser CORS. |
+| Live probe 2026-08-27 | `GET https://api.adsb.lol/v2/lat/35.88/lon/-78.79/dist/50` → HTTP 200, 27 aircraft near RDU. |
+
+Useful endpoints:[3]
+
+```
+GET /v2/lat/{lat}/lon/{lon}/dist/{radius_nm}   # radius cap 250 nm
+GET /v2/point/{lat}/{lon}/{radius}
+GET /v2/sqk/{squawk}                          # e.g. 7700/7600/7500
+GET /v2/mil
+GET /v2/hex/{icao}
+GET /v2/callsign/{callsign}
+```
+
+Sample fields from live payload: `hex, flight, r, t, lat, lon, alt_baro, gs, track, squawk, emergency, category, seen_pos`.
+
+**Backend:** poll the **map viewport** (center + radius from bbox, clamp ≤ 150 nm) every 5–10 s. Dedup on `hex`. Do not poll `/v2/mil` on a timer unless the user toggles a mil filter.
+
+**Frontend:** circleMarker or rotating icon, tooltip = `flight`/`hex`/`alt`. Cluster above ~200 markers. Color by altitude or `emergency != none`.
+
+### 3.2 OpenSky Network — research-grade fallback
+
+| | |
+|---|---|
+| Cost / key | Free for non-commercial / research. Anonymous OK. Authenticated uses OAuth2 **client credentials** (not basic auth). Create `client_id` + `client_secret` on the account page.[1][26] |
+| Terms | Cite the OpenSky paper + URL for publications. Commercial needs a license. They may block AWS/hyperscaler IPs.[2] |
+| Live probe | `GET /api/states/all?lamin=33.8&lomin=-84.5&lamax=36.6&lomax=-75.4` → HTTP 200, **270** state vectors over NC in 0.63 s. |
+
+Root: `https://opensky-network.org/api`[1]
+
+```
+GET /states/all?lamin=&lomin=&lamax=&lomax=&extended=1
+```
+
+State vector indexes: `0 icao24, 1 callsign, 2 origin_country, 5 lon, 6 lat, 7 baro_alt m, 8 on_ground, 9 velocity m/s, 10 true_track, 14 squawk`.[1]
+
+Credits (independent buckets for `/states/*`, `/tracks/*`, `/flights/*`):[1]
+
+| Tier | Credits / refill |
+|---|---|
+| Anonymous | 400 / day |
+| Standard user (OAuth) | 4,000 / day |
+| Active feeder ≥30% uptime | 8,000 / day |
+
+`/states/all` costs **1–4 credits** by bbox area (≤25 sq° = 1 credit; global = 4). Exhaustion → HTTP 429 + `X-Rate-Limit-Retry-After-Seconds`.[1]
+
+Anonymous: live only, 10 s time resolution. Authenticated: up to 1 hour of history, 5 s resolution.[1]
+
+Token:
+
+```
+POST https://auth.opensky-network.org/auth/realms/opensky-network/protocol/openid-connect/token
+grant_type=client_credentials&client_id=&client_secret=
+Authorization: Bearer <access_token>   # ~30 min TTL
+```
+
+**Backend:** use OpenSky as fallback when ADSB.lol 429s, or for a “research” toggle. Always send a bbox. NC-sized box is ~25 sq° → 1 credit; 10 s poll as anonymous = 8,640 credits/day → **will 429**. Authenticated 4,000/day ≈ one poll every ~22 s if cost=1. Prefer ADSB.lol for the live map; keep OpenSky for sparse/historical.
+
+**Do not** run this from AWS/the Pi if OpenSky starts blocking the IP; the docs warn about hyperscalers.[2]
+
+### 3.3 AviationWeather.gov METARs (context, not tracks)
+
+No key. User-Agent required. **100 req/min**. **No CORS** — backend only.[22]
+
+```
+GET https://aviationweather.gov/api/data/metar?ids=KRDU&format=json
+```
+
+Live probe: KRDU thunderstorm METAR returned JSON with `lat/lon/temp/rawOb`. Use as airport popup, not as an aircraft layer. Prefer the gzip cache files for bulk.
+
+### 3.4 Skip
+
+- ADS-B Exchange — paid gateway / API key.
+- FlightAware / AeroAPI — paid.
+- OpenSky Trino historical — overkill for the live map.
+
+---
+
+## 4. Boat / AIS traffic
+
+### 4.1 AISStream — **recommended live ships**
+
+| | |
+|---|---|
+| Cost / key | Free. Sign in with GitHub, create key at `/account`. Shown once; rotate from Account.[5][6] |
+| Transport | `wss://stream.aisstream.io/v0/stream` |
+| Browser | **Forbidden.** Connect from the FastAPI/ingest container and proxy a downsampled last-known feed.[5] |
+| Limits | 3 connections / account, 3 / IP. Subscribe JSON **within 3 seconds** or the socket closes. Subscription replace ≤ 1/s. No SLA, no replay.[5] |
+| Compression | Negotiate `permessage-deflate`. Uncompressed connections get bandwidth caps starting **September 2026**.[5] |
+
+Subscribe (bbox is **\[\[lat, lon\], \[lat, lon\]\]** — not lon/lat):[5]
+
+```json
+{
+  "APIKey": "<AISSTREAM_API_KEY>",
+  "BoundingBoxes": [[[33.8, -84.5], [36.6, -75.4]]],
+  "FilterMessageTypes": [
+    "PositionReport",
+    "StandardClassBPositionReport",
+    "ExtendedClassBPositionReport",
+    "ShipStaticData"
+  ]
+}
+```
+
+Frames are **binary WebSocket** containing UTF-8 JSON. Decode bytes, then `json.loads`.[5]
+
+Position envelope (docs + site examples): `MessageType`, `MetaData.{MMSI,ShipName,Latitude,Longitude}`, `Message.PositionReport.{Sog,Cog,TrueHeading,NavigationalStatus}`.[5][6]
+
+**Backend:** one long-lived WS in the `ingest` profile. Filter to the dashboard’s configured bbox (default CONUS coasts + Great Lakes, not world). Upsert last-known by MMSI. Expose `GET /api/vessels?bbox=`. Reconnect with jittered backoff; resend full subscription in <3 s. Do not fan the raw stream to browsers.
+
+**Frontend:** circleMarker, color by nav status / SOG, tooltip name+MMSI. Optional 5-minute trail from in-memory ring, not DB.
+
+### 4.2 AISHub — only if you feed a receiver
+
+Free **if you contribute a live AIS UDP feed**. Then JSON/XML/CSV at most **once per minute**.[7][8]
+
+```
+https://data.aishub.net/ws.php?username=USER&format=1&output=json&latmin=&latmax=&lonmin=&lonmax=
+```
+
+`format=1` = human-readable degrees / knots. Faster than once/minute returns empty.[8]
+
+Not useful on the Pi unless Lance later stands up an AIS dongle. Keep as a footnote.
+
+### 4.3 Skip / later
+
+- MarineTraffic REST — commercial.
+- Global Fishing Watch — free **non-commercial** token, fishing-effort tiles / vessel identity, not a general AIS live layer.[24] Optional Phase D for a “fishing effort” overlay.
+- AISHub without a feeder — no data.
+
+---
+
+## 5. Train traffic
+
+There is **no free global live train API**. US intercity is solvable; European freight is not, cheaply.
+
+### 5.1 Amtraker — **recommended US passenger trains**
+
+Community API that rehosts Amtrak + Brightline + VIA Rail. Free. **Identifying `User-Agent` required or the request is blocked.**[9] Data license ODC-By 1.0; attribute “Amtraker” on the map (collapsed OSM-style credit is OK).[9][10]
+
+Live probe: `GET https://api.amtraker.com/v3/trains` and `https://api-v3.amtraker.com/v3/trains` both HTTP 200, **~1.3 MB**, **197 train numbers**. Sample: Sunset Limited `#1`, `lat/lon`, `heading`, `stations[]`, `iconColor`.
+
+```
+GET https://api.amtraker.com/v3/trains
+GET https://api.amtraker.com/v3/trains/:trainId     # e.g. 5-9
+GET https://api.amtraker.com/v3/stations
+GET https://api.amtraker.com/v3/stale               # stale=true if Amtrak >15 min old
+```
+
+Poll **15–30 s**. Response is `{ "1": [Train, ...], "5": [...], ... }` — flatten to one row per `trainID`. Do not hammer; the author asks not to spam.[10]
+
+**Frontend:** train icon, color from `iconColor` (or compute lateness). Popup: route, number, next station, delay.
+
+### 5.2 Transitland — multi-agency GTFS-RT (optional, key)
+
+Free plan: **10,000 REST queries/month**, 100k vector tiles/month, 1,000 routing/month. Key via Interline signup (`apikey=` query or header; **not** `Authorization`). Attribution required.[11][12]
+
+```
+https://transit.land/api/v2/rest/
+```
+
+Use later if you want commuter rail / subway vehicle positions (LA Metro, MTA, etc.). 10k/month is only ~13 queries/hour — **cannot** poll every agency every 15 s. Cache GTFS-RT protobufs yourself from the **agency’s own feed URL** (those are usually free, no Transitland meter). Transitland is a directory; the live positions should come from the agency GTFS-RT URL listed in the Atlas.
+
+### 5.3 OpenRailwayMap — infrastructure tiles only
+
+`https://tiles.openrailwaymap.org/{style}/{z}/{x}/{y}.png` — OSM-derived tracks, not live trains.[25] This probe’s HEAD got **HTTP 403** (UA/hotlink?). Treat as optional overlay; do not depend on it. No vehicle data.
+
+### 5.4 Skip
+
+- Official Amtrak Track-A-Train JSON — unofficial reverse-engineering; Amtraker already does this legally enough for a hobby map, with a documented UA policy.
+- Deutsche Bahn / Network Rail — free-ish developer portals but EU-centric and key + ToS heavy. Out of scope unless the map goes global-rail.
+
+---
+
+## 6. Open video / camera feeds (official public only)
+
+Do **not** add Insecam-style random IP cams as a new source. The scraper already has a public list + masscan; this section is **agency-published** JPEG/HLS.
+
+### 6.1 Already wired
+
+- ALERTWest / ALERTCalifornia: `https://api.cdn.prod.alertwest.com/api/getCameraDataByLoc` — live probe HTTP 200, **6.6 MB** JSON with `data.locs` + `data.cams`. Snapshots at `https://img.cdn.prod.alertwest.com/data/img/{cid}/...` (see `parse_alertwest_json`).
+- Live-Environment-Streams GeoJSON (HLS / YouTube / JPEG).
+- `public-ip-cams` README list.
+
+### 6.2 Caltrans CWWP2 — **add to CAMERA_SOURCE_URLS**
+
+Official CA DOT CCTV. No key.
+
+```
+https://cwwp2.dot.ca.gov/data/d{N}/cctv/cctvStatusD{NN}.json
+```
+
+Live probe D4: HTTP 200, **2.4 MB**, **749 cameras**.
+
+Each `data[].cctv`:
+
+- `location.{latitude,longitude,locationName,nearbyPlace,district,route,county,direction}`
+- `inService` (`"true"`/`"false"`)
+- `imageData.static.currentImageURL` — JPEG, updates ~5 s (`https://cwwp2.dot.ca.gov/data/d4/cctv/image/...jpg`)
+- `imageData.streamingVideoURL` — **HLS** `https://wzmedia.dot.ca.gov/D4/....stream/playlist.m3u8`
+
+Districts 1–12 exist with the same schema (D4 verified). Backend: new parser next to `parse_alertwest_json`, skip `inService != true`, store JPEG as `snapshot_url`, HLS as stream URL (dashboard already vendors `hls.min.js`). Frontend: same camera popup; prefer JPEG for map thumbs, HLS in the preview pane. Never RTSP.
+
+Politeness: scrape hourly like other sources (`CAMERA_SCRAPE_INTERVAL=3600`). Do not scrape every JPEG every cycle — store URLs, let the snapshot cache fetch on demand.
+
+### 6.3 IEM webcams — small, Iowa-centric bonus
+
+```
+GET https://mesonet.agron.iastate.edu/geojson/webcam.geojson
+```
+
+No key, CORS permissive.[23] Live probe: HTTP 200, **3** cameras at probe time (not a national network). Fields: `cid, name, imgurl, utc_valid, state`. Nice extra source, low value vs ALERTWest/Caltrans.
+
+### 6.4 YouTube live (already in Live-Environment-Streams)
+
+Do not call the YouTube Data API unless you want search. Embedding existing stream URLs from the GeoJSON is enough. YouTube Data API **requires a Google key** and quota.
+
+### 6.5 Skip
+
+- Insecam / random “public IP cam” aggregators — ToS / privacy / already covered by masscan ethics.
+- TrafficLand, EarthCam commercial APIs.
+- SkylineWebcams — scraping, not an API.
+
+---
+
+## 7. Fire maps (beyond current FIRMS points)
+
+FIRMS CSV = hotspot **points** (already ingested). WFIGS = **incident identity + perimeters**. GIBS = **pixel overlay**. Use all three; they answer different questions.
+
+### 7.1 WFIGS current incident locations — **recommended vector layer**
+
+No key. ArcGIS FeatureServer, GeoJSON.
+
+```
+https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Incident_Locations_Current/FeatureServer/0/query
+  ?where=1=1&outFields=IncidentName,IncidentSize,FireDiscoveryDateTime,POOState,PercentContained,IncidentTypeCategory,FireCause
+  &outSR=4326&f=geojson
+```
+
+Live probe: **635** current locations. Sample: `Foster Bridge`, 675 acres, `US-FL`, 100% contained, cause Human. `FireDiscoveryDateTime` is **epoch ms**.
+
+Poll 5–15 min (NIFC says related perimeter service refreshes ~5 min).[21] Backend can proxy `GET /api/fire-incidents` so the keyless URL is cached. Frontend: labeled point, popup acres / contained %.
+
+Some similarly named services (`Current_WildlandFire_Locations`) return **Token Required** — stick to `WFIGS_Incident_Locations_Current`.
+
+### 7.2 WFIGS current perimeters — **recommended polygon overlay**
+
+```
+https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Interagency_Perimeters_Current/FeatureServer/0/query
+  ?where=1=1&outFields=poly_IncidentName,poly_GISAcres,attr_IncidentSize,attr_PercentContained,attr_FireDiscoveryDateTime
+  &outSR=4326&f=geojson
+```
+
+Live probe: **236** current perimeters. Historic `WFIGS_Interagency_Perimeters` counted **41,078** — do not fetch that unfiltered. Use Current only, or `resultRecordCount` + bbox `geometry` + `spatialRel=esriSpatialRelIntersects`.
+
+Authoritative US wildland-fire polygons; not every incident has a perimeter. Refresh ~5 min, fall-off rules drop stale small fires.[21]
+
+**Frontend:** `L.geoJSON` fill red/orange by acres. Click for name/acres. This is the layer that makes FIRMS dots make sense.
+
+### 7.3 GIBS thermal tiles (no key, no storage)
+
+Already in `docs/firms.md`. Leaflet:
+
+```
+https://gibs.earthdata.nasa.gov/wmts/epsg3857/best/VIIRS_SNPP_Thermal_Anomalies_375m_All/default/{time}/GoogleMapsCompatible_Level{z}/{y}/{x}.png
+```
+
+Max zoom ~9. `{time}` = `YYYY-MM-DD`. Same GIBS CORS `*` as existing basemaps. Use as a toggle under FIRMS points.
+
+### 7.4 FIRMS dataset note
+
+Keep the CSV ingest. Add NOAA-20/21 datasets. Ultra-real-time US/Canada is available on those products (seconds-scale for much of US/Canada).[20]
+
+---
+
+## 8. Weather radar + alerts
+
+### 8.1 RainViewer — **recommended global radar overlay**
+
+No key. Personal / educational / small community. Attribution: “Weather data by RainViewer” + link.[13]
+
+Live probe: `GET https://api.rainviewer.com/public/weather-maps.json` → version `2.0`, host `https://tilecache.rainviewer.com`, **13** past frames (~2 h, 10 min steps). Tile HEAD: HTTP 200, CORS `*`, **`x-ratelimit-limit: 500` / 60 s**, burst 300.
+
+Tile URL:[14]
+
+```
+{host}{path}/{size}/{z}/{x}/{y}/{color}/{smooth}_{snow}.png
+```
+
+Example: `https://tilecache.rainviewer.com/v2/radar/cb581daa2c0f/256/{z}/{x}/{y}/2/1_1.png`
+
+- `{size}` 256 or 512
+- `{color}` scheme id (2 is a common default)
+- `{smooth}_{snow}` e.g. `1_1`
+- **max zoom 7** (coarse vs IEM)
+
+**Frontend:** fetch the JSON every 5 min, set `L.tileLayer` to the latest `past[].path`. Optional animation through `past` frames. No backend.
+
+### 8.2 IEM NEXRAD / MRMS — **recommended CONUS quality radar**
+
+Iowa Environmental Mesonet. No key. Free including commercial, as-is, no SLA.[16] CORS `*` on tiles (live HEAD confirmed).
+
+Current CONUS base reflectivity TMS:[15]
+
+```
+https://mesonet.agron.iastate.edu/cache/tile.py/1.0.0/nexrad-n0q/{z}/{x}/{y}.png
+```
+
+Cache-Control 5 min. Aliases `mesonet1/2/3.agron.iastate.edu` for parallel browser tile fetches.[15]
+
+Loop (past 5–55 min, step 5): layer `nexrad-n0q-mXXm` e.g. `nexrad-n0q-m05m`.
+
+Also useful:
+
+| Layer | Meaning |
+|---|---|
+| `nexrad-eet` | Echo tops |
+| `q2-hsr` | MRMS seamless hybrid-scan reflectivity |
+| `q2-n1p` / `q2-p24h` | MRMS 1 h / 24 h precip |
+| `goes_east_conus_ch02` | GOES-East visible |
+| `goes_east_conus_ch13` | GOES-East IR |
+
+WMS alternative: `https://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0q.cgi?` — TMS is simpler for Leaflet.
+
+**Frontend:** use IEM as default when the map is over CONUS (higher res than RainViewer z7); RainViewer when zoomed outside the US or as a global fallback.
+
+### 8.3 NWS API — alerts / forecasts (no radar tiles)
+
+`https://api.weather.gov` — US public domain, no fee. **User-Agent required** (app + contact). Rate limit unpublished; 429s clear in ~5 s. CORS `*`.[17]
+
+Live probe: `GET /alerts/active?area=NC` → HTTP 200 GeoJSON, 4 alerts, CORS `*`.
+
+```
+GET https://api.weather.gov/alerts/active?area=NC
+GET https://api.weather.gov/alerts/active?point={lat},{lon}
+GET https://api.weather.gov/points/{lat},{lon}          # then follow forecast URLs
+```
+
+**Frontend can hit this directly** (CORS open) **or** backend can cache 30 s to protect the Pi if many tabs are open. Prefer backend cache. Draw `features[].geometry` with color by `properties.severity` / `event`.
+
+### 8.4 IEM storm-based warnings GeoJSON
+
+```
+GET https://mesonet.agron.iastate.edu/geojson/sbw.geojson
+```
+
+Live probe: **46** current polygons. Sample: Severe Thunderstorm Warning, WFO BGM. Good companion to NWS alerts (polygon-accurate).
+
+### 8.5 Open-Meteo — point forecast, not radar
+
+No key for non-commercial **< 10,000 calls/day**. CC BY 4.0 data, attribution required. Commercial needs a plan or self-host.[18][19]
+
+```
+GET https://api.open-meteo.com/v1/forecast?latitude=35.88&longitude=-78.79&current=temperature_2m,precipitation,weather_code,wind_speed_10m
+```
+
+Live probe: RDU 28.4 °C, weather_code 80 (rain showers). Use for click-to-forecast popups, not a map overlay. Cache by grid cell.
+
+### 8.6 NHC active storms
+
+```
+GET https://www.nhc.noaa.gov/CurrentStorms.json
+```
+
+Live probe: HTTP 200, `activeStorms[]` — Tropical Storm Dolly (`al042026`, 13.6N 38.7W, 35 kt). No key. Plot as a hurricane marker + optional NHC cone GeoJSON (separate NHC GIS products). Nice extra during Atlantic season.
+
+---
+
+## 9. Keys / env vars to add
+
+Store new secrets in the existing Keys UI (`api_keys` table) **and** `.env` fallback, same planned pattern as FIRMS.
+
+| Var | Required for | How to get | Cost |
+|---|---|---|---|
+| `FIRMS_MAP_KEY` | Existing fire CSV | https://firms.modaps.eosdis.nasa.gov/api/map_key_info/ | Free |
+| `OPENSKY_CLIENT_ID` / `OPENSKY_CLIENT_SECRET` | OpenSky authenticated | OpenSky account → API Client | Free |
+| `AISSTREAM_API_KEY` | Live AIS | GitHub login at aisstream.io/account | Free |
+| `TRANSITLAND_API_KEY` | Transitland REST (optional) | Interline free plan | Free 10k/mo |
+| `GFW_TOKEN` | Fishing-effort overlay (optional) | globalfishingwatch.org/our-apis/tokens | Free, non-commercial |
+| *(none)* | ADSB.lol, Amtraker, RainViewer, IEM, NWS, WFIGS, Caltrans, Open-Meteo, NHC | — | Free |
+
+Amtraker: no key, but set `OSINT_USER_AGENT` to something like `osint-dashboard/1.0 (lancewalters94@gmail.com)` — already have `USER_AGENT` in `camera_config.py`; reuse it on **all** outbound HTTP.
+
+NWS: same User-Agent header is mandatory.[17]
+
+---
+
+## 10. Suggested API surface (backend engineer)
+
+Keep FIRMS `GET /api/fires`. Add:
+
+```
+GET /api/aircraft?bbox=minlon,minlat,maxlon,maxlat
+GET /api/vessels?bbox=...
+GET /api/trains?bbox=...
+GET /api/fire-incidents?bbox=...          # WFIGS points
+GET /api/fire-perimeters?bbox=...         # WFIGS polygons (or proxy GeoJSON)
+GET /api/alerts?area=NC|&bbox=...         # cached NWS + IEM SBW
+GET /api/map/radar                         # { provider, tileUrl, frames[] } for RainViewer metadata
+GET /api/map/layers                        # already have GIBS; extend with radar/fire tile templates
+```
+
+All vector endpoints: default `limit=2000`, require bbox except trains (Amtrak is ~200 rows total). Return arrays of `{id, lat, lon, heading, speed, label, extra}` so the frontend can share one marker renderer.
+
+Do **not** proxy RainViewer/IEM/GIBS tiles through the Pi (bandwidth). Return the template; browser fetches tiles.
+
+AISStream stays inside `ingest` profile (needs a long-lived process, like FIRMS). Aircraft/trains can live in `run_ingester.py` loops.
+
+---
+
+## 11. Suggested UI (frontend bot)
+
+Layer toggles on the existing Leaflet map (same pattern as GIBS + fire heatmap):
+
+| Toggle | Default | Source |
+|---|---|---|
+| Radar | on (CONUS) | IEM `nexrad-n0q` if view intersects US, else RainViewer |
+| NWS alerts | on | polygons |
+| FIRMS heat | existing | existing |
+| WFIGS perimeters | on | polygons |
+| WFIGS incidents | off | points |
+| Aircraft | on | `/api/aircraft` |
+| Vessels | off until AIS key | `/api/vessels` |
+| Trains | on | `/api/trains` |
+| Cameras | existing | existing |
+
+Viewport-driven: on `moveend` (debounced 300 ms), refetch aircraft/vessels for the new bbox. Do not refetch radar tiles except RainViewer frame JSON every 5 min.
+
+Marker performance: canvas renderer (`L.canvas()`), cluster at zoom < 7, hide labels until zoom ≥ 8.
+
+Camera preview: Caltrans HLS via existing hls.js path; JPEG thumb from `currentImageURL`. Still never emit RTSP.
+
+Attribution bar (required): OpenSky / ADSB.lol ODbL / Amtraker / RainViewer / IEM / NWS / NASA FIRMS-GIBS / NIFC WFIGS / OSM as applicable.
+
+---
+
+## 12. Legal / ethics (non-negotiable)
+
+- Masscan / RTSP policy unchanged.
+- AISStream: server-side only; do not put the key in JS.[5]
+- OpenSky: non-commercial unless licensed; cite if you publish.[2]
+- ADSB.lol: ODbL share-alike on derived databases.[4]
+- Amtraker: ODC-By, visible credit, identifying UA.[9]
+- RainViewer: personal/educational; credit + link.[13]
+- Open-Meteo: non-commercial 10k/day on the public instance.[18]
+- GFW: non-commercial.[24]
+- Caltrans / NWS / NIFC / NASA / IEM: US public or explicitly open; still cache politely.
+- Do not scrape MarineTraffic / FlightRadar24 / FlightAware map tiles.
+
+---
+
+## 13. Verification log (this research session)
+
+| Endpoint | Result |
+|---|---|
+| OpenSky `/states/all` NC bbox | 200, 270 aircraft, 0.63 s |
+| ADSB.lol `/v2/lat/35.88/lon/-78.79/dist/50` | 200, 27 aircraft |
+| RainViewer `weather-maps.json` | 200, 13 past frames |
+| RainViewer tile | 200, CORS `*`, 500 req/min header |
+| Open-Meteo forecast RDU | 200, 28.4 °C, code 80 |
+| NWS `/alerts/active?area=NC` | 200, 4 alerts, CORS `*` |
+| IEM `nexrad-n0q` tile | 200, CORS `*`, max-age 300 |
+| IEM `sbw.geojson` | 200, 46 polygons |
+| IEM `webcam.geojson` | 200, 3 cams |
+| WFIGS locations count | 635 |
+| WFIGS current perimeters count | 236 |
+| ALERTWest `getCameraDataByLoc` | 200, 6.6 MB |
+| Caltrans D4 CCTV JSON | 200, 749 cams, JPEG + HLS |
+| Amtraker `/v3/trains` | 200, ~197 numbers, 1.3 MB |
+| AviationWeather METAR KRDU | 200 JSON |
+| NHC `CurrentStorms.json` | 200, TS Dolly |
+| OpenRailwayMap tile HEAD | **403** — do not rely on |
+
+---
+
+## 14. Concrete next tickets
+
+**Backend**
+
+1. `FIRMS_DATASET` dual-write NOAA-20/21; document S-NPP sunset.
+2. `aircraft` ingest: ADSB.lol viewport poll + `/api/aircraft`.
+3. `trains` ingest: Amtraker 30 s poll + User-Agent + `/api/trains`.
+4. Camera parser: Caltrans CWWP2 D1–D12 → existing `cameras` rows (`discovery_source=caltrans`).
+5. WFIGS proxy endpoints (locations + current perimeters), 5–15 min cache.
+6. AISStream worker behind `AISSTREAM_API_KEY` (idle if unset, same as FIRMS).
+7. `/api/alerts` 30 s cache of NWS + IEM SBW.
+
+**Frontend**
+
+1. Layer control: radar (IEM/RainViewer), alerts, WFIGS, aircraft, trains, vessels.
+2. Shared point-layer component (id, lat, lon, heading, label).
+3. RainViewer frame JSON → animated overlay.
+4. Caltrans HLS in existing preview pane.
+5. Attribution strip.
+
+**Do not start with:** Transitland, GFW, AISHub, OpenSky OAuth (until ADSB.lol is live and still needs a fallback).
+
+## Sources
+
+[1] https://openskynetwork.github.io/opensky-api/rest.html
+[2] https://openskynetwork.github.io/opensky-api
+[3] https://api.adsb.lol
+[4] https://www.adsb.lol/docs/open-data/api
+[5] https://aisstream.io/documentation
+[6] https://aisstream.io
+[7] https://www.aishub.net
+[8] https://www.aishub.net/api
+[9] https://github.com/piemadd/amtrak
+[10] https://amtraker.com/about
+[11] https://www.transit.land/documentation
+[12] https://www.transit.land/plans-pricing
+[13] https://www.rainviewer.com/api.html
+[14] https://www.rainviewer.com/api/weather-maps-api.html
+[15] https://mesonet.agron.iastate.edu/ogc
+[16] https://mesonet.agron.iastate.edu/api
+[17] https://www.weather.gov/documentation/services-web-API
+[18] https://open-meteo.com
+[19] https://open-meteo.com/en/docs
+[20] https://firms.modaps.eosdis.nasa.gov/api/area
+[21] https://data-nifc.opendata.arcgis.com/datasets/nifc::wfigs-current-interagency-fire-perimeters/about
+[22] https://aviationweather.gov/data/api
+[23] https://mesonet.agron.iastate.edu/geojson/webcam.py?help
+[24] https://globalfishingwatch.org/our-apis/documentation
+[25] https://openrailwaymap.org
+[26] https://opensky-network.org/about/faq
+[27] https://cwwp2.dot.ca.gov/vm/iframemap.htm
				`@ -0,0 +1 @@`
				`Subproject commit 6f0fc8dbe3fdd5d0a8c4abb444dd751acd4f9186`