Builder brief for backend + frontend. Researched 2026-08-27. Every endpoint below was either live-probed from this machine or taken from the provider’s current docs. Prefer **free, no-key, CORS-open** sources first. Keys are called out explicitly.
This is **not** a camera-discovery change. Existing camera rules still apply: never emit `rtsp://` hrefs; camera pins go through `/api/cameras/{id}/snapshot`; HTTP directory cams use `/stream` MJPEG.
| News / RSS / GDELT / USGS quakes | Ingest | Out of scope for this brief. |
**Action for existing fire ingest:** NASA will stop Suomi NPP product delivery on **2026-11-01**. Switch `FIRMS_DATASET` from `VIIRS_SNPP_NRT` to `VIIRS_NOAA20_NRT` and/or `VIIRS_NOAA21_NRT` before then.[20]
---
## 1. Architecture rules (backend + frontend)
Moving objects (aircraft, vessels, trains) and alerts are **vectors**. Radar / satellite / GIBS fire tiles are **rasters**.
| Kind | Where it runs | Persist? | API shape |
|---|---|---|---|
| Radar, GOES, GIBS, OpenRailwayMap tiles | **Browser**`L.tileLayer` (CORS `*` on the recommended ones) | No | Tile URL template |
| Aircraft / trains / last-known ships | **Backend poll** → last-known row → `GET /api/{layer}?bbox=` | Last-known only (Timescale, 1-day chunks, drop after 24–72 h) | JSON array of points |
| AIS live stream | **Backend WebSocket** (aisstream forbids browsers)[5] | Last-known + optional 15-min trail | Same `/api/vessels` |
| Official JPEG/HLS cameras | Existing `cameras` scraper | Yes, same table | Existing `/stream` + HLS |
**Do not** hit third-party APIs from the Leaflet client when: (a) an API key would leak, (b) CORS is closed, (c) the provider rate-limits by IP and many map users would share one Pi egress. Proxy those through FastAPI.
**Pi / home-uplink constraints:** viewport bbox only. Never poll global ADS-B or global AIS. Cap aircraft poll to 5–10 s, trains 15–30 s, FIRMS already 900 s. Do not store 1 Hz AIS history.
### 3.1 ADSB.lol — **recommended primary for the map**
| | |
|---|---|
| Cost / key | Free, no key today. Docs say a feeder key *may* be required in the future; contact them before treating this as a production SLA.[3] |
| License | ODbL 1.0 (same family as OSM; share-alike on the database).[4] |
| CORS | Treat as **backend-only**. Do not assume browser CORS. |
| Live probe 2026-08-27 | `GET https://api.adsb.lol/v2/lat/35.88/lon/-78.79/dist/50` → HTTP 200, 27 aircraft near RDU. |
Useful endpoints:[3]
```
GET /v2/lat/{lat}/lon/{lon}/dist/{radius_nm} # radius cap 250 nm
GET /v2/point/{lat}/{lon}/{radius}
GET /v2/sqk/{squawk} # e.g. 7700/7600/7500
GET /v2/mil
GET /v2/hex/{icao}
GET /v2/callsign/{callsign}
```
Sample fields from live payload: `hex, flight, r, t, lat, lon, alt_baro, gs, track, squawk, emergency, category, seen_pos`.
**Backend:** poll the **map viewport** (center + radius from bbox, clamp ≤ 150 nm) every 5–10 s. Dedup on `hex`. Do not poll `/v2/mil` on a timer unless the user toggles a mil filter.
**Frontend:** circleMarker or rotating icon, tooltip = `flight`/`hex`/`alt`. Cluster above ~200 markers. Color by altitude or `emergency != none`.
### 3.2 OpenSky Network — research-grade fallback
| | |
|---|---|
| Cost / key | Free for non-commercial / research. Anonymous OK. Authenticated uses OAuth2 **client credentials** (not basic auth). Create `client_id` + `client_secret` on the account page.[1][26] |
| Terms | Cite the OpenSky paper + URL for publications. Commercial needs a license. They may block AWS/hyperscaler IPs.[2] |
| Live probe | `GET /api/states/all?lamin=33.8&lomin=-84.5&lamax=36.6&lomax=-75.4` → HTTP 200, **270** state vectors over NC in 0.63 s. |
Root: `https://opensky-network.org/api`[1]
```
GET /states/all?lamin=&lomin=&lamax=&lomax=&extended=1
```
State vector indexes: `0 icao24, 1 callsign, 2 origin_country, 5 lon, 6 lat, 7 baro_alt m, 8 on_ground, 9 velocity m/s, 10 true_track, 14 squawk`.[1]
Credits (independent buckets for `/states/*`, `/tracks/*`, `/flights/*`):[1]
| Tier | Credits / refill |
|---|---|
| Anonymous | 400 / day |
| Standard user (OAuth) | 4,000 / day |
| Active feeder ≥30% uptime | 8,000 / day |
`/states/all` costs **1–4 credits** by bbox area (≤25 sq° = 1 credit; global = 4). Exhaustion → HTTP 429 + `X-Rate-Limit-Retry-After-Seconds`.[1]
Anonymous: live only, 10 s time resolution. Authenticated: up to 1 hour of history, 5 s resolution.[1]
Token:
```
POST https://auth.opensky-network.org/auth/realms/opensky-network/protocol/openid-connect/token
Authorization: Bearer <access_token> # ~30 min TTL
```
**Backend:** use OpenSky as fallback when ADSB.lol 429s, or for a “research” toggle. Always send a bbox. NC-sized box is ~25 sq° → 1 credit; 10 s poll as anonymous = 8,640 credits/day → **will 429**. Authenticated 4,000/day ≈ one poll every ~22 s if cost=1. Prefer ADSB.lol for the live map; keep OpenSky for sparse/historical.
**Do not** run this from AWS/the Pi if OpenSky starts blocking the IP; the docs warn about hyperscalers.[2]
### 3.3 AviationWeather.gov METARs (context, not tracks)
GET https://aviationweather.gov/api/data/metar?ids=KRDU&format=json
```
Live probe: KRDU thunderstorm METAR returned JSON with `lat/lon/temp/rawOb`. Use as airport popup, not as an aircraft layer. Prefer the gzip cache files for bulk.
### 3.4 Skip
- ADS-B Exchange — paid gateway / API key.
- FlightAware / AeroAPI — paid.
- OpenSky Trino historical — overkill for the live map.
---
## 4. Boat / AIS traffic
### 4.1 AISStream — **recommended live ships**
| | |
|---|---|
| Cost / key | Free. Sign in with GitHub, create key at `/account`. Shown once; rotate from Account.[5][6] |
| Transport | `wss://stream.aisstream.io/v0/stream` |
| Browser | **Forbidden.** Connect from the FastAPI/ingest container and proxy a downsampled last-known feed.[5] |
| Limits | 3 connections / account, 3 / IP. Subscribe JSON **within 3 seconds** or the socket closes. Subscription replace ≤ 1/s. No SLA, no replay.[5] |
Frames are **binary WebSocket** containing UTF-8 JSON. Decode bytes, then `json.loads`.[5]
Position envelope (docs + site examples): `MessageType`, `MetaData.{MMSI,ShipName,Latitude,Longitude}`, `Message.PositionReport.{Sog,Cog,TrueHeading,NavigationalStatus}`.[5][6]
**Backend:** one long-lived WS in the `ingest` profile. Filter to the dashboard’s configured bbox (default CONUS coasts + Great Lakes, not world). Upsert last-known by MMSI. Expose `GET /api/vessels?bbox=`. Reconnect with jittered backoff; resend full subscription in <3s.Donotfantherawstreamtobrowsers.
**Frontend:** circleMarker, color by nav status / SOG, tooltip name+MMSI. Optional 5-minute trail from in-memory ring, not DB.
### 4.2 AISHub — only if you feed a receiver
Free **if you contribute a live AIS UDP feed**. Then JSON/XML/CSV at most **once per minute**.[7][8]
Not useful on the Pi unless Lance later stands up an AIS dongle. Keep as a footnote.
### 4.3 Skip / later
- MarineTraffic REST — commercial.
- Global Fishing Watch — free **non-commercial** token, fishing-effort tiles / vessel identity, not a general AIS live layer.[24] Optional Phase D for a “fishing effort” overlay.
- AISHub without a feeder — no data.
---
## 5. Train traffic
There is **no free global live train API**. US intercity is solvable; European freight is not, cheaply.
### 5.1 Amtraker — **recommended US passenger trains**
Community API that rehosts Amtrak + Brightline + VIA Rail. Free. **Identifying `User-Agent` required or the request is blocked.**[9] Data license ODC-By 1.0; attribute “Amtraker” on the map (collapsed OSM-style credit is OK).[9][10]
Live probe: `GET https://api.amtraker.com/v3/trains` and `https://api-v3.amtraker.com/v3/trains` both HTTP 200, **~1.3 MB**, **197 train numbers**. Sample: Sunset Limited `#1`, `lat/lon`, `heading`, `stations[]`, `iconColor`.
```
GET https://api.amtraker.com/v3/trains
GET https://api.amtraker.com/v3/trains/:trainId # e.g. 5-9
GET https://api.amtraker.com/v3/stations
GET https://api.amtraker.com/v3/stale # stale=true if Amtrak >15 min old
```
Poll **15–30 s**. Response is `{ "1": [Train, ...], "5": [...], ... }` — flatten to one row per `trainID`. Do not hammer; the author asks not to spam.[10]
**Frontend:** train icon, color from `iconColor` (or compute lateness). Popup: route, number, next station, delay.
Use later if you want commuter rail / subway vehicle positions (LA Metro, MTA, etc.). 10k/month is only ~13 queries/hour — **cannot** poll every agency every 15 s. Cache GTFS-RT protobufs yourself from the **agency’s own feed URL** (those are usually free, no Transitland meter). Transitland is a directory; the live positions should come from the agency GTFS-RT URL listed in the Atlas.
### 5.3 OpenRailwayMap — infrastructure tiles only
`https://tiles.openrailwaymap.org/{style}/{z}/{x}/{y}.png` — OSM-derived tracks, not live trains.[25] This probe’s HEAD got **HTTP 403** (UA/hotlink?). Treat as optional overlay; do not depend on it. No vehicle data.
### 5.4 Skip
- Official Amtrak Track-A-Train JSON — unofficial reverse-engineering; Amtraker already does this legally enough for a hobby map, with a documented UA policy.
- Deutsche Bahn / Network Rail — free-ish developer portals but EU-centric and key + ToS heavy. Out of scope unless the map goes global-rail.
---
## 6. Open video / camera feeds (official public only)
Districts 1–12 exist with the same schema (D4 verified). Backend: new parser next to `parse_alertwest_json`, skip `inService != true`, store JPEG as `snapshot_url`, HLS as stream URL (dashboard already vendors `hls.min.js`). Frontend: same camera popup; prefer JPEG for map thumbs, HLS in the preview pane. Never RTSP.
Politeness: scrape hourly like other sources (`CAMERA_SCRAPE_INTERVAL=3600`). Do not scrape every JPEG every cycle — store URLs, let the snapshot cache fetch on demand.
### 6.3 IEM webcams — small, Iowa-centric bonus
```
GET https://mesonet.agron.iastate.edu/geojson/webcam.geojson
```
No key, CORS permissive.[23] Live probe: HTTP 200, **3** cameras at probe time (not a national network). Fields: `cid, name, imgurl, utc_valid, state`. Nice extra source, low value vs ALERTWest/Caltrans.
### 6.4 YouTube live (already in Live-Environment-Streams)
Do not call the YouTube Data API unless you want search. Embedding existing stream URLs from the GeoJSON is enough. YouTube Data API **requires a Google key** and quota.
Live probe: **635** current locations. Sample: `Foster Bridge`, 675 acres, `US-FL`, 100% contained, cause Human. `FireDiscoveryDateTime` is **epoch ms**.
Poll 5–15 min (NIFC says related perimeter service refreshes ~5 min).[21] Backend can proxy `GET /api/fire-incidents` so the keyless URL is cached. Frontend: labeled point, popup acres / contained %.
Some similarly named services (`Current_WildlandFire_Locations`) return **Token Required** — stick to `WFIGS_Incident_Locations_Current`.
### 7.2 WFIGS current perimeters — **recommended polygon overlay**
Live probe: **236** current perimeters. Historic `WFIGS_Interagency_Perimeters` counted **41,078** — do not fetch that unfiltered. Use Current only, or `resultRecordCount` + bbox `geometry` + `spatialRel=esriSpatialRelIntersects`.
Authoritative US wildland-fire polygons; not every incident has a perimeter. Refresh ~5 min, fall-off rules drop stale small fires.[21]
**Frontend:** `L.geoJSON` fill red/orange by acres. Click for name/acres. This is the layer that makes FIRMS dots make sense.
WMS alternative: `https://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0q.cgi?` — TMS is simpler for Leaflet.
**Frontend:** use IEM as default when the map is over CONUS (higher res than RainViewer z7); RainViewer when zoomed outside the US or as a global fallback.
### 8.3 NWS API — alerts / forecasts (no radar tiles)
`https://api.weather.gov` — US public domain, no fee. **User-Agent required** (app + contact). Rate limit unpublished; 429s clear in ~5 s. CORS `*`.[17]
Live probe: `GET /alerts/active?area=NC` → HTTP 200 GeoJSON, 4 alerts, CORS `*`.
```
GET https://api.weather.gov/alerts/active?area=NC
GET https://api.weather.gov/alerts/active?point={lat},{lon}
GET https://api.weather.gov/points/{lat},{lon} # then follow forecast URLs
```
**Frontend can hit this directly** (CORS open) **or** backend can cache 30 s to protect the Pi if many tabs are open. Prefer backend cache. Draw `features[].geometry` with color by `properties.severity` / `event`.
### 8.4 IEM storm-based warnings GeoJSON
```
GET https://mesonet.agron.iastate.edu/geojson/sbw.geojson
```
Live probe: **46** current polygons. Sample: Severe Thunderstorm Warning, WFO BGM. Good companion to NWS alerts (polygon-accurate).
### 8.5 Open-Meteo — point forecast, not radar
No key for non-commercial **<10,000calls/day**.CCBY4.0data,attributionrequired.Commercialneedsaplanorself-host.[18][19]
```
GET https://api.open-meteo.com/v1/forecast?latitude=35.88&longitude=-78.79¤t=temperature_2m,precipitation,weather_code,wind_speed_10m
```
Live probe: RDU 28.4 °C, weather_code 80 (rain showers). Use for click-to-forecast popups, not a map overlay. Cache by grid cell.
### 8.6 NHC active storms
```
GET https://www.nhc.noaa.gov/CurrentStorms.json
```
Live probe: HTTP 200, `activeStorms[]` — Tropical Storm Dolly (`al042026`, 13.6N 38.7W, 35 kt). No key. Plot as a hurricane marker + optional NHC cone GeoJSON (separate NHC GIS products). Nice extra during Atlantic season.
---
## 9. Keys / env vars to add
Store new secrets in the existing Keys UI (`api_keys` table) **and**`.env` fallback, same planned pattern as FIRMS.
Amtraker: no key, but set `OSINT_USER_AGENT` to something like `osint-dashboard/1.0 (lancewalters94@gmail.com)` — already have `USER_AGENT` in `camera_config.py`; reuse it on **all** outbound HTTP.
NWS: same User-Agent header is mandatory.[17]
---
## 10. Suggested API surface (backend engineer)
Keep FIRMS `GET /api/fires`. Add:
```
GET /api/aircraft?bbox=minlon,minlat,maxlon,maxlat
GET /api/vessels?bbox=...
GET /api/trains?bbox=...
GET /api/fire-incidents?bbox=... # WFIGS points
GET /api/fire-perimeters?bbox=... # WFIGS polygons (or proxy GeoJSON)
GET /api/alerts?area=NC|&bbox=... # cached NWS + IEM SBW
GET /api/map/radar # { provider, tileUrl, frames[] } for RainViewer metadata
GET /api/map/layers # already have GIBS; extend with radar/fire tile templates
```
All vector endpoints: default `limit=2000`, require bbox except trains (Amtrak is ~200 rows total). Return arrays of `{id, lat, lon, heading, speed, label, extra}` so the frontend can share one marker renderer.
Do **not** proxy RainViewer/IEM/GIBS tiles through the Pi (bandwidth). Return the template; browser fetches tiles.
AISStream stays inside `ingest` profile (needs a long-lived process, like FIRMS). Aircraft/trains can live in `run_ingester.py` loops.
---
## 11. Suggested UI (frontend bot)
Layer toggles on the existing Leaflet map (same pattern as GIBS + fire heatmap):
| Toggle | Default | Source |
|---|---|---|
| Radar | on (CONUS) | IEM `nexrad-n0q` if view intersects US, else RainViewer |
| NWS alerts | on | polygons |
| FIRMS heat | existing | existing |
| WFIGS perimeters | on | polygons |
| WFIGS incidents | off | points |
| Aircraft | on | `/api/aircraft` |
| Vessels | off until AIS key | `/api/vessels` |
| Trains | on | `/api/trains` |
| Cameras | existing | existing |
Viewport-driven: on `moveend` (debounced 300 ms), refetch aircraft/vessels for the new bbox. Do not refetch radar tiles except RainViewer frame JSON every 5 min.
Marker performance: canvas renderer (`L.canvas()`), cluster at zoom <7,hidelabelsuntilzoom≥8.
Camera preview: Caltrans HLS via existing hls.js path; JPEG thumb from `currentImageURL`. Still never emit RTSP.
Attribution bar (required): OpenSky / ADSB.lol ODbL / Amtraker / RainViewer / IEM / NWS / NASA FIRMS-GIBS / NIFC WFIGS / OSM as applicable.
