# Free data streams for the OSINT dashboard

Builder brief for backend + frontend. Researched 2026-08-27. Every endpoint below was either live-probed from this machine or taken from the provider’s current docs. Prefer **free, no-key, CORS-open** sources first. Keys are called out explicitly.

This is **not** a camera-discovery / masscan change. Existing camera rules still apply: never emit `rtsp://` hrefs; masscan pins go through `/api/cameras/{id}/snapshot`; HTTP directory cams use `/stream` MJPEG.

---

## 0. What is already in the repo (do not rebuild)

| Layer | Status | Notes |
|---|---|---|
| NASA FIRMS VIIRS hotspots | Ingested (`app/fire_sources.py` → NATS `events.fire` → `fires` hypertable → `GET /api/fires`) | Needs free `FIRMS_MAP_KEY`. See `docs/firms.md`. |
| NASA GIBS basemaps | Frontend tiles via `app/gibs_map.py` | No key. CORS `*`. |
| GIBS VIIRS thermal tiles | Documented, not wired as overlay | Same GIBS stack; no key. |
| Cameras | Scraper + masscan → `cameras` table | Defaults already include ALERTWest JPEGs + Live-Environment-Streams HLS/YouTube GeoJSON. |
| News / RSS / GDELT / USGS quakes | Ingest | Out of scope for this brief. |

**Action for existing fire ingest:** NASA will stop Suomi NPP product delivery on **2026-11-01**. Switch `FIRMS_DATASET` from `VIIRS_SNPP_NRT` to `VIIRS_NOAA20_NRT` and/or `VIIRS_NOAA21_NRT` before then.[20]

---

## 1. Architecture rules (backend + frontend)

Moving objects (aircraft, vessels, trains) and alerts are **vectors**. Radar / satellite / GIBS fire tiles are **rasters**.

| Kind | Where it runs | Persist? | API shape |
|---|---|---|---|
| Radar, GOES, GIBS, OpenRailwayMap tiles | **Browser** `L.tileLayer` (CORS `*` on the recommended ones) | No | Tile URL template |
| Aircraft / trains / last-known ships | **Backend poll** → last-known row → `GET /api/{layer}?bbox=` | Last-known only (Timescale, 1-day chunks, drop after 24–72 h) | JSON array of points |
| AIS live stream | **Backend WebSocket** (aisstream forbids browsers)[5] | Last-known + optional 15-min trail | Same `/api/vessels` |
| NWS / SPC polygons | Backend poll **or** browser (NWS CORS `*`)[17] | Optional 24 h | GeoJSON FeatureCollection |
| Official JPEG/HLS cameras | Existing `cameras` scraper | Yes, same table | Existing `/stream` + HLS |

**Do not** hit third-party APIs from the Leaflet client when: (a) an API key would leak, (b) CORS is closed, (c) the provider rate-limits by IP and many map users would share one Pi egress. Proxy those through FastAPI.

**Pi / home-uplink constraints:** viewport bbox only. Never poll global ADS-B or global AIS. Cap aircraft poll to 5–10 s, trains 15–30 s, FIRMS already 900 s. Do not store 1 Hz AIS history.

Suggested NATS subjects (match `events.fire`): `events.aircraft`, `events.vessel`, `events.train`, `events.alert`.

Suggested last-known tables (natural keys):

```
aircraft(icao24 PK, callsign, lat, lon, alt_baro_m, gs_kt, track_deg, squawk, src, seen_at)
vessels(mmsi PK, name, lat, lon, sog_kt, cog_deg, navstat, dest, src, seen_at)
trains(train_id PK, route, lat, lon, heading, late_min, src, seen_at)
```

---

## 2. Build order (what to ship first)

### Phase A — frontend-only, no new tables (1–2 days)

1. RainViewer radar overlay (global, no key, CORS `*`).[13][14]
2. IEM NEXRAD CONUS overlay as US-quality alternative (no key, CORS `*`).[15]
3. GIBS `VIIRS_SNPP_Thermal_Anomalies_375m_All` fire-pixel overlay (already documented in `docs/firms.md`).
4. NWS active alerts GeoJSON overlay (no key, User-Agent required, CORS `*`).[17]
5. IEM current storm-based warnings GeoJSON.[16]

### Phase B — backend last-known overlays

6. ADSB.lol viewport aircraft (no key today).[3]
7. OpenSky bbox as fallback / research-grade (anonymous works; OAuth recommended).[1]
8. Amtraker trains (no key; **must** send identifying `User-Agent`).[9]
9. WFIGS incident points + current perimeters (no key).[21]
10. Caltrans CWWP2 cameras into existing `cameras` scraper (JPEG + HLS).[27]

### Phase C — free keys, streams

11. AISStream WebSocket (free GitHub-login key; **server-side only**).[5]
12. Optional OpenSky OAuth client for 4,000 credits/day vs 400 anonymous.[1]
13. Optional Transitland key later for multi-agency GTFS-RT (10k REST/month free).[12]

Skip paid: MarineTraffic, ADS-B Exchange, FlightAware, TrafficLand, OpenWeatherMap One Call paid tiers.

---

## 3. Air traffic

### 3.1 ADSB.lol — **recommended primary for the map**

| | |
|---|---|
| Cost / key | Free, no key today. Docs say a feeder key *may* be required in the future; contact them before treating this as a production SLA.[3] |
| License | ODbL 1.0 (same family as OSM; share-alike on the database).[4] |
| CORS | Treat as **backend-only**. Do not assume browser CORS. |
| Live probe 2026-08-27 | `GET https://api.adsb.lol/v2/lat/35.88/lon/-78.79/dist/50` → HTTP 200, 27 aircraft near RDU. |

Useful endpoints:[3]

```
GET /v2/lat/{lat}/lon/{lon}/dist/{radius_nm}   # radius cap 250 nm
GET /v2/point/{lat}/{lon}/{radius}
GET /v2/sqk/{squawk}                          # e.g. 7700/7600/7500
GET /v2/mil
GET /v2/hex/{icao}
GET /v2/callsign/{callsign}
```

Sample fields from live payload: `hex, flight, r, t, lat, lon, alt_baro, gs, track, squawk, emergency, category, seen_pos`.

**Backend:** poll the **map viewport** (center + radius from bbox, clamp ≤ 150 nm) every 5–10 s. Dedup on `hex`. Do not poll `/v2/mil` on a timer unless the user toggles a mil filter.

**Frontend:** circleMarker or rotating icon, tooltip = `flight`/`hex`/`alt`. Cluster above ~200 markers. Color by altitude or `emergency != none`.

### 3.2 OpenSky Network — research-grade fallback

| | |
|---|---|
| Cost / key | Free for non-commercial / research. Anonymous OK. Authenticated uses OAuth2 **client credentials** (not basic auth). Create `client_id` + `client_secret` on the account page.[1][26] |
| Terms | Cite the OpenSky paper + URL for publications. Commercial needs a license. They may block AWS/hyperscaler IPs.[2] |
| Live probe | `GET /api/states/all?lamin=33.8&lomin=-84.5&lamax=36.6&lomax=-75.4` → HTTP 200, **270** state vectors over NC in 0.63 s. |

Root: `https://opensky-network.org/api`[1]

```
GET /states/all?lamin=&lomin=&lamax=&lomax=&extended=1
```

State vector indexes: `0 icao24, 1 callsign, 2 origin_country, 5 lon, 6 lat, 7 baro_alt m, 8 on_ground, 9 velocity m/s, 10 true_track, 14 squawk`.[1]

Credits (independent buckets for `/states/*`, `/tracks/*`, `/flights/*`):[1]

| Tier | Credits / refill |
|---|---|
| Anonymous | 400 / day |
| Standard user (OAuth) | 4,000 / day |
| Active feeder ≥30% uptime | 8,000 / day |

`/states/all` costs **1–4 credits** by bbox area (≤25 sq° = 1 credit; global = 4). Exhaustion → HTTP 429 + `X-Rate-Limit-Retry-After-Seconds`.[1]

Anonymous: live only, 10 s time resolution. Authenticated: up to 1 hour of history, 5 s resolution.[1]

Token:

```
POST https://auth.opensky-network.org/auth/realms/opensky-network/protocol/openid-connect/token
grant_type=client_credentials&client_id=&client_secret=
Authorization: Bearer <access_token>   # ~30 min TTL
```

**Backend:** use OpenSky as fallback when ADSB.lol 429s, or for a “research” toggle. Always send a bbox. NC-sized box is ~25 sq° → 1 credit; 10 s poll as anonymous = 8,640 credits/day → **will 429**. Authenticated 4,000/day ≈ one poll every ~22 s if cost=1. Prefer ADSB.lol for the live map; keep OpenSky for sparse/historical.

**Do not** run this from AWS/the Pi if OpenSky starts blocking the IP; the docs warn about hyperscalers.[2]

### 3.3 AviationWeather.gov METARs (context, not tracks)

No key. User-Agent required. **100 req/min**. **No CORS** — backend only.[22]

```
GET https://aviationweather.gov/api/data/metar?ids=KRDU&format=json
```

Live probe: KRDU thunderstorm METAR returned JSON with `lat/lon/temp/rawOb`. Use as airport popup, not as an aircraft layer. Prefer the gzip cache files for bulk.

### 3.4 Skip

- ADS-B Exchange — paid gateway / API key.
- FlightAware / AeroAPI — paid.
- OpenSky Trino historical — overkill for the live map.

---

## 4. Boat / AIS traffic

### 4.1 AISStream — **recommended live ships**

| | |
|---|---|
| Cost / key | Free. Sign in with GitHub, create key at `/account`. Shown once; rotate from Account.[5][6] |
| Transport | `wss://stream.aisstream.io/v0/stream` |
| Browser | **Forbidden.** Connect from the FastAPI/ingest container and proxy a downsampled last-known feed.[5] |
| Limits | 3 connections / account, 3 / IP. Subscribe JSON **within 3 seconds** or the socket closes. Subscription replace ≤ 1/s. No SLA, no replay.[5] |
| Compression | Negotiate `permessage-deflate`. Uncompressed connections get bandwidth caps starting **September 2026**.[5] |

Subscribe (bbox is **\[\[lat, lon\], \[lat, lon\]\]** — not lon/lat):[5]

```json
{
  "APIKey": "<AISSTREAM_API_KEY>",
  "BoundingBoxes": [[[33.8, -84.5], [36.6, -75.4]]],
  "FilterMessageTypes": [
    "PositionReport",
    "StandardClassBPositionReport",
    "ExtendedClassBPositionReport",
    "ShipStaticData"
  ]
}
```

Frames are **binary WebSocket** containing UTF-8 JSON. Decode bytes, then `json.loads`.[5]

Position envelope (docs + site examples): `MessageType`, `MetaData.{MMSI,ShipName,Latitude,Longitude}`, `Message.PositionReport.{Sog,Cog,TrueHeading,NavigationalStatus}`.[5][6]

**Backend:** one long-lived WS in the `ingest` profile. Filter to the dashboard’s configured bbox (default CONUS coasts + Great Lakes, not world). Upsert last-known by MMSI. Expose `GET /api/vessels?bbox=`. Reconnect with jittered backoff; resend full subscription in <3 s. Do not fan the raw stream to browsers.

**Frontend:** circleMarker, color by nav status / SOG, tooltip name+MMSI. Optional 5-minute trail from in-memory ring, not DB.

### 4.2 AISHub — only if you feed a receiver

Free **if you contribute a live AIS UDP feed**. Then JSON/XML/CSV at most **once per minute**.[7][8]

```
https://data.aishub.net/ws.php?username=USER&format=1&output=json&latmin=&latmax=&lonmin=&lonmax=
```

`format=1` = human-readable degrees / knots. Faster than once/minute returns empty.[8]

Not useful on the Pi unless Lance later stands up an AIS dongle. Keep as a footnote.

### 4.3 Skip / later

- MarineTraffic REST — commercial.
- Global Fishing Watch — free **non-commercial** token, fishing-effort tiles / vessel identity, not a general AIS live layer.[24] Optional Phase D for a “fishing effort” overlay.
- AISHub without a feeder — no data.

---

## 5. Train traffic

There is **no free global live train API**. US intercity is solvable; European freight is not, cheaply.

### 5.1 Amtraker — **recommended US passenger trains**

Community API that rehosts Amtrak + Brightline + VIA Rail. Free. **Identifying `User-Agent` required or the request is blocked.**[9] Data license ODC-By 1.0; attribute “Amtraker” on the map (collapsed OSM-style credit is OK).[9][10]

Live probe: `GET https://api.amtraker.com/v3/trains` and `https://api-v3.amtraker.com/v3/trains` both HTTP 200, **~1.3 MB**, **197 train numbers**. Sample: Sunset Limited `#1`, `lat/lon`, `heading`, `stations[]`, `iconColor`.

```
GET https://api.amtraker.com/v3/trains
GET https://api.amtraker.com/v3/trains/:trainId     # e.g. 5-9
GET https://api.amtraker.com/v3/stations
GET https://api.amtraker.com/v3/stale               # stale=true if Amtrak >15 min old
```

Poll **15–30 s**. Response is `{ "1": [Train, ...], "5": [...], ... }` — flatten to one row per `trainID`. Do not hammer; the author asks not to spam.[10]

**Frontend:** train icon, color from `iconColor` (or compute lateness). Popup: route, number, next station, delay.

### 5.2 Transitland — multi-agency GTFS-RT (optional, key)

Free plan: **10,000 REST queries/month**, 100k vector tiles/month, 1,000 routing/month. Key via Interline signup (`apikey=` query or header; **not** `Authorization`). Attribution required.[11][12]

```
https://transit.land/api/v2/rest/
```

Use later if you want commuter rail / subway vehicle positions (LA Metro, MTA, etc.). 10k/month is only ~13 queries/hour — **cannot** poll every agency every 15 s. Cache GTFS-RT protobufs yourself from the **agency’s own feed URL** (those are usually free, no Transitland meter). Transitland is a directory; the live positions should come from the agency GTFS-RT URL listed in the Atlas.

### 5.3 OpenRailwayMap — infrastructure tiles only

`https://tiles.openrailwaymap.org/{style}/{z}/{x}/{y}.png` — OSM-derived tracks, not live trains.[25] This probe’s HEAD got **HTTP 403** (UA/hotlink?). Treat as optional overlay; do not depend on it. No vehicle data.

### 5.4 Skip

- Official Amtrak Track-A-Train JSON — unofficial reverse-engineering; Amtraker already does this legally enough for a hobby map, with a documented UA policy.
- Deutsche Bahn / Network Rail — free-ish developer portals but EU-centric and key + ToS heavy. Out of scope unless the map goes global-rail.

---

## 6. Open video / camera feeds (official public only)

Do **not** add Insecam-style random IP cams as a new source. The scraper already has a public list + masscan; this section is **agency-published** JPEG/HLS.

### 6.1 Already wired

- ALERTWest / ALERTCalifornia: `https://api.cdn.prod.alertwest.com/api/getCameraDataByLoc` — live probe HTTP 200, **6.6 MB** JSON with `data.locs` + `data.cams`. Snapshots at `https://img.cdn.prod.alertwest.com/data/img/{cid}/...` (see `parse_alertwest_json`).
- Live-Environment-Streams GeoJSON (HLS / YouTube / JPEG).
- `public-ip-cams` README list.

### 6.2 Caltrans CWWP2 — **add to CAMERA_SOURCE_URLS**

Official CA DOT CCTV. No key.

```
https://cwwp2.dot.ca.gov/data/d{N}/cctv/cctvStatusD{NN}.json
```

Live probe D4: HTTP 200, **2.4 MB**, **749 cameras**.

Each `data[].cctv`:

- `location.{latitude,longitude,locationName,nearbyPlace,district,route,county,direction}`
- `inService` (`"true"`/`"false"`)
- `imageData.static.currentImageURL` — JPEG, updates ~5 s (`https://cwwp2.dot.ca.gov/data/d4/cctv/image/...jpg`)
- `imageData.streamingVideoURL` — **HLS** `https://wzmedia.dot.ca.gov/D4/....stream/playlist.m3u8`

Districts 1–12 exist with the same schema (D4 verified). Backend: new parser next to `parse_alertwest_json`, skip `inService != true`, store JPEG as `snapshot_url`, HLS as stream URL (dashboard already vendors `hls.min.js`). Frontend: same camera popup; prefer JPEG for map thumbs, HLS in the preview pane. Never RTSP.

Politeness: scrape hourly like other sources (`CAMERA_SCRAPE_INTERVAL=3600`). Do not scrape every JPEG every cycle — store URLs, let the snapshot cache fetch on demand.

### 6.3 IEM webcams — small, Iowa-centric bonus

```
GET https://mesonet.agron.iastate.edu/geojson/webcam.geojson
```

No key, CORS permissive.[23] Live probe: HTTP 200, **3** cameras at probe time (not a national network). Fields: `cid, name, imgurl, utc_valid, state`. Nice extra source, low value vs ALERTWest/Caltrans.

### 6.4 YouTube live (already in Live-Environment-Streams)

Do not call the YouTube Data API unless you want search. Embedding existing stream URLs from the GeoJSON is enough. YouTube Data API **requires a Google key** and quota.

### 6.5 Skip

- Insecam / random “public IP cam” aggregators — ToS / privacy / already covered by masscan ethics.
- TrafficLand, EarthCam commercial APIs.
- SkylineWebcams — scraping, not an API.

---

## 7. Fire maps (beyond current FIRMS points)

FIRMS CSV = hotspot **points** (already ingested). WFIGS = **incident identity + perimeters**. GIBS = **pixel overlay**. Use all three; they answer different questions.

### 7.1 WFIGS current incident locations — **recommended vector layer**

No key. ArcGIS FeatureServer, GeoJSON.

```
https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Incident_Locations_Current/FeatureServer/0/query
  ?where=1=1&outFields=IncidentName,IncidentSize,FireDiscoveryDateTime,POOState,PercentContained,IncidentTypeCategory,FireCause
  &outSR=4326&f=geojson
```

Live probe: **635** current locations. Sample: `Foster Bridge`, 675 acres, `US-FL`, 100% contained, cause Human. `FireDiscoveryDateTime` is **epoch ms**.

Poll 5–15 min (NIFC says related perimeter service refreshes ~5 min).[21] Backend can proxy `GET /api/fire-incidents` so the keyless URL is cached. Frontend: labeled point, popup acres / contained %.

Some similarly named services (`Current_WildlandFire_Locations`) return **Token Required** — stick to `WFIGS_Incident_Locations_Current`.

### 7.2 WFIGS current perimeters — **recommended polygon overlay**

```
https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Interagency_Perimeters_Current/FeatureServer/0/query
  ?where=1=1&outFields=poly_IncidentName,poly_GISAcres,attr_IncidentSize,attr_PercentContained,attr_FireDiscoveryDateTime
  &outSR=4326&f=geojson
```

Live probe: **236** current perimeters. Historic `WFIGS_Interagency_Perimeters` counted **41,078** — do not fetch that unfiltered. Use Current only, or `resultRecordCount` + bbox `geometry` + `spatialRel=esriSpatialRelIntersects`.

Authoritative US wildland-fire polygons; not every incident has a perimeter. Refresh ~5 min, fall-off rules drop stale small fires.[21]

**Frontend:** `L.geoJSON` fill red/orange by acres. Click for name/acres. This is the layer that makes FIRMS dots make sense.

### 7.3 GIBS thermal tiles (no key, no storage)

Already in `docs/firms.md`. Leaflet:

```
https://gibs.earthdata.nasa.gov/wmts/epsg3857/best/VIIRS_SNPP_Thermal_Anomalies_375m_All/default/{time}/GoogleMapsCompatible_Level{z}/{y}/{x}.png
```

Max zoom ~9. `{time}` = `YYYY-MM-DD`. Same GIBS CORS `*` as existing basemaps. Use as a toggle under FIRMS points.

### 7.4 FIRMS dataset note

Keep the CSV ingest. Add NOAA-20/21 datasets. Ultra-real-time US/Canada is available on those products (seconds-scale for much of US/Canada).[20]

---

## 8. Weather radar + alerts

### 8.1 RainViewer — **recommended global radar overlay**

No key. Personal / educational / small community. Attribution: “Weather data by RainViewer” + link.[13]

Live probe: `GET https://api.rainviewer.com/public/weather-maps.json` → version `2.0`, host `https://tilecache.rainviewer.com`, **13** past frames (~2 h, 10 min steps). Tile HEAD: HTTP 200, CORS `*`, **`x-ratelimit-limit: 500` / 60 s**, burst 300.

Tile URL:[14]

```
{host}{path}/{size}/{z}/{x}/{y}/{color}/{smooth}_{snow}.png
```

Example: `https://tilecache.rainviewer.com/v2/radar/cb581daa2c0f/256/{z}/{x}/{y}/2/1_1.png`

- `{size}` 256 or 512
- `{color}` scheme id (2 is a common default)
- `{smooth}_{snow}` e.g. `1_1`
- **max zoom 7** (coarse vs IEM)

**Frontend:** fetch the JSON every 5 min, set `L.tileLayer` to the latest `past[].path`. Optional animation through `past` frames. No backend.

### 8.2 IEM NEXRAD / MRMS — **recommended CONUS quality radar**

Iowa Environmental Mesonet. No key. Free including commercial, as-is, no SLA.[16] CORS `*` on tiles (live HEAD confirmed).

Current CONUS base reflectivity TMS:[15]

```
https://mesonet.agron.iastate.edu/cache/tile.py/1.0.0/nexrad-n0q/{z}/{x}/{y}.png
```

Cache-Control 5 min. Aliases `mesonet1/2/3.agron.iastate.edu` for parallel browser tile fetches.[15]

Loop (past 5–55 min, step 5): layer `nexrad-n0q-mXXm` e.g. `nexrad-n0q-m05m`.

Also useful:

| Layer | Meaning |
|---|---|
| `nexrad-eet` | Echo tops |
| `q2-hsr` | MRMS seamless hybrid-scan reflectivity |
| `q2-n1p` / `q2-p24h` | MRMS 1 h / 24 h precip |
| `goes_east_conus_ch02` | GOES-East visible |
| `goes_east_conus_ch13` | GOES-East IR |

WMS alternative: `https://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0q.cgi?` — TMS is simpler for Leaflet.

**Frontend:** use IEM as default when the map is over CONUS (higher res than RainViewer z7); RainViewer when zoomed outside the US or as a global fallback.

### 8.3 NWS API — alerts / forecasts (no radar tiles)

`https://api.weather.gov` — US public domain, no fee. **User-Agent required** (app + contact). Rate limit unpublished; 429s clear in ~5 s. CORS `*`.[17]

Live probe: `GET /alerts/active?area=NC` → HTTP 200 GeoJSON, 4 alerts, CORS `*`.

```
GET https://api.weather.gov/alerts/active?area=NC
GET https://api.weather.gov/alerts/active?point={lat},{lon}
GET https://api.weather.gov/points/{lat},{lon}          # then follow forecast URLs
```

**Frontend can hit this directly** (CORS open) **or** backend can cache 30 s to protect the Pi if many tabs are open. Prefer backend cache. Draw `features[].geometry` with color by `properties.severity` / `event`.

### 8.4 IEM storm-based warnings GeoJSON

```
GET https://mesonet.agron.iastate.edu/geojson/sbw.geojson
```

Live probe: **46** current polygons. Sample: Severe Thunderstorm Warning, WFO BGM. Good companion to NWS alerts (polygon-accurate).

### 8.5 Open-Meteo — point forecast, not radar

No key for non-commercial **< 10,000 calls/day**. CC BY 4.0 data, attribution required. Commercial needs a plan or self-host.[18][19]

```
GET https://api.open-meteo.com/v1/forecast?latitude=35.88&longitude=-78.79&current=temperature_2m,precipitation,weather_code,wind_speed_10m
```

Live probe: RDU 28.4 °C, weather_code 80 (rain showers). Use for click-to-forecast popups, not a map overlay. Cache by grid cell.

### 8.6 NHC active storms

```
GET https://www.nhc.noaa.gov/CurrentStorms.json
```

Live probe: HTTP 200, `activeStorms[]` — Tropical Storm Dolly (`al042026`, 13.6N 38.7W, 35 kt). No key. Plot as a hurricane marker + optional NHC cone GeoJSON (separate NHC GIS products). Nice extra during Atlantic season.

---

## 9. Keys / env vars to add

Store new secrets in the existing Keys UI (`api_keys` table) **and** `.env` fallback, same planned pattern as FIRMS.

| Var | Required for | How to get | Cost |
|---|---|---|---|
| `FIRMS_MAP_KEY` | Existing fire CSV | https://firms.modaps.eosdis.nasa.gov/api/map_key_info/ | Free |
| `OPENSKY_CLIENT_ID` / `OPENSKY_CLIENT_SECRET` | OpenSky authenticated | OpenSky account → API Client | Free |
| `AISSTREAM_API_KEY` | Live AIS | GitHub login at aisstream.io/account | Free |
| `TRANSITLAND_API_KEY` | Transitland REST (optional) | Interline free plan | Free 10k/mo |
| `GFW_TOKEN` | Fishing-effort overlay (optional) | globalfishingwatch.org/our-apis/tokens | Free, non-commercial |
| *(none)* | ADSB.lol, Amtraker, RainViewer, IEM, NWS, WFIGS, Caltrans, Open-Meteo, NHC | — | Free |

Amtraker: no key, but set `OSINT_USER_AGENT` to something like `osint-dashboard/1.0 (lancewalters94@gmail.com)` — already have `USER_AGENT` in `camera_config.py`; reuse it on **all** outbound HTTP.

NWS: same User-Agent header is mandatory.[17]

---

## 10. Suggested API surface (backend engineer)

Keep FIRMS `GET /api/fires`. Add:

```
GET /api/aircraft?bbox=minlon,minlat,maxlon,maxlat
GET /api/vessels?bbox=...
GET /api/trains?bbox=...
GET /api/fire-incidents?bbox=...          # WFIGS points
GET /api/fire-perimeters?bbox=...         # WFIGS polygons (or proxy GeoJSON)
GET /api/alerts?area=NC|&bbox=...         # cached NWS + IEM SBW
GET /api/map/radar                         # { provider, tileUrl, frames[] } for RainViewer metadata
GET /api/map/layers                        # already have GIBS; extend with radar/fire tile templates
```

All vector endpoints: default `limit=2000`, require bbox except trains (Amtrak is ~200 rows total). Return arrays of `{id, lat, lon, heading, speed, label, extra}` so the frontend can share one marker renderer.

Do **not** proxy RainViewer/IEM/GIBS tiles through the Pi (bandwidth). Return the template; browser fetches tiles.

AISStream stays inside `ingest` profile (needs a long-lived process, like FIRMS). Aircraft/trains can live in `run_ingester.py` loops.

---

## 11. Suggested UI (frontend bot)

Layer toggles on the existing Leaflet map (same pattern as GIBS + fire heatmap):

| Toggle | Default | Source |
|---|---|---|
| Radar | on (CONUS) | IEM `nexrad-n0q` if view intersects US, else RainViewer |
| NWS alerts | on | polygons |
| FIRMS heat | existing | existing |
| WFIGS perimeters | on | polygons |
| WFIGS incidents | off | points |
| Aircraft | on | `/api/aircraft` |
| Vessels | off until AIS key | `/api/vessels` |
| Trains | on | `/api/trains` |
| Cameras | existing | existing |

Viewport-driven: on `moveend` (debounced 300 ms), refetch aircraft/vessels for the new bbox. Do not refetch radar tiles except RainViewer frame JSON every 5 min.

Marker performance: canvas renderer (`L.canvas()`), cluster at zoom < 7, hide labels until zoom ≥ 8.

Camera preview: Caltrans HLS via existing hls.js path; JPEG thumb from `currentImageURL`. Still never emit RTSP.

Attribution bar (required): OpenSky / ADSB.lol ODbL / Amtraker / RainViewer / IEM / NWS / NASA FIRMS-GIBS / NIFC WFIGS / OSM as applicable.

---

## 12. Legal / ethics (non-negotiable)

- Masscan / RTSP policy unchanged.
- AISStream: server-side only; do not put the key in JS.[5]
- OpenSky: non-commercial unless licensed; cite if you publish.[2]
- ADSB.lol: ODbL share-alike on derived databases.[4]
- Amtraker: ODC-By, visible credit, identifying UA.[9]
- RainViewer: personal/educational; credit + link.[13]
- Open-Meteo: non-commercial 10k/day on the public instance.[18]
- GFW: non-commercial.[24]
- Caltrans / NWS / NIFC / NASA / IEM: US public or explicitly open; still cache politely.
- Do not scrape MarineTraffic / FlightRadar24 / FlightAware map tiles.

---

## 13. Verification log (this research session)

| Endpoint | Result |
|---|---|
| OpenSky `/states/all` NC bbox | 200, 270 aircraft, 0.63 s |
| ADSB.lol `/v2/lat/35.88/lon/-78.79/dist/50` | 200, 27 aircraft |
| RainViewer `weather-maps.json` | 200, 13 past frames |
| RainViewer tile | 200, CORS `*`, 500 req/min header |
| Open-Meteo forecast RDU | 200, 28.4 °C, code 80 |
| NWS `/alerts/active?area=NC` | 200, 4 alerts, CORS `*` |
| IEM `nexrad-n0q` tile | 200, CORS `*`, max-age 300 |
| IEM `sbw.geojson` | 200, 46 polygons |
| IEM `webcam.geojson` | 200, 3 cams |
| WFIGS locations count | 635 |
| WFIGS current perimeters count | 236 |
| ALERTWest `getCameraDataByLoc` | 200, 6.6 MB |
| Caltrans D4 CCTV JSON | 200, 749 cams, JPEG + HLS |
| Amtraker `/v3/trains` | 200, ~197 numbers, 1.3 MB |
| AviationWeather METAR KRDU | 200 JSON |
| NHC `CurrentStorms.json` | 200, TS Dolly |
| OpenRailwayMap tile HEAD | **403** — do not rely on |

---

## 14. Concrete next tickets

**Backend**

1. `FIRMS_DATASET` dual-write NOAA-20/21; document S-NPP sunset.
2. `aircraft` ingest: ADSB.lol viewport poll + `/api/aircraft`.
3. `trains` ingest: Amtraker 30 s poll + User-Agent + `/api/trains`.
4. Camera parser: Caltrans CWWP2 D1–D12 → existing `cameras` rows (`discovery_source=caltrans`).
5. WFIGS proxy endpoints (locations + current perimeters), 5–15 min cache.
6. AISStream worker behind `AISSTREAM_API_KEY` (idle if unset, same as FIRMS).
7. `/api/alerts` 30 s cache of NWS + IEM SBW.

**Frontend**

1. Layer control: radar (IEM/RainViewer), alerts, WFIGS, aircraft, trains, vessels.
2. Shared point-layer component (id, lat, lon, heading, label).
3. RainViewer frame JSON → animated overlay.
4. Caltrans HLS in existing preview pane.
5. Attribution strip.

**Do not start with:** Transitland, GFW, AISHub, OpenSky OAuth (until ADSB.lol is live and still needs a fallback).

## Sources

[1] https://openskynetwork.github.io/opensky-api/rest.html
[2] https://openskynetwork.github.io/opensky-api
[3] https://api.adsb.lol
[4] https://www.adsb.lol/docs/open-data/api
[5] https://aisstream.io/documentation
[6] https://aisstream.io
[7] https://www.aishub.net
[8] https://www.aishub.net/api
[9] https://github.com/piemadd/amtrak
[10] https://amtraker.com/about
[11] https://www.transit.land/documentation
[12] https://www.transit.land/plans-pricing
[13] https://www.rainviewer.com/api.html
[14] https://www.rainviewer.com/api/weather-maps-api.html
[15] https://mesonet.agron.iastate.edu/ogc
[16] https://mesonet.agron.iastate.edu/api
[17] https://www.weather.gov/documentation/services-web-API
[18] https://open-meteo.com
[19] https://open-meteo.com/en/docs
[20] https://firms.modaps.eosdis.nasa.gov/api/area
[21] https://data-nifc.opendata.arcgis.com/datasets/nifc::wfigs-current-interagency-fire-perimeters/about
[22] https://aviationweather.gov/data/api
[23] https://mesonet.agron.iastate.edu/geojson/webcam.py?help
[24] https://globalfishingwatch.org/our-apis/documentation
[25] https://openrailwaymap.org
[26] https://opensky-network.org/about/faq
[27] https://cwwp2.dot.ca.gov/vm/iframemap.htm