osint-dashboard/docs/free-data-streams.md

Free data streams for the OSINT dashboard

Builder brief for backend + frontend. Researched 2026-08-27. Every endpoint below was either live-probed from this machine or taken from the provider’s current docs. Prefer free, no-key, CORS-open sources first. Keys are called out explicitly.

This is not a camera-discovery change. Existing camera rules still apply: never emit rtsp:// hrefs; camera pins go through /api/cameras/{id}/snapshot; HTTP directory cams use /stream MJPEG.

---

0. What is already in the repo (do not rebuild)

Layer	Status	Notes
NASA FIRMS VIIRS hotspots	Ingested (app/fire_sources.py → NATS events.fire → fires hypertable → GET /api/fires)	Needs free FIRMS_MAP_KEY. See docs/firms.md.
NASA GIBS basemaps	Frontend tiles via app/gibs_map.py	No key. CORS *.
GIBS VIIRS thermal tiles	Documented, not wired as overlay	Same GIBS stack; no key.
Cameras	Scraper → cameras table	Defaults already include ALERTWest JPEGs + Live-Environment-Streams HLS/YouTube GeoJSON.
News / RSS / GDELT / USGS quakes	Ingest	Out of scope for this brief.

Action for existing fire ingest: NASA will stop Suomi NPP product delivery on 2026-11-01. Switch FIRMS_DATASET from VIIRS_SNPP_NRT to VIIRS_NOAA20_NRT and/or VIIRS_NOAA21_NRT before then.[20]

---

1. Architecture rules (backend + frontend)

Moving objects (aircraft, vessels, trains) and alerts are vectors. Radar / satellite / GIBS fire tiles are rasters.

Kind	Where it runs	Persist?	API shape
Radar, GOES, GIBS, OpenRailwayMap tiles	Browser L.tileLayer (CORS * on the recommended ones)	No	Tile URL template
Aircraft / trains / last-known ships	Backend poll → last-known row → GET /api/{layer}?bbox=	Last-known only (Timescale, 1-day chunks, drop after 24–72 h)	JSON array of points
AIS live stream	Backend WebSocket (aisstream forbids browsers)[5]	Last-known + optional 15-min trail	Same /api/vessels
NWS / SPC polygons	Backend poll or browser (NWS CORS *)[17]	Optional 24 h	GeoJSON FeatureCollection
Official JPEG/HLS cameras	Existing cameras scraper	Yes, same table	Existing /stream + HLS

Do not hit third-party APIs from the Leaflet client when: (a) an API key would leak, (b) CORS is closed, (c) the provider rate-limits by IP and many map users would share one Pi egress. Proxy those through FastAPI.

Pi / home-uplink constraints: viewport bbox only. Never poll global ADS-B or global AIS. Cap aircraft poll to 5–10 s, trains 15–30 s, FIRMS already 900 s. Do not store 1 Hz AIS history.

Suggested NATS subjects (match events.fire): events.aircraft, events.vessel, events.train, events.alert.

Suggested last-known tables (natural keys):

aircraft(icao24 PK, callsign, lat, lon, alt_baro_m, gs_kt, track_deg, squawk, src, seen_at)
vessels(mmsi PK, name, lat, lon, sog_kt, cog_deg, navstat, dest, src, seen_at)
trains(train_id PK, route, lat, lon, heading, late_min, src, seen_at)

---

2. Build order (what to ship first)

Phase A — frontend-only, no new tables (1–2 days)

1. RainViewer radar overlay (global, no key, CORS *).[13][14]
2. IEM NEXRAD CONUS overlay as US-quality alternative (no key, CORS *).[15]
3. GIBS VIIRS_SNPP_Thermal_Anomalies_375m_All fire-pixel overlay (already documented in docs/firms.md).
4. NWS active alerts GeoJSON overlay (no key, User-Agent required, CORS *).[17]
5. IEM current storm-based warnings GeoJSON.[16]

Phase B — backend last-known overlays

6. ADSB.lol viewport aircraft (no key today).[3]
7. OpenSky bbox as fallback / research-grade (anonymous works; OAuth recommended).[1]
8. Amtraker trains (no key; must send identifying User-Agent).[9]
9. WFIGS incident points + current perimeters (no key).[21]
10. Caltrans CWWP2 cameras into existing cameras scraper (JPEG + HLS).[27]

Phase C — free keys, streams

11. AISStream WebSocket (free GitHub-login key; server-side only).[5]
12. Optional OpenSky OAuth client for 4,000 credits/day vs 400 anonymous.[1]
13. Optional Transitland key later for multi-agency GTFS-RT (10k REST/month free).[12]

Skip paid: MarineTraffic, ADS-B Exchange, FlightAware, TrafficLand, OpenWeatherMap One Call paid tiers.

---

3. Air traffic

3.1 ADSB.lol — recommended primary for the map

Cost / key	Free, no key today. Docs say a feeder key may be required in the future; contact them before treating this as a production SLA.[3]
License	ODbL 1.0 (same family as OSM; share-alike on the database).[4]
CORS	Treat as backend-only. Do not assume browser CORS.
Live probe 2026-08-27	GET https://api.adsb.lol/v2/lat/35.88/lon/-78.79/dist/50 → HTTP 200, 27 aircraft near RDU.

Useful endpoints:[3]

GET /v2/lat/{lat}/lon/{lon}/dist/{radius_nm}   # radius cap 250 nm
GET /v2/point/{lat}/{lon}/{radius}
GET /v2/sqk/{squawk}                          # e.g. 7700/7600/7500
GET /v2/mil
GET /v2/hex/{icao}
GET /v2/callsign/{callsign}

Sample fields from live payload: hex, flight, r, t, lat, lon, alt_baro, gs, track, squawk, emergency, category, seen_pos.

Backend: poll the map viewport (center + radius from bbox, clamp ≤ 150 nm) every 5–10 s. Dedup on hex. Do not poll /v2/mil on a timer unless the user toggles a mil filter.

Frontend: circleMarker or rotating icon, tooltip = flight/hex/alt. Cluster above ~200 markers. Color by altitude or emergency != none.

3.2 OpenSky Network — research-grade fallback

Cost / key	Free for non-commercial / research. Anonymous OK. Authenticated uses OAuth2 client credentials (not basic auth). Create client_id + client_secret on the account page.[1][26]
Terms	Cite the OpenSky paper + URL for publications. Commercial needs a license. They may block AWS/hyperscaler IPs.[2]
Live probe	GET /api/states/all?lamin=33.8&lomin=-84.5&lamax=36.6&lomax=-75.4 → HTTP 200, 270 state vectors over NC in 0.63 s.

Root: https://opensky-network.org/api[1]

GET /states/all?lamin=&lomin=&lamax=&lomax=&extended=1

State vector indexes: 0 icao24, 1 callsign, 2 origin_country, 5 lon, 6 lat, 7 baro_alt m, 8 on_ground, 9 velocity m/s, 10 true_track, 14 squawk.[1]

Credits (independent buckets for /states/*, /tracks/*, /flights/*):[1]

Tier	Credits / refill
Anonymous	400 / day
Standard user (OAuth)	4,000 / day
Active feeder ≥30% uptime	8,000 / day

/states/all costs 1–4 credits by bbox area (≤25 sq° = 1 credit; global = 4). Exhaustion → HTTP 429 + X-Rate-Limit-Retry-After-Seconds.[1]

Anonymous: live only, 10 s time resolution. Authenticated: up to 1 hour of history, 5 s resolution.[1]

Token:

POST https://auth.opensky-network.org/auth/realms/opensky-network/protocol/openid-connect/token
grant_type=client_credentials&client_id=&client_secret=
Authorization: Bearer <access_token>   # ~30 min TTL

Backend: use OpenSky as fallback when ADSB.lol 429s, or for a “research” toggle. Always send a bbox. NC-sized box is ~25 sq° → 1 credit; 10 s poll as anonymous = 8,640 credits/day → will 429. Authenticated 4,000/day ≈ one poll every ~22 s if cost=1. Prefer ADSB.lol for the live map; keep OpenSky for sparse/historical.

Do not run this from AWS/the Pi if OpenSky starts blocking the IP; the docs warn about hyperscalers.[2]

3.3 AviationWeather.gov METARs (context, not tracks)

No key. User-Agent required. 100 req/min. No CORS — backend only.[22]

GET https://aviationweather.gov/api/data/metar?ids=KRDU&format=json

Live probe: KRDU thunderstorm METAR returned JSON with lat/lon/temp/rawOb. Use as airport popup, not as an aircraft layer. Prefer the gzip cache files for bulk.

3.4 Skip

• ADS-B Exchange — paid gateway / API key.
• FlightAware / AeroAPI — paid.
• OpenSky Trino historical — overkill for the live map.

---

4. Boat / AIS traffic

4.1 AISStream — recommended live ships

Cost / key	Free. Sign in with GitHub, create key at /account. Shown once; rotate from Account.[5][6]
Transport	wss://stream.aisstream.io/v0/stream
Browser	Forbidden. Connect from the FastAPI/ingest container and proxy a downsampled last-known feed.[5]
Limits	3 connections / account, 3 / IP. Subscribe JSON within 3 seconds or the socket closes. Subscription replace ≤ 1/s. No SLA, no replay.[5]
Compression	Negotiate permessage-deflate. Uncompressed connections get bandwidth caps starting September 2026.[5]

Subscribe (bbox is lat, lon], [lat, lon — not lon/lat):[5]

{
  "APIKey": "<AISSTREAM_API_KEY>",
  "BoundingBoxes": [[[33.8, -84.5], [36.6, -75.4]]],
  "FilterMessageTypes": [
    "PositionReport",
    "StandardClassBPositionReport",
    "ExtendedClassBPositionReport",
    "ShipStaticData"
  ]
}

Frames are binary WebSocket containing UTF-8 JSON. Decode bytes, then json.loads.[5]

Position envelope (docs + site examples): MessageType, MetaData.{MMSI,ShipName,Latitude,Longitude}, Message.PositionReport.{Sog,Cog,TrueHeading,NavigationalStatus}.[5][6]

Backend: one long-lived WS in the ingest profile. Filter to the dashboard’s configured bbox (default CONUS coasts + Great Lakes, not world). Upsert last-known by MMSI. Expose GET /api/vessels?bbox=. Reconnect with jittered backoff; resend full subscription in <3 s. Do not fan the raw stream to browsers.

Frontend: circleMarker, color by nav status / SOG, tooltip name+MMSI. Optional 5-minute trail from in-memory ring, not DB.

4.2 AISHub — only if you feed a receiver

Free if you contribute a live AIS UDP feed. Then JSON/XML/CSV at most once per minute.[7][8]

https://data.aishub.net/ws.php?username=USER&format=1&output=json&latmin=&latmax=&lonmin=&lonmax=

format=1 = human-readable degrees / knots. Faster than once/minute returns empty.[8]

Not useful on the Pi unless Lance later stands up an AIS dongle. Keep as a footnote.

4.3 Skip / later

• MarineTraffic REST — commercial.
• Global Fishing Watch — free non-commercial token, fishing-effort tiles / vessel identity, not a general AIS live layer.[24] Optional Phase D for a “fishing effort” overlay.
• AISHub without a feeder — no data.

---

5. Train traffic

There is no free global live train API. US intercity is solvable; European freight is not, cheaply.

5.1 Amtraker — recommended US passenger trains

Community API that rehosts Amtrak + Brightline + VIA Rail. Free. Identifying User-Agent required or the request is blocked.[9] Data license ODC-By 1.0; attribute “Amtraker” on the map (collapsed OSM-style credit is OK).[9][10]

Live probe: GET https://api.amtraker.com/v3/trains and https://api-v3.amtraker.com/v3/trains both HTTP 200, ~1.3 MB, 197 train numbers. Sample: Sunset Limited #1, lat/lon, heading, stations[], iconColor.

GET https://api.amtraker.com/v3/trains
GET https://api.amtraker.com/v3/trains/:trainId     # e.g. 5-9
GET https://api.amtraker.com/v3/stations
GET https://api.amtraker.com/v3/stale               # stale=true if Amtrak >15 min old

Poll 15–30 s. Response is { "1": [Train, ...], "5": [...], ... } — flatten to one row per trainID. Do not hammer; the author asks not to spam.[10]

Frontend: train icon, color from iconColor (or compute lateness). Popup: route, number, next station, delay.

5.2 Transitland — multi-agency GTFS-RT (optional, key)

Free plan: 10,000 REST queries/month, 100k vector tiles/month, 1,000 routing/month. Key via Interline signup (apikey= query or header; not Authorization). Attribution required.[11][12]

https://transit.land/api/v2/rest/

Use later if you want commuter rail / subway vehicle positions (LA Metro, MTA, etc.). 10k/month is only ~13 queries/hour — cannot poll every agency every 15 s. Cache GTFS-RT protobufs yourself from the agency’s own feed URL (those are usually free, no Transitland meter). Transitland is a directory; the live positions should come from the agency GTFS-RT URL listed in the Atlas.

5.3 OpenRailwayMap — infrastructure tiles only

https://tiles.openrailwaymap.org/{style}/{z}/{x}/{y}.png — OSM-derived tracks, not live trains.[25] This probe’s HEAD got HTTP 403 (UA/hotlink?). Treat as optional overlay; do not depend on it. No vehicle data.

5.4 Skip

• Official Amtrak Track-A-Train JSON — unofficial reverse-engineering; Amtraker already does this legally enough for a hobby map, with a documented UA policy.
• Deutsche Bahn / Network Rail — free-ish developer portals but EU-centric and key + ToS heavy. Out of scope unless the map goes global-rail.

---

6. Open video / camera feeds (official public only)

Do not add Insecam-style random IP cams as a new source. The scraper already has a public list; this section is agency-published JPEG/HLS.

6.1 Already wired

• ALERTWest / ALERTCalifornia: https://api.cdn.prod.alertwest.com/api/getCameraDataByLoc — live probe HTTP 200, 6.6 MB JSON with data.locs + data.cams. Snapshots at https://img.cdn.prod.alertwest.com/data/img/{cid}/... (see parse_alertwest_json).
• Live-Environment-Streams GeoJSON (HLS / YouTube / JPEG).
• public-ip-cams README list.

6.2 Caltrans CWWP2 — add to CAMERA_SOURCE_URLS

Official CA DOT CCTV. No key.

https://cwwp2.dot.ca.gov/data/d{N}/cctv/cctvStatusD{NN}.json

Live probe D4: HTTP 200, 2.4 MB, 749 cameras.

Each data[].cctv:

• location.{latitude,longitude,locationName,nearbyPlace,district,route,county,direction}
• inService ("true"/"false")
• imageData.static.currentImageURL — JPEG, updates ~5 s (https://cwwp2.dot.ca.gov/data/d4/cctv/image/...jpg)
• imageData.streamingVideoURL — HLS https://wzmedia.dot.ca.gov/D4/....stream/playlist.m3u8

Districts 1–12 exist with the same schema (D4 verified). Backend: new parser next to parse_alertwest_json, skip inService != true, store JPEG as snapshot_url, HLS as stream URL (dashboard already vendors hls.min.js). Frontend: same camera popup; prefer JPEG for map thumbs, HLS in the preview pane. Never RTSP.

Politeness: scrape hourly like other sources (CAMERA_SCRAPE_INTERVAL=3600). Do not scrape every JPEG every cycle — store URLs, let the snapshot cache fetch on demand.

6.3 IEM webcams — small, Iowa-centric bonus

GET https://mesonet.agron.iastate.edu/geojson/webcam.geojson

No key, CORS permissive.[23] Live probe: HTTP 200, 3 cameras at probe time (not a national network). Fields: cid, name, imgurl, utc_valid, state. Nice extra source, low value vs ALERTWest/Caltrans.

6.4 YouTube live (already in Live-Environment-Streams)

Do not call the YouTube Data API unless you want search. Embedding existing stream URLs from the GeoJSON is enough. YouTube Data API requires a Google key and quota.

6.5 Skip

• Insecam / random “public IP cam” aggregators — ToS / privacy.
• TrafficLand, EarthCam commercial APIs.
• SkylineWebcams — scraping, not an API.

---

7. Fire maps (beyond current FIRMS points)

FIRMS CSV = hotspot points (already ingested). WFIGS = incident identity + perimeters. GIBS = pixel overlay. Use all three; they answer different questions.

7.1 WFIGS current incident locations — recommended vector layer

No key. ArcGIS FeatureServer, GeoJSON.

https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Incident_Locations_Current/FeatureServer/0/query
  ?where=1=1&outFields=IncidentName,IncidentSize,FireDiscoveryDateTime,POOState,PercentContained,IncidentTypeCategory,FireCause
  &outSR=4326&f=geojson

Live probe: 635 current locations. Sample: Foster Bridge, 675 acres, US-FL, 100% contained, cause Human. FireDiscoveryDateTime is epoch ms.

Poll 5–15 min (NIFC says related perimeter service refreshes ~5 min).[21] Backend can proxy GET /api/fire-incidents so the keyless URL is cached. Frontend: labeled point, popup acres / contained %.

Some similarly named services (Current_WildlandFire_Locations) return Token Required — stick to WFIGS_Incident_Locations_Current.

7.2 WFIGS current perimeters — recommended polygon overlay

https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/WFIGS_Interagency_Perimeters_Current/FeatureServer/0/query
  ?where=1=1&outFields=poly_IncidentName,poly_GISAcres,attr_IncidentSize,attr_PercentContained,attr_FireDiscoveryDateTime
  &outSR=4326&f=geojson

Live probe: 236 current perimeters. Historic WFIGS_Interagency_Perimeters counted 41,078 — do not fetch that unfiltered. Use Current only, or resultRecordCount + bbox geometry + spatialRel=esriSpatialRelIntersects.

Authoritative US wildland-fire polygons; not every incident has a perimeter. Refresh ~5 min, fall-off rules drop stale small fires.[21]

Frontend: L.geoJSON fill red/orange by acres. Click for name/acres. This is the layer that makes FIRMS dots make sense.

7.3 GIBS thermal tiles (no key, no storage)

Already in docs/firms.md. Leaflet:

https://gibs.earthdata.nasa.gov/wmts/epsg3857/best/VIIRS_SNPP_Thermal_Anomalies_375m_All/default/{time}/GoogleMapsCompatible_Level{z}/{y}/{x}.png

Max zoom ~9. {time} = YYYY-MM-DD. Same GIBS CORS * as existing basemaps. Use as a toggle under FIRMS points.

7.4 FIRMS dataset note

Keep the CSV ingest. Add NOAA-20/21 datasets. Ultra-real-time US/Canada is available on those products (seconds-scale for much of US/Canada).[20]

---

8. Weather radar + alerts

8.1 RainViewer — recommended global radar overlay

No key. Personal / educational / small community. Attribution: “Weather data by RainViewer” + link.[13]

Live probe: GET https://api.rainviewer.com/public/weather-maps.json → version 2.0, host https://tilecache.rainviewer.com, 13 past frames (~2 h, 10 min steps). Tile HEAD: HTTP 200, CORS *, x-ratelimit-limit: 500 / 60 s, burst 300.

Tile URL:[14]

{host}{path}/{size}/{z}/{x}/{y}/{color}/{smooth}_{snow}.png

Example: https://tilecache.rainviewer.com/v2/radar/cb581daa2c0f/256/{z}/{x}/{y}/2/1_1.png

• {size} 256 or 512
• {color} scheme id (2 is a common default)
• {smooth}_{snow} e.g. 1_1
• max zoom 7 (coarse vs IEM)

Frontend: fetch the JSON every 5 min, set L.tileLayer to the latest past[].path. Optional animation through past frames. No backend.

8.2 IEM NEXRAD / MRMS — recommended CONUS quality radar

Iowa Environmental Mesonet. No key. Free including commercial, as-is, no SLA.[16] CORS * on tiles (live HEAD confirmed).

Current CONUS base reflectivity TMS:[15]

https://mesonet.agron.iastate.edu/cache/tile.py/1.0.0/nexrad-n0q/{z}/{x}/{y}.png

Cache-Control 5 min. Aliases mesonet1/2/3.agron.iastate.edu for parallel browser tile fetches.[15]

Loop (past 5–55 min, step 5): layer nexrad-n0q-mXXm e.g. nexrad-n0q-m05m.

Also useful:

Layer	Meaning
nexrad-eet	Echo tops
q2-hsr	MRMS seamless hybrid-scan reflectivity
q2-n1p / q2-p24h	MRMS 1 h / 24 h precip
goes_east_conus_ch02	GOES-East visible
goes_east_conus_ch13	GOES-East IR

WMS alternative: https://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0q.cgi? — TMS is simpler for Leaflet.

Frontend: use IEM as default when the map is over CONUS (higher res than RainViewer z7); RainViewer when zoomed outside the US or as a global fallback.

8.3 NWS API — alerts / forecasts (no radar tiles)

https://api.weather.gov — US public domain, no fee. User-Agent required (app + contact). Rate limit unpublished; 429s clear in ~5 s. CORS *.[17]

Live probe: GET /alerts/active?area=NC → HTTP 200 GeoJSON, 4 alerts, CORS *.

GET https://api.weather.gov/alerts/active?area=NC
GET https://api.weather.gov/alerts/active?point={lat},{lon}
GET https://api.weather.gov/points/{lat},{lon}          # then follow forecast URLs

Frontend can hit this directly (CORS open) or backend can cache 30 s to protect the Pi if many tabs are open. Prefer backend cache. Draw features[].geometry with color by properties.severity / event.

8.4 IEM storm-based warnings GeoJSON

GET https://mesonet.agron.iastate.edu/geojson/sbw.geojson

Live probe: 46 current polygons. Sample: Severe Thunderstorm Warning, WFO BGM. Good companion to NWS alerts (polygon-accurate).

8.5 Open-Meteo — point forecast, not radar

No key for non-commercial < 10,000 calls/day. CC BY 4.0 data, attribution required. Commercial needs a plan or self-host.[18][19]

GET https://api.open-meteo.com/v1/forecast?latitude=35.88&longitude=-78.79&current=temperature_2m,precipitation,weather_code,wind_speed_10m

Live probe: RDU 28.4 °C, weather_code 80 (rain showers). Use for click-to-forecast popups, not a map overlay. Cache by grid cell.

8.6 NHC active storms

GET https://www.nhc.noaa.gov/CurrentStorms.json

Live probe: HTTP 200, activeStorms[] — Tropical Storm Dolly (al042026, 13.6N 38.7W, 35 kt). No key. Plot as a hurricane marker + optional NHC cone GeoJSON (separate NHC GIS products). Nice extra during Atlantic season.

---

9. Keys / env vars to add

Store new secrets in the existing Keys UI (api_keys table) and .env fallback, same planned pattern as FIRMS.

Var	Required for	How to get	Cost
FIRMS_MAP_KEY	Existing fire CSV	https://firms.modaps.eosdis.nasa.gov/api/map_key_info/	Free
OPENSKY_CLIENT_ID / OPENSKY_CLIENT_SECRET	OpenSky authenticated	OpenSky account → API Client	Free
AISSTREAM_API_KEY	Live AIS	GitHub login at aisstream.io/account	Free
TRANSITLAND_API_KEY	Transitland REST (optional)	Interline free plan	Free 10k/mo
GFW_TOKEN	Fishing-effort overlay (optional)	globalfishingwatch.org/our-apis/tokens	Free, non-commercial
(none)	ADSB.lol, Amtraker, RainViewer, IEM, NWS, WFIGS, Caltrans, Open-Meteo, NHC	—	Free

Amtraker: no key, but set OSINT_USER_AGENT to something like osint-dashboard/1.0 (lancewalters94@gmail.com) — already have USER_AGENT in camera_config.py; reuse it on all outbound HTTP.

NWS: same User-Agent header is mandatory.[17]

---

10. Suggested API surface (backend engineer)

Keep FIRMS GET /api/fires. Add:

GET /api/aircraft?bbox=minlon,minlat,maxlon,maxlat
GET /api/vessels?bbox=...
GET /api/trains?bbox=...
GET /api/fire-incidents?bbox=...          # WFIGS points
GET /api/fire-perimeters?bbox=...         # WFIGS polygons (or proxy GeoJSON)
GET /api/alerts?area=NC|&bbox=...         # cached NWS + IEM SBW
GET /api/map/radar                         # { provider, tileUrl, frames[] } for RainViewer metadata
GET /api/map/layers                        # already have GIBS; extend with radar/fire tile templates

All vector endpoints: default limit=2000, require bbox except trains (Amtrak is ~200 rows total). Return arrays of {id, lat, lon, heading, speed, label, extra} so the frontend can share one marker renderer.

Do not proxy RainViewer/IEM/GIBS tiles through the Pi (bandwidth). Return the template; browser fetches tiles.

AISStream stays inside ingest profile (needs a long-lived process, like FIRMS). Aircraft/trains can live in run_ingester.py loops.

---

11. Suggested UI (frontend bot)

Layer toggles on the existing Leaflet map (same pattern as GIBS + fire heatmap):

Toggle	Default	Source
Radar	on (CONUS)	IEM nexrad-n0q if view intersects US, else RainViewer
NWS alerts	on	polygons
FIRMS heat	existing	existing
WFIGS perimeters	on	polygons
WFIGS incidents	off	points
Aircraft	on	/api/aircraft
Vessels	off until AIS key	/api/vessels
Trains	on	/api/trains
Cameras	existing	existing

Viewport-driven: on moveend (debounced 300 ms), refetch aircraft/vessels for the new bbox. Do not refetch radar tiles except RainViewer frame JSON every 5 min.

Marker performance: canvas renderer (L.canvas()), cluster at zoom < 7, hide labels until zoom ≥ 8.

Camera preview: Caltrans HLS via existing hls.js path; JPEG thumb from currentImageURL. Still never emit RTSP.

Attribution bar (required): OpenSky / ADSB.lol ODbL / Amtraker / RainViewer / IEM / NWS / NASA FIRMS-GIBS / NIFC WFIGS / OSM as applicable.

---

12. Legal / ethics (non-negotiable)

• RTSP policy unchanged (never emit rtsp:// hrefs).
• AISStream: server-side only; do not put the key in JS.[5]
• OpenSky: non-commercial unless licensed; cite if you publish.[2]
• ADSB.lol: ODbL share-alike on derived databases.[4]
• Amtraker: ODC-By, visible credit, identifying UA.[9]
• RainViewer: personal/educational; credit + link.[13]
• Open-Meteo: non-commercial 10k/day on the public instance.[18]
• GFW: non-commercial.[24]
• Caltrans / NWS / NIFC / NASA / IEM: US public or explicitly open; still cache politely.
• Do not scrape MarineTraffic / FlightRadar24 / FlightAware map tiles.

---

13. Verification log (this research session)

Endpoint	Result
OpenSky /states/all NC bbox	200, 270 aircraft, 0.63 s
ADSB.lol /v2/lat/35.88/lon/-78.79/dist/50	200, 27 aircraft
RainViewer weather-maps.json	200, 13 past frames
RainViewer tile	200, CORS *, 500 req/min header
Open-Meteo forecast RDU	200, 28.4 °C, code 80
NWS /alerts/active?area=NC	200, 4 alerts, CORS *
IEM nexrad-n0q tile	200, CORS *, max-age 300
IEM sbw.geojson	200, 46 polygons
IEM webcam.geojson	200, 3 cams
WFIGS locations count	635
WFIGS current perimeters count	236
ALERTWest getCameraDataByLoc	200, 6.6 MB
Caltrans D4 CCTV JSON	200, 749 cams, JPEG + HLS
Amtraker /v3/trains	200, ~197 numbers, 1.3 MB
AviationWeather METAR KRDU	200 JSON
NHC CurrentStorms.json	200, TS Dolly
OpenRailwayMap tile HEAD	403 — do not rely on

---

14. Concrete next tickets

Backend

1. FIRMS_DATASET dual-write NOAA-20/21; document S-NPP sunset.
2. aircraft ingest: ADSB.lol viewport poll + /api/aircraft.
3. trains ingest: Amtraker 30 s poll + User-Agent + /api/trains.
4. Camera parser: Caltrans CWWP2 D1–D12 → existing cameras rows (discovery_source=caltrans).
5. WFIGS proxy endpoints (locations + current perimeters), 5–15 min cache.
6. AISStream worker behind AISSTREAM_API_KEY (idle if unset, same as FIRMS).
7. /api/alerts 30 s cache of NWS + IEM SBW.

Frontend

1. Layer control: radar (IEM/RainViewer), alerts, WFIGS, aircraft, trains, vessels.
2. Shared point-layer component (id, lat, lon, heading, label).
3. RainViewer frame JSON → animated overlay.
4. Caltrans HLS in existing preview pane.
5. Attribution strip.

Do not start with: Transitland, GFW, AISHub, OpenSky OAuth (until ADSB.lol is live and still needs a fallback).

Sources

[1] https://openskynetwork.github.io/opensky-api/rest.html [2] https://openskynetwork.github.io/opensky-api [3] https://api.adsb.lol [4] https://www.adsb.lol/docs/open-data/api [5] https://aisstream.io/documentation [6] https://aisstream.io [7] https://www.aishub.net [8] https://www.aishub.net/api [9] https://github.com/piemadd/amtrak [10] https://amtraker.com/about [11] https://www.transit.land/documentation [12] https://www.transit.land/plans-pricing [13] https://www.rainviewer.com/api.html [14] https://www.rainviewer.com/api/weather-maps-api.html [15] https://mesonet.agron.iastate.edu/ogc [16] https://mesonet.agron.iastate.edu/api [17] https://www.weather.gov/documentation/services-web-API [18] https://open-meteo.com [19] https://open-meteo.com/en/docs [20] https://firms.modaps.eosdis.nasa.gov/api/area [21] https://data-nifc.opendata.arcgis.com/datasets/nifc::wfigs-current-interagency-fire-perimeters/about [22] https://aviationweather.gov/data/api [23] https://mesonet.agron.iastate.edu/geojson/webcam.py?help [24] https://globalfishingwatch.org/our-apis/documentation [25] https://openrailwaymap.org [26] https://opensky-network.org/about/faq [27] https://cwwp2.dot.ca.gov/vm/iframemap.htm