Commit graph

8 commits

Author SHA1 Message Date
Sirius DevOps
604738e176
Merge origin/master into ODOT TripCheck cameras
Keep both ODOT TripCheck and already-merged MDOT MiDrive parsers
(disjoint camera sources on the same dispatch/config/tests files).
2026-08-31 22:09:34 -04:00
Sirius DevOps
41433be574 feat(cameras): ODOT TripCheck parser -> cameras table
Add parse_odot_json (Esri-style cctvinventory.js) next to parse_caltrans_json:
JPEG snapshot_url, discovery_source=odot, Oregon bbox + filename guard,
no RTSP. Wire dispatch + default CAMERA_SOURCE_URLS entry. Unit tests.
2026-08-31 21:38:09 -04:00
Sirius DevOps
fbff9e5415 feat(cameras): MDOT MiDrive parser -> cameras table
Add parse_mdot_json for the MDOT MiDrive camera/list JSON where
coordinates/id live in the county field's map link (lat=/lon=/id=) and
the JPEG still lives in the image field's <img src>. Michigan bbox
filter, missing-coords drop, RTSP reject. discovery_source=mdot,
stable source_url keyed on camera id, url_hash dedupe. Wired into
scrape_source dispatch + CAMERA_SOURCE_URLS defaults.

Unit tests: HTML field extract, bbox drop, missing-coords/image drop,
malformed payload.
2026-08-31 21:31:12 -04:00
Sirius DevOps
f779b1f225 feat: toggleable live map feeds (ADS-B, trains, AIS, radar, WFIGS, NWS)
Wire the free data streams from docs/free-data-streams.md into the
dashboard as layer-panel toggles. Third-party APIs are proxied/cached
in FastAPI; raster tiles (IEM, RainViewer, GIBS) stay in the browser.

- Aircraft via ADSB.lol viewport poll (bbox required, radius ≤ 150 nm)
- Amtraker trains, NHC storms, WFIGS incidents/perimeters
- NWS + IEM SBW as /api/weather-alerts (does not collide with /api/alerts)
- AISStream worker is server-side only and idles without AISSTREAM_API_KEY
- Caltrans CWWP2 D1–D12 camera parser; FIRMS dual-write NOAA-20/21
2026-08-27 19:08:30 -04:00
Sirius DevOps
0c7f80655e cameras: Live-Environment-Streams + HLS.js player
All checks were successful
build-and-deploy / build (push) Successful in 3m8s
Ingest ~4.2k direct HLS/YouTube feeds (VDOT, MDSHA, DelDOT, Iowa DOT,
OpenCCTV, etc.) from the public GeoJSON catalog. Popup plays HLS via a
CORS-safe playlist proxy + vendored hls.js, YouTube via embed, with
ffmpeg-MJPEG fallback. ALERTWest rows now use a stable camera id so
hourly scrapes do not duplicate.
2026-08-27 15:54:54 -04:00
Sirius DevOps
f3c35c2230 cameras: ingest ALERTCalifornia/ALERTWest official public JPEGs
All checks were successful
build-and-deploy / build (push) Successful in 3m36s
Adds the documented getCameraDataByLoc API (~10k public wildfire, DOT,
and FAA stills with lat/lon). Masscan will not produce viewable feeds;
this will. Map bbox fetch raised to 2000 markers.
2026-08-27 15:46:49 -04:00
Sirius DevOps
347ee3a642 Camera scraper: live public source (fury999io/public-ip-cams), markdown-list parser, batch IP geolocation via ip-api; fix empty-env fallback so default source applies
All checks were successful
build-and-deploy / build (push) Successful in 1m38s
2026-08-24 21:03:33 -04:00
Sirius DevOps
69a30115cc Add open IP camera discovery scraper (OSINT map)
Some checks failed
build-and-deploy / build (push) Failing after 5s
- camera_scraper: public-directory-only discovery (Insecam-style HTML +
  plain-text lists), hard private-range guard (fail closed), per-host
  rate limiting, Nominatim geocoding at <=1 req/s, TTL'd local snapshot
  cache, sha256 url_hash dedupe with Postgres upsert
- cameras table (migration 002) + /api/cameras?bbox= + snapshot endpoint
  with cache passthrough in main.py
- run_camera_service: long-running cycle worker following NATS->ingester
  pattern; publishes events.camera for shared ingester
- docker-compose camera-service profile, .env.example knobs

Verified E2E against TimescaleDB+PostGIS: private-range entries dropped,
dedupe across cycles holds, bbox query returns expected rows.
2026-08-24 14:54:51 -04:00