sirius/osint-dashboard
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(vessels): protect Hormuz VesselAPI rows from AISStream eviction #25

Merged
sirius merged 1 commit from fix/vesselapi-hormuz-eviction into master 2026-08-29 13:04:29 -04:00
Conversation 0 Commits 1 Files changed 2 +45 -1
sirius commented 2026-08-29 13:04:10 -04:00
Owner
Copy link

Summary

/api/vessels stayed empty for the Hormuz bbox even though the VesselAPI poller was fetching 50 vessels per 2xx (logs: VesselAPI poll OK: 50 vessels (remaining=148, calls_today=3)).

Root cause: vessel_last_known is a single in-memory store capped at _MAX_VESSELS (6000). AISStream's live US-coast feed crowds it past the cap, and the evict oldest seen_at pass reaps the Hormuz VesselAPI rows — they're only refreshed every VESSELAPI_INTERVAL (~4.8h), so they always sort as the oldest and get evicted within seconds of landing.

Fix

app/live_layers.py upsert_vessel eviction now excludes src=vesselapi rows from eviction candidates (clamped to the available evictable rows), so the Hormuz blind-spot stays resident between polls. Stops short of disabling AISStream or partitioning the cap.

Test

tests/test_vesselapi.py adds test_eviction_keeps_vesselapi_rows: fills the store past a shrunk _MAX_VESSELS with AISStream rows plus the oldest-seen_at VesselAPI row, triggers an eviction, asserts the VesselAPI row survives.

Full suite: 152 passed / 16 skipped.

Live verify (post-merge)

curl -4 -k -sS 'https://rpi.tail14a963.ts.net:8443/api/vessels?bbox=55.4,25.5,57.2,27.3&limit=50'

Expect ≥1 extra.src=vesselapi point inside Hormuz.

## Summary `/api/vessels` stayed empty for the Hormuz bbox even though the VesselAPI poller was fetching 50 vessels per 2xx (logs: `VesselAPI poll OK: 50 vessels (remaining=148, calls_today=3)`). Root cause: `vessel_last_known` is a single in-memory store capped at `_MAX_VESSELS` (6000). AISStream's live US-coast feed crowds it past the cap, and the `evict oldest seen_at` pass reaps the Hormuz VesselAPI rows — they're only refreshed every `VESSELAPI_INTERVAL` (~4.8h), so they always sort as the oldest and get evicted within seconds of landing. ## Fix `app/live_layers.py` `upsert_vessel` eviction now excludes `src=vesselapi` rows from eviction candidates (clamped to the available evictable rows), so the Hormuz blind-spot stays resident between polls. Stops short of disabling AISStream or partitioning the cap. ## Test `tests/test_vesselapi.py` adds `test_eviction_keeps_vesselapi_rows`: fills the store past a shrunk `_MAX_VESSELS` with AISStream rows plus the oldest-`seen_at` VesselAPI row, triggers an eviction, asserts the VesselAPI row survives. Full suite: 152 passed / 16 skipped. ## Live verify (post-merge) ``` curl -4 -k -sS 'https://rpi.tail14a963.ts.net:8443/api/vessels?bbox=55.4,25.5,57.2,27.3&limit=50' ``` Expect ≥1 `extra.src=vesselapi` point inside Hormuz.
sirius added 1 commit 2026-08-29 13:04:11 -04:00 
AISStream's live US-coast feed crowds vessel_last_known past _MAX_VESSELS
(6000) and the 'evict oldest seen_at' pass reaps the Hormuz VesselAPI rows,
which are only refreshed every VESSELAPI_INTERVAL (~4.8h). Result: the poller
was fetching 50 Hormuz vessels per 2xx but /api/vessels?bbox=Hormuz stayed
empty. Eviction now skips src=vesselapi rows (falls back to evictable-only,
clamped) so the blind-spot box stays resident between polls.
sirius merged commit b6da7b7bd4 into master 2026-08-29 13:04:29 -04:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: sirius/osint-dashboard#25
No description provided.
Delete branch "fix/vesselapi-hormuz-eviction"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?