sirius/osint-dashboard
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: chunk FIRMS bulk inserts under asyncpg 32767 bind cap #14

Merged
sirius merged 1 commit from fix/firms-bind-limit into master 2026-08-28 22:20:49 -04:00
Conversation 0 Commits 1 Files changed 2 +67 -7
sirius commented 2026-08-28 22:20:23 -04:00
Owner
Copy link

Summary

/api/health is degraded because fires.ok is false (last write ~50 min ago).

Root cause: ingest_fire_rows sent ~90k rows in one INSERT, which exceeds asyncpg's 32767 bind-parameter limit. fire_loop logs InterfaceError and skips the poll.

Fix: chunk 2000 rows/statement (~14 cols → 28k binds), still one commit per poll.

Tests

pytest -q → 108 passed, 15 skipped.

Note

CI deploys on merge to master. HUD will stay DEGRADED until the next successful FIRMS poll after this ships.

## Summary `/api/health` is **degraded** because `fires.ok` is false (last write ~50 min ago). Root cause: `ingest_fire_rows` sent ~90k rows in one `INSERT`, which exceeds asyncpg's 32767 bind-parameter limit. `fire_loop` logs `InterfaceError` and skips the poll. Fix: chunk 2000 rows/statement (~14 cols → 28k binds), still **one commit** per poll. ## Tests `pytest -q` → 108 passed, 15 skipped. ## Note CI deploys on merge to master. HUD will stay DEGRADED until the next successful FIRMS poll after this ships.
sirius added 1 commit 2026-08-28 22:20:23 -04:00 
A ~90k-row INSERT ... ON CONFLICT dies with InterfaceError, so fire_loop
fails every poll and /api/health stays degraded (fires age > 30 min).
Chunk 2000 rows per statement, still one commit per poll.
sirius merged commit d25ca8ed9c into master 2026-08-28 22:20:49 -04:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: sirius/osint-dashboard#14
No description provided.
Delete branch "fix/firms-bind-limit"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?