627990efde
Coherent merge of two coordinated features on the shared working tree: FIRMS fire heatmap (backend, t_6e404c14): - app/fire_sources.py: fetch FIRMS VIIRS area CSV (free MAP_KEY) -> NATS events.fire - fires hypertable (TimescaleDB, 1-day chunks) with natural-key PK (latitude, longitude, acq_time, satellite); idempotent ON CONFLICT DO NOTHING - alembic/versions/002_fires.py; GET /api/fires?bbox=&since= (JSON only) - POST /api/ingest/fires; ~15 min poll loop (FIRMS_INTERVAL=900) in ingester - env-driven config (FIRMS_MAP_KEY/DATASET/BBOX/INTERVAL); docs/firms.md covers the zero-cost GIBS VIIRS_SNPP_Thermal_Anomalies_375m_All tile alternative - 18 tests (parser, mapping, idempotency, API contract) verified vs real TimescaleDB+PostGIS (localhost/osint-dashboard-pg image) API keys page (frontend, t_4433cff2): - app/keystore.py: api_keys table (self-creating), FIRMS/GEMINI/TELEGRAM registry with format validation, ****last4 masking, get_api_key() - GET/POST/DELETE /api/keys (never returns full values); Keys tab in index.html DB_NULL_POOL env switch in app/database.py enables a NullPool for tests / short-lived processes that open a fresh event loop per unit.
32 lines
1.2 KiB
Python
32 lines
1.2 KiB
Python
import os
|
|
|
|
from sqlalchemy import MetaData, event, text
|
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
|
from sqlalchemy.pool import NullPool
|
|
|
|
from config import DATABASE_URL
|
|
|
|
# Pooled connections are bound to the event loop they were created on. Long-lived
|
|
# servers (uvicorn, the ingester) want a pool; short-lived / test processes that
|
|
# open a fresh event loop per unit (asyncio.run, pytest) must not reuse pooled
|
|
# connections across loops, so allow a NullPool (new connection per session).
|
|
_NULL_POOL = os.getenv("DB_NULL_POOL", "").lower() in ("1", "true", "yes")
|
|
|
|
engine = create_async_engine(
|
|
DATABASE_URL,
|
|
echo=False,
|
|
**({"poolclass": NullPool} if _NULL_POOL else
|
|
{"pool_size": 5, "max_overflow": 10, "pool_recycle": 300}),
|
|
)
|
|
async_session = async_sessionmaker(
|
|
engine, class_=AsyncSession, expire_on_commit=False
|
|
)
|
|
metadata = MetaData()
|
|
|
|
|
|
async def init_extensions():
|
|
"""Initialize PostGIS and TimescaleDB extensions on first connection."""
|
|
async with engine.connect() as conn:
|
|
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS postgis"))
|
|
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS timescaledb"))
|
|
await conn.commit()
|