- De-duplicate URLs in scrapy-urls configmap
- Switch CronJob image from siriussec/newsscraper:0.5.4 to ghcr.io/sirius0xdev/newsscraper:latest
- imagePullPolicy already set to Always
Co-authored-by: Sirius Devops <sirius@siriusdevops.com>
CNPG cluster was missing the news_app role — scraper cronjob
was failing with 'connection to server at port 5432 failed'
because the role had no login/password configured in managed roles.
Adds news_app to managed roles, pointing to existing
news-app-password secret.
Co-authored-by: Hermes DevOps <hermes@devops.local>
Adds NEWS_DB_PASSWORD env var from existing news-app-password secret
so the secondary DB connection to news_app_db can authenticate.
Co-authored-by: Hermes DevOps <hermes@devops.local>
Data service needs to reach Helius WSS, Solana RPC, Jupiter API,
Hyperliquid — all external. Network policy had no egress rule for
external traffic, so all API/websocket connections were blocked.
Data-service pods use NetworkPolicy egress selector app: trading-kafka
but Strimzi Kafka pods are labeled strimzi.io/name=trading-kafka.
This silently blocked all Kafka traffic causing NoBrokersAvailable.
- Remove helius/jupiter placeholders from trading-secrets.yaml
- Create new solana-api-keys.yaml with helius-api-key and jupiter-api-key
- Update kustomization.yaml to include solana-api-keys.yaml
Convention: each secret group gets its own SOPS-encrypted file
instead of adding new keys to an already encrypted file.
- Replace Gemini API and A100 vLLM references with central rtx6000-brain vLLM
endpoint (http://rtx6000-brain-service.customer1.svc.cluster.local:8000/v1)
- Update all ConfigMaps to use model edp1096/Huihui-Qwen3.6-27B-abliterated-FP8
- Add LLM_API_KEY: sk-dummy to all ConfigMaps
- Remove ollama sidecars, initContainers, PVC volumes from all analyst CronJobs
- Remove GPU nodeSelector, tolerations, and GPU resource requests from CronJobs
- Clean up kustomization.yaml: remove ollama-storage.yaml, deepseek-storage.yaml,
gemini-apikey.yaml references
- Activate deepseek-summarizer.yaml in kustomization, keep others commented
- Create solana-ingester Deployment + Service in trading-platform/
(uses same data-service image, runs WebSocket ingester mode)
- Add solana-ingester ConfigMap (HELIUS_RPC_ENDPOINT, JUPITER_API_URL,
KAFKA_BROKER, KAFKA_TOPIC)
- Add helius-api-key and jupiter-api-key placeholders to trading-secrets.yaml
(TODO: replace with real values and re-encrypt with SOPS)
- Update kustomization to include new resources
Kraft mode fails with AccessDeniedException on meta.properties
because PVC mounts with root ownership. Adding fsGroup: 1000
ensures the volume group matches the Kafka container user.
- Removed template.pod.brokers (not valid in Strimzi 1.0.0 Kafka schema)
- Moved broker resources to KafkaNodePool.spec.resources
- Added controller role to KafkaNodePool (KRaft requires both controller+broker)
strimzi.io/install/1.0.0 is not a valid git repo or kustomize remote URL.
Download the full install bundle (strimzi-1.0.0.yaml) and reference it
locally in the strimzi kustomization.
Split Strimzi operator deployment into its own Flux Kustomization so
CRDs are applied before the Kafka CR in customer1.
Chain: infrastructure-controllers -> customer1-strimzi -> customer1
customer1 now has force: true to skip dry-run validation when CRDs are
recently applied but not yet visible to the APIServer cache.
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.
Two kustomizations now:
strimzi/ - Strimzi 1.0.0 CRDs + operator
siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)
Wire them in Flux:
Kustomization 'siriusdevops-strimzi' -> path: strimzi/
Kustomization 'siriusdevops-db' -> path: siriusdevops-db/
dependsOn: [{name: siriusdevops-strimzi}]
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.
Pinned to 1.0.0 for reproducible deployments.
/etc/kafka doesn't exist in the apache/kafka image - subPath could not
create the parent directory. Switch to /opt/kafka/config/ which exists
in the image, avoiding both the directory conflict and the missing path.
Container fails with 'read-only file system' when K8s tries to mount
a secret into /etc/kafka/secrets alongside the ConfigMap at /etc/kafka.
Switch to subPath mount of server.properties directly.
Trading services depend on Kafka (trading-kafka.customer1.svc.cluster.local:9092)
but it was never deployed — lived under hermes-db kustomization which isn't
applied. Move to siriusdevops-db where trading infra belongs.
Also remove stale redis-cluster.yaml reference from hermes-db (file missing).
Migrate trading platform DB egress rule from hermes-pgdb to
siriusdevops-pgdb to complete the trading DB migration. The
configmaps already point to siriusdevops-pgdb-rw but the
network policy still allowed egress to the old cluster.
- news-app-db.yaml: point Database CR to siriusdevops-pgdb
- All ConfigMaps: DB_HOST=siriusdevops-pgdb-rw, LLM_BASE_URL for vLLM
- hermes-pgdb now dedicated to pgvector/RAG for agents only
- Aligns with CNPG extension pattern and active kanban (devops startupProbes, backend exec-service)
PR to master per repo rules.
- Add build-pgvector-image.yml GH Actions workflow for GCR
- Set imageName to gcr.io/devops-lab-cluster/postgres-pgvector:15.2-0.8.0
- Image change only (CNPG rule: no config changes in same update)
- Add 'trading' managed role to siriusdevops-pgdb cluster
- Move trading-db-credentials from hermes-db to siriusdevops-db
- Create trading-data-db.yaml (Database kind) pointing to siriusdevops-pgdb
- Update DB_HOST in all 4 trading-platform configmaps to siriusdevops-pgdb-rw
- Remove trading-data-db.yaml and trading-db-credentials.yaml from hermes-db
- Update siriusdevops-db kustomization.yaml to include new resources
