Commit graph

987 commits

Author SHA1 Message Date
Hermes Agent
f9623b9ca2 config(execute-service): enable auto-create tables in production
Sets EXECUTE_DB_AUTO_CREATE_TABLES=true so the service creates
its own tables on startup instead of relying on non-existent
alembic migrations.
2026-05-27 19:20:31 +00:00
sirius0xdev
267356de97
Merge pull request #187 from sirius0xdev/fix/kafka-netpol-label
fix: network policy Kafka selector app → strimzi labels
2026-05-27 10:20:42 -04:00
Hermes Agent
bb170365cc fix: network policy Kafka selector app -> strimzi labels
Data-service pods use NetworkPolicy egress selector app: trading-kafka
but Strimzi Kafka pods are labeled strimzi.io/name=trading-kafka.
This silently blocked all Kafka traffic causing NoBrokersAvailable.
2026-05-27 14:19:26 +00:00
sirius0xdev
037ba16543
Merge pull request #186 from sirius0xdev/fix/redis-maxmemory-unit
fix: redis maxmemory unit 1gi -> 1gb for Redis 7.4.7 compat
2026-05-27 10:05:29 -04:00
Hermes Agent
6447f92157 fix: redis maxmemory unit 1gi -> 1gb for Redis 7.4.7 compat 2026-05-27 13:59:06 +00:00
sirius0xdev
87c0835e72
Merge pull request #185 from sirius0xdev/fix/redis-stack-image-tag
fix: redis-stack image tag 7.4.0-v8
2026-05-27 09:39:54 -04:00
Hermes Agent
18b728c95e fix: redis-stack image tag 7.4.0-v8 (7.4.2-v16 not found) 2026-05-27 13:37:47 +00:00
sirius0xdev
f8fa436b8b
Merge RediSearch PR
Redis Stack with RediSearch for HNSW vector search. Memory bumped to 1Gi.
2026-05-27 08:17:36 -04:00
sirius0xdev
bf37ebd3b3
Merge pull request #180 from sirius0xdev/feat/trading-api-keys-secret
feat: add trading-api-keys secret for Helius API key
2026-05-27 01:55:40 -04:00
Hermes DevOps
9eecad47bb fix: point data-service to solana-api-keys secret + set Helius key
- Fix deployment env refs: trading-platform-secrets -> solana-api-keys
- Replace placeholder helius-api-key with actual value
2026-05-27 05:52:43 +00:00
sirius0xdev
380325c981
Merge pull request #179 from sirius0xdev/feat/merge-solana-ingester-into-data-service
Merge solana-ingester into data-service
2026-05-27 01:34:26 -04:00
sirius0xdev
6b2472f376
Merge pull request #177 from sirius0xdev/migrate-news-bot-to-rtx6000-vllm
Migrate news bot to rtx6000-brain vLLM endpoint
2026-05-27 01:31:02 -04:00
Hermes Agent
26059b01e6 Merge solana-ingester into data-service, remove standalone deployment
- Remove solana-ingester deployment (was using a nonexistent CLI entrypoint)
- Merge all ingestion into data-service via env vars + shared image
- Add HELIUS_API_KEY, JUPITER_API_KEY, KAFKA_BROKER to data-service configmap
- Add solana secret refs to data-service deployment
- Remove solana-ingester-config configmap
2026-05-27 05:27:55 +00:00
sirius0xdev
39c2e1121a
Merge pull request #178 from sirius0xdev/fix/split-solana-secrets
fix(secrets): split Solana API keys into separate SOPS file
2026-05-27 00:36:43 -04:00
c7731aeec0 fix(secrets): split Solana API keys into separate SOPS file
- Remove helius/jupiter placeholders from trading-secrets.yaml
- Create new solana-api-keys.yaml with helius-api-key and jupiter-api-key
- Update kustomization.yaml to include solana-api-keys.yaml

Convention: each secret group gets its own SOPS-encrypted file
instead of adding new keys to an already encrypted file.
2026-05-27 04:35:47 +00:00
Hermes Agent
e8f33a7bd0 Migrate news bot to rtx6000-brain vLLM endpoint
- Replace Gemini API and A100 vLLM references with central rtx6000-brain vLLM
  endpoint (http://rtx6000-brain-service.customer1.svc.cluster.local:8000/v1)
- Update all ConfigMaps to use model edp1096/Huihui-Qwen3.6-27B-abliterated-FP8
- Add LLM_API_KEY: sk-dummy to all ConfigMaps
- Remove ollama sidecars, initContainers, PVC volumes from all analyst CronJobs
- Remove GPU nodeSelector, tolerations, and GPU resource requests from CronJobs
- Clean up kustomization.yaml: remove ollama-storage.yaml, deepseek-storage.yaml,
  gemini-apikey.yaml references
- Activate deepseek-summarizer.yaml in kustomization, keep others commented
2026-05-27 04:25:41 +00:00
sirius0xdev
0768af1a31
Merge pull request #176 from sirius0xdev/feat/solana-ingester-redis-pvc
feat: add Solana WebSocket ingester + Redis persistent volume
2026-05-27 00:07:57 -04:00
Hermes DevOps
e24e82107d feat: add solana-ingester deployment for real-time market data
- Create solana-ingester Deployment + Service in trading-platform/
  (uses same data-service image, runs WebSocket ingester mode)
- Add solana-ingester ConfigMap (HELIUS_RPC_ENDPOINT, JUPITER_API_URL,
  KAFKA_BROKER, KAFKA_TOPIC)
- Add helius-api-key and jupiter-api-key placeholders to trading-secrets.yaml
  (TODO: replace with real values and re-encrypt with SOPS)
- Update kustomization to include new resources
2026-05-27 04:05:41 +00:00
Hermes DevOps
c587a707a6 feat(redis): add persistent volume for trading-redis
Replace emptyDir with 10Gi PVC (ReadWriteOnce) so hot cache
survives pod restarts.
2026-05-27 04:05:41 +00:00
sirius0xdev
7e55874faf
Update kustomization.yaml 2026-05-26 22:38:39 -04:00
sirius0xdev
e78674a964 Fix data-service port mismatch: container runs on 8000, not 8001 2026-05-26 10:54:23 -09:00
sirius0xdev
c5e2ef16d2 Move Redis cluster to trading-platform and wire into kustomization 2026-05-26 09:52:16 -09:00
sirius0xdev
777f98a5ff
Update kustomization.yaml 2026-05-26 00:10:34 -04:00
sirius0xdev
f9ac2d553b
Merge pull request #175 from sirius0xdev/fix/kafka-fsgroup-permissions
fix(kafka): add fsGroup to pod securityContext for PVC write permissions
2026-05-25 23:34:42 -04:00
Hermes Agent
308d54cc0b fix(kafka): add fsGroup to pod securityContext for PVC write permissions
Kraft mode fails with AccessDeniedException on meta.properties
because PVC mounts with root ownership. Adding fsGroup: 1000
ensures the volume group matches the Kafka container user.
2026-05-26 03:33:43 +00:00
sirius0xdev
0f5488f8e3
Merge pull request #174 from sirius0xdev/fix/kafka-version-4.1.0
bump Kafka version from 3.8.0 to 4.1.0
2026-05-25 23:11:03 -04:00
Hermes Agent
207733f4df bump Kafka version from 3.8.0 to 4.1.0 (supported by Strimzi 1.0.0) 2026-05-26 03:01:40 +00:00
sirius0xdev
472ccc4943
Merge pull request #173 from sirius0xdev/fix/strimzi-add-controller-role
fix trading-kafka: add controller role + move resources to KafkaNodePool
2026-05-25 22:48:44 -04:00
Hermes Agent
f8b97537d7 fix trading-kafka: remove invalid template.pod.brokers, add controller role to KafkaNodePool
- Removed template.pod.brokers (not valid in Strimzi 1.0.0 Kafka schema)
- Moved broker resources to KafkaNodePool.spec.resources
- Added controller role to KafkaNodePool (KRaft requires both controller+broker)
2026-05-26 02:47:43 +00:00
sirius0xdev
506b093377
Merge pull request #172 from sirius0xdev/fix/strimzi-template-fix
fix trading-kafka: move broker resources to KafkaNodePool
2026-05-25 22:29:24 -04:00
Hermes Agent
9eb36d7292 fix trading-kafka: move broker resources to KafkaNodePool 2026-05-26 02:29:04 +00:00
sirius0xdev
631a47cc5c
Merge pull request #171 from sirius0xdev/fix/strimzi-v1-api-v2
bump trading-kafka to Strimzi 1.0.0 v1 API (KafkaNodePool)
2026-05-25 22:19:48 -04:00
Hermes Agent
c477e80790 bump trading-kafka to Strimzi 1.0.0 v1 API (KafkaNodePool) 2026-05-26 02:11:25 +00:00
sirius0xdev
289acdf7d6
Merge pull request #170 from sirius0xdev/fix/strimzi-v1-api
bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0
2026-05-25 22:05:46 -04:00
Hermes Agent
b70f0514f4 bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0 2026-05-26 02:05:24 +00:00
Hermes Agent
8865b76154 Revert "bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0"
This reverts commit cbdea178cf.
2026-05-26 02:05:15 +00:00
Hermes Agent
cbdea178cf bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0 2026-05-26 02:02:35 +00:00
sirius0xdev
d21a272b2e
Merge pull request #169 from sirius0xdev/fix/strimzi-vendor-local
vendor Strimzi 1.0.0 YAML locally instead of remote URL
2026-05-25 21:58:59 -04:00
Hermes Agent
7e9b563257 vendor Strimzi 1.0.0 YAML locally instead of remote URL
strimzi.io/install/1.0.0 is not a valid git repo or kustomize remote URL.
Download the full install bundle (strimzi-1.0.0.yaml) and reference it
locally in the strimzi kustomization.
2026-05-26 01:53:06 +00:00
sirius0xdev
b439052079
Merge pull request #168 from sirius0xdev/feat/replace-kafka-with-strimzi
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:36:41 -04:00
Hermes Agent
d920cc7c63 add customer1-strimzi Flux Kustomization + force:true on customer1
Split Strimzi operator deployment into its own Flux Kustomization so
CRDs are applied before the Kafka CR in customer1.

Chain: infrastructure-controllers -> customer1-strimzi -> customer1

customer1 now has force: true to skip dry-run validation when CRDs are
recently applied but not yet visible to the APIServer cache.
2026-05-26 01:32:57 +00:00
Hermes Agent
47aae85ed3 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:32:57 +00:00
Hermes Agent
94f6b04c28 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 01:32:57 +00:00
Hermes Agent
25850b5b04 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 01:32:57 +00:00
sirius0xdev
e060e7a108
Merge pull request #167 from sirius0xdev/feat/strimzi-kafka-replacement
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:15:18 -04:00
Hermes Agent
1d6f33d449 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:09:56 +00:00
Hermes Agent
07e4b07608 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 00:49:08 +00:00
sirius0xdev
55b2a6a306
Merge pull request #166 from sirius0xdev/feat/strimzi-kafka-replacement
replace manual Kafka StatefulSet with Strimzi operator CR
2026-05-25 20:40:59 -04:00
Hermes Agent
d012cac7f4 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 00:38:57 +00:00
sirius0xdev
572f9f2c0f
Merge pull request #165 from sirius0xdev/fix/kafka-mount-path
fix Kafka config mount: use /opt/kafka/config/ path instead of /etc/k…
2026-05-25 20:17:14 -04:00