Commit graph

957 commits

Author SHA1 Message Date
Hermes Agent
9eb36d7292 fix trading-kafka: move broker resources to KafkaNodePool 2026-05-26 02:29:04 +00:00
sirius0xdev
631a47cc5c
Merge pull request #171 from sirius0xdev/fix/strimzi-v1-api-v2
bump trading-kafka to Strimzi 1.0.0 v1 API (KafkaNodePool)
2026-05-25 22:19:48 -04:00
Hermes Agent
c477e80790 bump trading-kafka to Strimzi 1.0.0 v1 API (KafkaNodePool) 2026-05-26 02:11:25 +00:00
sirius0xdev
289acdf7d6
Merge pull request #170 from sirius0xdev/fix/strimzi-v1-api
bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0
2026-05-25 22:05:46 -04:00
Hermes Agent
b70f0514f4 bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0 2026-05-26 02:05:24 +00:00
Hermes Agent
8865b76154 Revert "bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0"
This reverts commit cbdea178cf.
2026-05-26 02:05:15 +00:00
Hermes Agent
cbdea178cf bump trading-kafka API from v1beta2 to v1 for Strimzi 1.0.0 2026-05-26 02:02:35 +00:00
sirius0xdev
d21a272b2e
Merge pull request #169 from sirius0xdev/fix/strimzi-vendor-local
vendor Strimzi 1.0.0 YAML locally instead of remote URL
2026-05-25 21:58:59 -04:00
Hermes Agent
7e9b563257 vendor Strimzi 1.0.0 YAML locally instead of remote URL
strimzi.io/install/1.0.0 is not a valid git repo or kustomize remote URL.
Download the full install bundle (strimzi-1.0.0.yaml) and reference it
locally in the strimzi kustomization.
2026-05-26 01:53:06 +00:00
sirius0xdev
b439052079
Merge pull request #168 from sirius0xdev/feat/replace-kafka-with-strimzi
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:36:41 -04:00
Hermes Agent
d920cc7c63 add customer1-strimzi Flux Kustomization + force:true on customer1
Split Strimzi operator deployment into its own Flux Kustomization so
CRDs are applied before the Kafka CR in customer1.

Chain: infrastructure-controllers -> customer1-strimzi -> customer1

customer1 now has force: true to skip dry-run validation when CRDs are
recently applied but not yet visible to the APIServer cache.
2026-05-26 01:32:57 +00:00
Hermes Agent
47aae85ed3 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:32:57 +00:00
Hermes Agent
94f6b04c28 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 01:32:57 +00:00
Hermes Agent
25850b5b04 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 01:32:57 +00:00
sirius0xdev
e060e7a108
Merge pull request #167 from sirius0xdev/feat/strimzi-kafka-replacement
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:15:18 -04:00
Hermes Agent
1d6f33d449 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:09:56 +00:00
Hermes Agent
07e4b07608 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 00:49:08 +00:00
sirius0xdev
55b2a6a306
Merge pull request #166 from sirius0xdev/feat/strimzi-kafka-replacement
replace manual Kafka StatefulSet with Strimzi operator CR
2026-05-25 20:40:59 -04:00
Hermes Agent
d012cac7f4 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 00:38:57 +00:00
sirius0xdev
572f9f2c0f
Merge pull request #165 from sirius0xdev/fix/kafka-mount-path
fix Kafka config mount: use /opt/kafka/config/ path instead of /etc/k…
2026-05-25 20:17:14 -04:00
Hermes Agent
68033faf4e fix Kafka config mount: use /opt/kafka/config/ path instead of /etc/kafka
/etc/kafka doesn't exist in the apache/kafka image - subPath could not
create the parent directory. Switch to /opt/kafka/config/ which exists
in the image, avoiding both the directory conflict and the missing path.
2026-05-26 00:16:42 +00:00
sirius0xdev
9c90b77210
Merge pull request #164 from sirius0xdev/fix/kafka-config-mount-subpath
fix Kafka config mount: use subPath to avoid /etc/kafka/secrets conflict
2026-05-25 19:55:35 -04:00
Hermes Agent
5ab0098e34 fix Kafka config mount: use subPath to avoid /etc/kafka/secrets conflict
Container fails with 'read-only file system' when K8s tries to mount
a secret into /etc/kafka/secrets alongside the ConfigMap at /etc/kafka.
Switch to subPath mount of server.properties directly.
2026-05-25 23:53:55 +00:00
sirius0xdev
8fc6bc16c3
Merge pull request #163 from sirius0xdev/fix/kafka-env-var-mismatch
fix configmap Kafka env var keys to match app code
2026-05-25 19:42:11 -04:00
Hermes Agent
32a855c457 fix configmap Kafka env var keys to match app code
App code reads KAFKA_BOOTSTRAP_SERVERS (all services), but configmaps
had KAFKA_BROKER (data) and EXECUTE_KAFKA_BOOTSTRAP_SERVERS (execute).
Services silently fell through to hardcoded defaults — Kafka unreachable.

- data-service: KAFKA_BROKER -> KAFKA_BOOTSTRAP_SERVERS
- execute-service: EXECUTE_KAFKA_BOOTSTRAP_SERVERS -> KAFKA_BOOTSTRAP_SERVERS
2026-05-25 23:40:15 +00:00
sirius0xdev
a2a4d9d732
Merge pull request #162 from sirius0xdev/fix/move-kafka-to-siriusdevops-db
move Kafka broker from hermes-db to siriusdevops-db kustomization
2026-05-25 19:37:28 -04:00
Hermes Agent
691d7c68e7 move Kafka broker from hermes-db to siriusdevops-db kustomization
Trading services depend on Kafka (trading-kafka.customer1.svc.cluster.local:9092)
but it was never deployed — lived under hermes-db kustomization which isn't
applied. Move to siriusdevops-db where trading infra belongs.

Also remove stale redis-cluster.yaml reference from hermes-db (file missing).
2026-05-25 23:36:14 +00:00
sirius0xdev
c92cbafd75
Merge pull request #161 from sirius0xdev/fix/pgvector-sa-key-auth
fix: switch pgvector image build to SA key auth
2026-05-25 18:24:48 -04:00
Hermes DevOps
eec93f6216 fix: switch pgvector image build to SA key auth (bypass broken WIF) 2026-05-25 21:19:44 +00:00
sirius0xdev
3643cf40de
Merge pull request #160 from sirius0xdev/fix/data-service-probes
fix(data-service): probe /health/ready instead of /health
2026-05-25 17:18:57 -04:00
Hermes Agent
9cbf21173f fix(data-service): probe /health/ready instead of /health 2026-05-25 21:17:10 +00:00
sirius0xdev
c98a69580f
Merge pull request #159 from sirius0xdev/fix/pgvector-workflow-wif
fix: use correct workload identity provider for GCR auth
2026-05-25 16:40:56 -04:00
sirius0xdev
6877736916
Merge pull request #157 from sirius0xdev/migrate-trading-db-network-policy
fix(trading): update network policy egress to siriusdevops-pgdb
2026-05-25 16:40:42 -04:00
Hermes Agent
1b62af55df fix: use correct workload identity provider for GCR auth 2026-05-25 20:29:49 +00:00
sirius0xdev
33118b04eb
fix: add id-token permission for OIDC auth to GCR (#158)
Co-authored-by: Hermes Agent <hermes-agent@sirius0xdev.com>
2026-05-25 16:24:46 -04:00
sirius0xdev
e28d6ccf10 fix(trading): update network policy egress to siriusdevops-pgdb
Migrate trading platform DB egress rule from hermes-pgdb to
siriusdevops-pgdb to complete the trading DB migration. The
configmaps already point to siriusdevops-pgdb-rw but the
network policy still allowed egress to the old cluster.
2026-05-25 20:08:07 +00:00
sirius0xdev
558a5a7c9e turn on gpu 2026-05-25 19:08:02 +00:00
sirius0xdev
c357b69743
Merge pull request #156 from sirius0xdev/fix/pgvector-image-and-cluster
chore(news-bot): switch DB to siriusdevops-pgdb (main apps cluster)
2026-05-25 14:56:56 -04:00
Hermes Agent
22498cd97b chore(news-bot): switch to siriusdevops-pgdb as main apps cluster
- news-app-db.yaml: point Database CR to siriusdevops-pgdb
- All ConfigMaps: DB_HOST=siriusdevops-pgdb-rw, LLM_BASE_URL for vLLM
- hermes-pgdb now dedicated to pgvector/RAG for agents only
- Aligns with CNPG extension pattern and active kanban (devops startupProbes, backend exec-service)

PR to master per repo rules.
2026-05-25 18:56:08 +00:00
sirius0xdev
67410a8ac0
Merge pull request #155 from sirius0xdev/fix/pgvector-image-and-cluster
feat: add pgvector image build and update cluster imageName
2026-05-25 14:03:31 -04:00
sirius0xdev
6ca9d9532f
Merge pull request #154 from sirius0xdev/feat/migrate-trading-to-siriusdevops-pgdb
feat: migrate all trading-platform services from hermes-pgdb to siriusdevops-pgdb
2026-05-25 14:03:20 -04:00
sirius0xdev
9642795258
Update kustomization.yaml 2026-05-25 08:49:12 -04:00
Hermes Agent
2f15684895 feat: add pgvector image build workflow and update cluster imageName
- Add build-pgvector-image.yml GH Actions workflow for GCR
- Set imageName to gcr.io/devops-lab-cluster/postgres-pgvector:15.2-0.8.0
- Image change only (CNPG rule: no config changes in same update)
2026-05-25 08:15:12 +00:00
Hermes Agent
3f96800222 feat: migrate all trading-platform services from hermes-pgdb to siriusdevops-pgdb
- Add 'trading' managed role to siriusdevops-pgdb cluster
- Move trading-db-credentials from hermes-db to siriusdevops-db
- Create trading-data-db.yaml (Database kind) pointing to siriusdevops-pgdb
- Update DB_HOST in all 4 trading-platform configmaps to siriusdevops-pgdb-rw
- Remove trading-data-db.yaml and trading-db-credentials.yaml from hermes-db
- Update siriusdevops-db kustomization.yaml to include new resources
2026-05-25 07:55:15 +00:00
sirius0xdev
1ba5a4e8a6
Merge pull request #153 from sirius0xdev/fix/trading-dashboard-tailscale-netpol
fix(platform): allow tailscale namespace in NetworkPolicy for dashboard proxy access
2026-05-25 03:16:28 -04:00
Hermes DevOps
7e8fd296c4 fix(platform): allow tailscale namespace in NetworkPolicy for proxy access
Tailscale proxy pods run in the tailscale namespace, so the existing
customer1-only namespaceSelector blocked proxy → dashboard traffic.
2026-05-25 07:15:59 +00:00
sirius0xdev
10d9295eb2
Merge pull request #152 from sirius0xdev/fix/trading-dashboard-network-policy
fix(platform): add port 3000 to trading-platform NetworkPolicy
2026-05-25 03:07:18 -04:00
Hermes DevOps
113324abc1 fix(platform): add port 3000 to trading-platform NetworkPolicy ingress rules
Dashboard listens on port 3000 but NetworkPolicy only allowed 8000-8003.
This blocked all traffic including Tailscale proxy connections to trade:3000.
2026-05-25 07:05:45 +00:00
sirius0xdev
4f4f761fdd
Update kustomization.yaml 2026-05-25 02:44:21 -04:00
Hermes DevOps
6cec18cdd9 fix(dashboard): add ProxyGroup for trading dashboard tailnet exposure
The Tailscale operator v0.40+ requires a ProxyGroup CRD to actually
expose services via the operator - annotations on the Service alone
are insufficient. This adds the missing ProxyGroup resource.

Also fixes the tailnet kustomization.yaml which referenced proxygroup
files that did not exist in that directory (they live in controllers/
and are deployed separately).
2026-05-25 06:44:14 +00:00