Commit graph

998 commits

Author SHA1 Message Date
sirius0xdev
b439052079
Merge pull request #168 from sirius0xdev/feat/replace-kafka-with-strimzi
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:36:41 -04:00
Hermes Agent
d920cc7c63 add customer1-strimzi Flux Kustomization + force:true on customer1
Split Strimzi operator deployment into its own Flux Kustomization so
CRDs are applied before the Kafka CR in customer1.

Chain: infrastructure-controllers -> customer1-strimzi -> customer1

customer1 now has force: true to skip dry-run validation when CRDs are
recently applied but not yet visible to the APIServer cache.
2026-05-26 01:32:57 +00:00
Hermes Agent
47aae85ed3 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:32:57 +00:00
Hermes Agent
94f6b04c28 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 01:32:57 +00:00
Hermes Agent
25850b5b04 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 01:32:57 +00:00
sirius0xdev
e060e7a108
Merge pull request #167 from sirius0xdev/feat/strimzi-kafka-replacement
Replace manual Kafka StatefulSet with Strimzi operator
2026-05-25 21:15:18 -04:00
Hermes Agent
1d6f33d449 split Strimzi operator into separate kustomization for Flux dependsOn
Flux dry-run fails when CRDs are in the same kustomization as resources
that reference them - CRDs need to exist before validation.

Two kustomizations now:
  strimzi/        - Strimzi 1.0.0 CRDs + operator
  siriusdevops-db - DBs, Kafka CR (depends on strimzi CRDs)

Wire them in Flux:
  Kustomization 'siriusdevops-strimzi' -> path: strimzi/
  Kustomization 'siriusdevops-db'      -> path: siriusdevops-db/
    dependsOn: [{name: siriusdevops-strimzi}]
2026-05-26 01:09:56 +00:00
Hermes Agent
07e4b07608 add Strimzi operator as kustomize remote URL (GitOps)
Reference Strimzi 1.0.0 install bundle directly from kustomization.
No separate install step needed — kubectl apply -k handles CRDs + operator
in one shot before the Kafka CR.

Pinned to 1.0.0 for reproducible deployments.
2026-05-26 00:49:08 +00:00
sirius0xdev
55b2a6a306
Merge pull request #166 from sirius0xdev/feat/strimzi-kafka-replacement
replace manual Kafka StatefulSet with Strimzi operator CR
2026-05-25 20:40:59 -04:00
Hermes Agent
d012cac7f4 replace manual Kafka StatefulSet with Strimzi operator CR
Remove hand-rolled KRaft StatefulSet that kept failing (mount conflicts,
advertised.listeners issues). Replace with a single Strimzi Kafka CR.

Strimzi handles:
- KRaft setup, storage, scaling
- Broker config & advertised listeners (auto-resolved)
- Topic Operator for proper topic lifecycle
- Bootstrap service at trading-kafka-kafka-bootstrap.customer1.svc.cluster.local:9092

Requires Strimzi 1.0.0 operator installed in the cluster first:
  kubectl apply -f https://strimzi.io/install/latest?namespace=customer1 -n customer1
2026-05-26 00:38:57 +00:00
sirius0xdev
572f9f2c0f
Merge pull request #165 from sirius0xdev/fix/kafka-mount-path
fix Kafka config mount: use /opt/kafka/config/ path instead of /etc/k…
2026-05-25 20:17:14 -04:00
Hermes Agent
68033faf4e fix Kafka config mount: use /opt/kafka/config/ path instead of /etc/kafka
/etc/kafka doesn't exist in the apache/kafka image - subPath could not
create the parent directory. Switch to /opt/kafka/config/ which exists
in the image, avoiding both the directory conflict and the missing path.
2026-05-26 00:16:42 +00:00
sirius0xdev
9c90b77210
Merge pull request #164 from sirius0xdev/fix/kafka-config-mount-subpath
fix Kafka config mount: use subPath to avoid /etc/kafka/secrets conflict
2026-05-25 19:55:35 -04:00
Hermes Agent
5ab0098e34 fix Kafka config mount: use subPath to avoid /etc/kafka/secrets conflict
Container fails with 'read-only file system' when K8s tries to mount
a secret into /etc/kafka/secrets alongside the ConfigMap at /etc/kafka.
Switch to subPath mount of server.properties directly.
2026-05-25 23:53:55 +00:00
sirius0xdev
8fc6bc16c3
Merge pull request #163 from sirius0xdev/fix/kafka-env-var-mismatch
fix configmap Kafka env var keys to match app code
2026-05-25 19:42:11 -04:00
Hermes Agent
32a855c457 fix configmap Kafka env var keys to match app code
App code reads KAFKA_BOOTSTRAP_SERVERS (all services), but configmaps
had KAFKA_BROKER (data) and EXECUTE_KAFKA_BOOTSTRAP_SERVERS (execute).
Services silently fell through to hardcoded defaults — Kafka unreachable.

- data-service: KAFKA_BROKER -> KAFKA_BOOTSTRAP_SERVERS
- execute-service: EXECUTE_KAFKA_BOOTSTRAP_SERVERS -> KAFKA_BOOTSTRAP_SERVERS
2026-05-25 23:40:15 +00:00
sirius0xdev
a2a4d9d732
Merge pull request #162 from sirius0xdev/fix/move-kafka-to-siriusdevops-db
move Kafka broker from hermes-db to siriusdevops-db kustomization
2026-05-25 19:37:28 -04:00
Hermes Agent
691d7c68e7 move Kafka broker from hermes-db to siriusdevops-db kustomization
Trading services depend on Kafka (trading-kafka.customer1.svc.cluster.local:9092)
but it was never deployed — lived under hermes-db kustomization which isn't
applied. Move to siriusdevops-db where trading infra belongs.

Also remove stale redis-cluster.yaml reference from hermes-db (file missing).
2026-05-25 23:36:14 +00:00
sirius0xdev
c92cbafd75
Merge pull request #161 from sirius0xdev/fix/pgvector-sa-key-auth
fix: switch pgvector image build to SA key auth
2026-05-25 18:24:48 -04:00
Hermes DevOps
eec93f6216 fix: switch pgvector image build to SA key auth (bypass broken WIF) 2026-05-25 21:19:44 +00:00
sirius0xdev
3643cf40de
Merge pull request #160 from sirius0xdev/fix/data-service-probes
fix(data-service): probe /health/ready instead of /health
2026-05-25 17:18:57 -04:00
Hermes Agent
9cbf21173f fix(data-service): probe /health/ready instead of /health 2026-05-25 21:17:10 +00:00
sirius0xdev
c98a69580f
Merge pull request #159 from sirius0xdev/fix/pgvector-workflow-wif
fix: use correct workload identity provider for GCR auth
2026-05-25 16:40:56 -04:00
sirius0xdev
6877736916
Merge pull request #157 from sirius0xdev/migrate-trading-db-network-policy
fix(trading): update network policy egress to siriusdevops-pgdb
2026-05-25 16:40:42 -04:00
Hermes Agent
1b62af55df fix: use correct workload identity provider for GCR auth 2026-05-25 20:29:49 +00:00
sirius0xdev
33118b04eb
fix: add id-token permission for OIDC auth to GCR (#158)
Co-authored-by: Hermes Agent <hermes-agent@sirius0xdev.com>
2026-05-25 16:24:46 -04:00
sirius0xdev
e28d6ccf10 fix(trading): update network policy egress to siriusdevops-pgdb
Migrate trading platform DB egress rule from hermes-pgdb to
siriusdevops-pgdb to complete the trading DB migration. The
configmaps already point to siriusdevops-pgdb-rw but the
network policy still allowed egress to the old cluster.
2026-05-25 20:08:07 +00:00
sirius0xdev
558a5a7c9e turn on gpu 2026-05-25 19:08:02 +00:00
sirius0xdev
c357b69743
Merge pull request #156 from sirius0xdev/fix/pgvector-image-and-cluster
chore(news-bot): switch DB to siriusdevops-pgdb (main apps cluster)
2026-05-25 14:56:56 -04:00
Hermes Agent
22498cd97b chore(news-bot): switch to siriusdevops-pgdb as main apps cluster
- news-app-db.yaml: point Database CR to siriusdevops-pgdb
- All ConfigMaps: DB_HOST=siriusdevops-pgdb-rw, LLM_BASE_URL for vLLM
- hermes-pgdb now dedicated to pgvector/RAG for agents only
- Aligns with CNPG extension pattern and active kanban (devops startupProbes, backend exec-service)

PR to master per repo rules.
2026-05-25 18:56:08 +00:00
sirius0xdev
67410a8ac0
Merge pull request #155 from sirius0xdev/fix/pgvector-image-and-cluster
feat: add pgvector image build and update cluster imageName
2026-05-25 14:03:31 -04:00
sirius0xdev
6ca9d9532f
Merge pull request #154 from sirius0xdev/feat/migrate-trading-to-siriusdevops-pgdb
feat: migrate all trading-platform services from hermes-pgdb to siriusdevops-pgdb
2026-05-25 14:03:20 -04:00
sirius0xdev
9642795258
Update kustomization.yaml 2026-05-25 08:49:12 -04:00
Hermes Agent
2f15684895 feat: add pgvector image build workflow and update cluster imageName
- Add build-pgvector-image.yml GH Actions workflow for GCR
- Set imageName to gcr.io/devops-lab-cluster/postgres-pgvector:15.2-0.8.0
- Image change only (CNPG rule: no config changes in same update)
2026-05-25 08:15:12 +00:00
Hermes Agent
3f96800222 feat: migrate all trading-platform services from hermes-pgdb to siriusdevops-pgdb
- Add 'trading' managed role to siriusdevops-pgdb cluster
- Move trading-db-credentials from hermes-db to siriusdevops-db
- Create trading-data-db.yaml (Database kind) pointing to siriusdevops-pgdb
- Update DB_HOST in all 4 trading-platform configmaps to siriusdevops-pgdb-rw
- Remove trading-data-db.yaml and trading-db-credentials.yaml from hermes-db
- Update siriusdevops-db kustomization.yaml to include new resources
2026-05-25 07:55:15 +00:00
sirius0xdev
1ba5a4e8a6
Merge pull request #153 from sirius0xdev/fix/trading-dashboard-tailscale-netpol
fix(platform): allow tailscale namespace in NetworkPolicy for dashboard proxy access
2026-05-25 03:16:28 -04:00
Hermes DevOps
7e8fd296c4 fix(platform): allow tailscale namespace in NetworkPolicy for proxy access
Tailscale proxy pods run in the tailscale namespace, so the existing
customer1-only namespaceSelector blocked proxy → dashboard traffic.
2026-05-25 07:15:59 +00:00
sirius0xdev
10d9295eb2
Merge pull request #152 from sirius0xdev/fix/trading-dashboard-network-policy
fix(platform): add port 3000 to trading-platform NetworkPolicy
2026-05-25 03:07:18 -04:00
Hermes DevOps
113324abc1 fix(platform): add port 3000 to trading-platform NetworkPolicy ingress rules
Dashboard listens on port 3000 but NetworkPolicy only allowed 8000-8003.
This blocked all traffic including Tailscale proxy connections to trade:3000.
2026-05-25 07:05:45 +00:00
sirius0xdev
4f4f761fdd
Update kustomization.yaml 2026-05-25 02:44:21 -04:00
Hermes DevOps
6cec18cdd9 fix(dashboard): add ProxyGroup for trading dashboard tailnet exposure
The Tailscale operator v0.40+ requires a ProxyGroup CRD to actually
expose services via the operator - annotations on the Service alone
are insufficient. This adds the missing ProxyGroup resource.

Also fixes the tailnet kustomization.yaml which referenced proxygroup
files that did not exist in that directory (they live in controllers/
and are deployed separately).
2026-05-25 06:44:14 +00:00
sirius0xdev
24a7a70719
Merge pull request #150 from sirius0xdev/fix/dashboard-tailscale-hostname
fix(dashboard): change tailscale hostname to "trade" for reliable resolution
2026-05-25 02:33:36 -04:00
569a3d5de9 fix(dashboard): change tailscale hostname to "trade" for reliable resolution
Short hostname "trade" (matches /trade HTTPRoute) instead of "trading-dashboard".

This fixes the "Could not resolve host: trading-dashboard" error on tailnet devices while keeping the same annotation style as Grafana.
2026-05-25 06:32:24 +00:00
4be12802d0 fix(execute-service): add timeoutSeconds: 10 to startup/readiness probes
The /health/ready endpoint calls await initialize() on Hyperliquid and Solana executors. Without explicit timeout, the default 1s probe timeout caused repeated failures even though the app was running.

Added timeoutSeconds: 10 + reduced readiness failureThreshold.

This is the final piece for the "running but not ready" state.
2026-05-25 06:26:09 +00:00
sirius0xdev
3d873e087e
Merge pull request #148 from sirius0xdev/fix/dashboard-volume-tmp
fix(dashboard): add tmp volumeMount for readOnlyRootFilesystem
2026-05-25 02:18:06 -04:00
605e15d55c fix(dashboard): add tmp volumeMount for readOnlyRootFilesystem
Next.js standalone mode needs a writable /tmp directory for cache, temp files, and logs.

With , the container was crashing after "Ready in 228ms" (EOF in log stream).

Added the standard emptyDir volumeMount used by other services.

This should make the dashboard stable and accessible on tailnet.
2026-05-25 06:17:28 +00:00
sirius0xdev
a2c12deff1
Merge pull request #147 from sirius0xdev/fix/execute-service-startup-probe
fix(execute-service): update startupProbe to /health/ready with failureThreshold 30
2026-05-25 02:09:53 -04:00
abecefe0e1 fix(execute-service): update startupProbe to /health/ready with 30 failureThreshold
The service was 'running but not ready' because:
- startupProbe used /health (always-ok) with only failureThreshold: 6 (~30s grace)
- Executor initialization (Hyperliquid SDK network calls + Solana client) takes longer

Now uses /health/ready (checks actual executor state) with failureThreshold: 30 (~5min grace), matching the pattern used for data-service and dashboard.

Also updated Helm template for consistency.
2026-05-25 06:09:20 +00:00
sirius0xdev
c34c669ec9
Merge pull request #146 from sirius0xdev/fix/dashboard-tailnet-annotation
fix(dashboard): match Grafana-style Tailscale annotation with explicit http:3000
2026-05-25 02:08:03 -04:00
ceb832b169 fix(dashboard): add explicit tailscale.com/ports: "http:3000" annotation
Matches the exact annotation pattern used for other tailnet services (Grafana, trade-dashboard, hermes-webui, etc.).

This ensures the Tailscale operator correctly maps the exposed port after the 3000 change.
2026-05-25 06:05:51 +00:00