sirius/gcloud-lab
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
888 commits 1 branch 0 tags 1.5 MiB
Commit graph

555 commits

Author SHA1 Message Date
Hermes Agent
3f96800222 feat: migrate all trading-platform services from hermes-pgdb to siriusdevops-pgdb 
- Add 'trading' managed role to siriusdevops-pgdb cluster
- Move trading-db-credentials from hermes-db to siriusdevops-db
- Create trading-data-db.yaml (Database kind) pointing to siriusdevops-pgdb
- Update DB_HOST in all 4 trading-platform configmaps to siriusdevops-pgdb-rw
- Remove trading-data-db.yaml and trading-db-credentials.yaml from hermes-db
- Update siriusdevops-db kustomization.yaml to include new resources
 2026-05-25 07:55:15 +00:00
Hermes Agent
f17ab24ecc fix: update embedding-service image to GHCR 
Point to ghcr.io/sirius0xdev/trading-embedding-service:latest
built automatically by hermes-projects CI on push to main
 2026-05-25 01:35:28 +00:00
Hermes Agent
27b7334dfb fix: match pg-cluster manifest to live CNPG cluster state 
- Remove imageName (was never applied, live cluster uses default CNPG image)
- Remove maintenance_work_mem (live cluster has no custom parameters)
- Set postgresql: {} to match live state
- Keep memory role superuser: true (the only needed change)

Fixes CNPG admission webhook: 'Can't change image name and configuration at the same time'
 2026-05-25 01:11:27 +00:00
Hermes Agent
9f60ca894d fix: remove unsupported sql field from agent-memory-rag Database 
CNPG version on cluster does not support .spec.sql on Database CR.
The rag-init-job.yaml already handles RAG schema initialization separately.

Unblocks Flux dry-run for agent-memory-rag.
 2026-05-25 01:08:12 +00:00
Hermes Agent
9754c2aed2 fix: remove unsupported CNPG fields, keep memory superuser 
- Remove sharedPreloadLibraries and allowedExtensions (not supported by installed CNPG version)
- Keep memory role superuser: true (from PR #135) — sufficient since custom pgvector image already bundles the extension
- Unblocks dry-run validation on hermes-pgdb cluster
 2026-05-25 01:01:52 +00:00
sirius0xdev
ead9cff2d3
Merge pull request #134 from sirius0xdev/backend/t_6977ed0c-fix-execute-db-config 
fix: align execute-service DB config with EXECUTE_ env prefix
 2026-05-24 20:59:27 -04:00
sirius0xdev
9731719564
Merge pull request #135 from sirius0xdev/fix/t_3d94b392-cnpg-allowed-pgvector 
fix: enable pgvector for mem0 plugin
 2026-05-24 20:54:15 -04:00
sirius0xdev
8ccd21500e fix secrets 2026-05-25 00:49:20 +00:00
Hermes Agent
9ca7f7bd7a fix: enable pgvector for mem0 plugin 
- Add allowedExtensions: ['*'] so roles can CREATE EXTENSION
- Grant superuser to 'memory' role (was createdb/createrole, insufficient)
- Needed for mem0 plugin pgvector backend on agent_memory DB

Unblocks kanban task t_3d94b392
 2026-05-25 00:36:04 +00:00
Hermes Agent
a2644b39c7 fix: rename K8s env vars to EXECUTE_ prefix for execute-service 
- ConfigMap keys: DB_HOST -> EXECUTE_DB_HOST, DB_PORT -> EXECUTE_DB_PORT, etc.
- Deployment secrets: DB_USER -> EXECUTE_DB_USER, DB_PASSWORD -> EXECUTE_DB_PASSWORD
- Helm configmap: add EXECUTE_DB_* keys alongside legacy POSTGRES_* keys
- Base deployment: add EXECUTE_DB_USER/PASSWORD from secrets
 2026-05-24 23:57:04 +00:00
Hermes Agent
c04823d416 Grant CREATEDB and CREATEROLE to memory role on hermes-pgdb 
The memory user needs CREATEDB and CREATEROLE privileges to install
the pgvector extension on the agent_memory database.
 2026-05-24 22:14:49 +00:00
sirius0xdev
8bc3738a3f
Merge pull request #127 from sirius0xdev/feat/add-trading-dashboard-db-to-siriusdevops-pgdb 
Feat/add trading dashboard db to siriusdevops pgdb
 2026-05-24 16:56:27 -04:00
sirius0xdev
9f9bdba2b7
Merge pull request #128 from sirius0xdev/backend/t_438b663e-pgvector-rag-kb 
feat(customer1): add pgvector RAG knowledge base with embedding service
 2026-05-24 16:56:06 -04:00
Hermes Agent
9723674d9e feat(customer1): add embedding service kustomization, Service manifest, and staging reference 2026-05-24 20:33:50 +00:00
Hermes Agent
4df7450461 feat(customer1): add pgvector RAG knowledge base with embedding service 
- Custom PostgreSQL 15.2 image with pgvector 0.8.0 extension
- Updated pg-cluster-hermes.yaml: custom image, sharedPreloadLibraries, maintenance_work_mem
- RAG schema: documents table with vector(768) embeddings + HNSW index
- RAG init job: ConfigMap + Job to apply schema to agent_memory db
- Embedding service: FastAPI with nomic-embed-text-v1.5
  - OpenAI-compatible /v1/embeddings endpoint
  - Deployment (1 replica, 2Gi-4Gi memory) + Service manifests
- Updated kustomization.yaml to include new resources
 2026-05-24 20:31:59 +00:00
Hermes Agent
57e41178c3 feat(customer1): add trading_dashboard database to siriusdevops-pgdb CNPG cluster 
- Add trading_dashboard role + Database CR on siriusdevops-pgdb
- Create SOPS-encrypted credentials secret (template)
- Update trade-dashboard to use the new dedicated DB
  - New host: siriusdevops-pgdb-rw
  - New database: trading_dashboard
  - New credentials secret
 2026-05-24 20:02:20 +00:00
Hermes Agent
1d2d016eda fix(data-service): inject REDIS_URL from ConfigMap 
- Add REDIS_URL env var using variable expansion from the existing
  trading-data-service-config ConfigMap (REDIS_HOST + REDIS_PORT).
- This resolves the 'connecting to localhost:6379' crash on startup.
- Matches the actual infrastructure configuration in customer1.
 2026-05-23 18:34:51 +00:00
sirius0xdev
ac60dc5a51 Fix NATS URL service name mismatch and split combined ConfigMap+StatefulSet 
- F8: Correct NATS_URLS, REDIS_URL, MINIO_ENDPOINT to use actual service names
  (nats, redis-master, minio) instead of fullname-prefixed names that don't exist
- F9: Split combined ConfigMap+StatefulSet in nats/statefulset.yaml into separate
  configmap.yaml and statefulset.yaml files for maintainability
 2026-05-23 05:12:24 +00:00
sirius0xdev
54f2f42710 fix(osint-dashboard): fix critical manifest issues found during audit 
- Fix PostgreSQL topologySpreadConstraints label selector (cnpg.io/cluster)
- Fix Ingress HTTPRoutes: route API and WS traffic to correct backends
- Fix API NetworkPolicy port mismatch (4000 -> 8000)
- Fix MinIO bucket index syntax (_0 -> 0) in ConfigMap
- Add postgresql.port and postgresql.database to values.yaml
- Add container-level securityContext to API deployment
- Restrict monitoring NetworkPolicy ingress to monitoring namespace
- Enable readOnlyRootFilesystem for MinIO container
- Add activeDeadlineSeconds (3600s) to all 4 CronJobs
 2026-05-23 04:38:51 +00:00
sirius0xdev
cc2074a4cb change hermes image 2026-05-22 03:09:59 +00:00
sirius0xdev
f3c2e4991b change hermes image 2026-05-22 03:06:36 +00:00
sirius0xdev
c76675335d switch hermes image to main 2026-05-22 02:33:54 +00:00
sirius0xdev
75cea5abb7 fix secret issue 2026-05-22 02:11:22 +00:00
Hermes Agent
a433be1195 fix: resolve merge conflict in osint-dashboard values.yaml (keep real API config) 2026-05-21 14:17:52 +00:00
Sirius Devops
93754dcdf6 feat: OSINT Dashboard app + Helm templates 
Add FastAPI backend for real-time geospatial OSINT dashboard:
- Full-text search via PostgreSQL tsvector (parameterized queries)
- Entity tracking, alert management, sentiment analytics
- Data ingestion: RSS feeds, GDELT, USGS earthquakes, social signals
- NATS JetStream consumer for event ingestion
- MinIO document storage integration
- Redis caching layer
- Alembic migrations with PostGIS + TimescaleDB extensions
- Single-page dashboard UI with live polling
- OpenTelemetry distributed tracing

Helm chart with infrastructure:
- CNPG PostgreSQL cluster (PostGIS + TimescaleDB)
- NATS JetStream with persistent streams
- MinIO distributed object storage (3 buckets)
- Redis Sentinel (1 primary + 2 replicas)
- NGINX Ingress with TLS and WebSocket support
- Prometheus + Grafana + Alertmanager monitoring stack
- Network policies with default deny
- ConfigMap, CronJob, Deployment, Service templates

Fixes applied during review:
- SQL injection in search endpoint (parameterized :q binding)
- Dockerfile PYTHONPATH mismatch (/app/app -> /app)
- Hardcoded DB credentials in alembic.ini
- RSS timestamp parsing (feedparser published_parsed -> parsedate_to_datetime)
- Removed dead PGVECTOR import
 2026-05-21 13:39:52 +00:00
sirius0xdev
4a01c457ab
Update kustomization.yaml 2026-05-21 09:36:11 -04:00
Sirius Devops
8255467313 feat: add OSINT Dashboard Kubernetes infrastructure 
- Helm chart scaffold (Chart.yaml, values.yaml, _helpers.tpl)
- Namespace + RBAC manifests
- PostgreSQL (CNPG, 3 replicas, PostGIS + TimescaleDB)
- NATS JetStream (3 replicas, persistent, custom subjects)
- Redis Sentinel (1 primary + 2 replicas, HA)
- MinIO distributed (4 replicas, bucket init job)
- Gateway API HTTPRoute + cert-manager TLS certificates
- Monitoring stack (Prometheus, Grafana, Alertmanager, exporters)
- NetworkPolicies (default deny + per-component policies)
- GitHub Actions CI/CD pipeline (lint, template, security scan)
- Flux CD staging overlay
 2026-05-21 13:25:15 +00:00
Sirius Devops
b0748538c0 feat: add trading-platform K8s manifests for customer1 
- Scaffold trading-platform under apps/base/customer1/trading-platform/
- Add 4 microservice deployments: data-service, execute-service, news-service, dashboard
- Add ConfigMaps with DB/Kafka/Redis connection configs
- Add HTTPRoutes via Gateway API (external-http-gateway)
- Add NetworkPolicies for inter-service and DB/Kafka/Redis egress
- Add SOPS-encrypted secrets (age key)
- Add Kafka (KRaft) and Redis infrastructure to hermes-db/
- Update root and staging kustomizations

All containers: non-root, readOnlyRootFilesystem, resource limits, health probes
Images: ghcr.io/sirius0xdev/trading-{service}:latest
 2026-05-21 04:17:05 +00:00
Sirius Devops
5e477492e6 feat: add OSINT Dashboard Kubernetes infrastructure 
- Helm chart scaffold (Chart.yaml, values.yaml, _helpers.tpl)
- Namespace + RBAC manifests
- PostgreSQL (CNPG, 3 replicas, PostGIS + TimescaleDB)
- NATS JetStream (3 replicas, persistent, custom subjects)
- Redis Sentinel (1 primary + 2 replicas, HA)
- MinIO distributed (4 replicas, bucket init job)
- Gateway API HTTPRoute + cert-manager TLS certificates
- Monitoring stack (Prometheus, Grafana, Alertmanager, exporters)
- NetworkPolicies (default deny + per-component policies)
- GitHub Actions CI/CD pipeline (lint, template, security scan)
- Flux CD staging overlay
 2026-05-21 04:15:55 +00:00
sirius0xdev
a06a105f31
Update kustomization.yaml 2026-05-20 22:18:37 -04:00
sirius0xdev
da0ad4b7b9
Merge pull request #116 from sirius0xdev/feat/siriusdevops-site-deploy 
feat: deploy siriusdevops.com main site to GKE
 2026-05-15 11:02:01 -04:00
Sirius Devops
65e174a86d feat: deploy siriusdevops.com main site to GKE 
Add siriusdevops-site manifests:
- Deployment + ClusterIP Service (nginx:alpine, port 80)
- HTTPRoute for siriusdevops.com via external-http-gateway
- Kustomization overlay wiring
 2026-05-15 14:56:10 +00:00
Sirius Devops
334be37781 fix: add real Telegram bot_token and chat_id 2026-05-15 02:10:47 +00:00
Sirius Devops
31fcd59a5f fix: SOPS-encrypt waitlist secrets with age key 
- DB password: random 48-char hex string
- admin_api_key: random 32-char hex string
- Telegram bot_token/chat_id still CHANGEME — fill before deploying
 2026-05-15 01:36:06 +00:00
Sirius Devops
ac994f144a fix: container hardening + NetworkPolicy for waitlist-api 
- deployment.yaml: securityContext (runAsNonRoot, readOnlyRootFilesystem, drop ALL caps)
- network-policy.yaml: egress only to CNPG PG (5432) + api.telegram.org (443)
- kustomization.yaml: register new NetworkPolicy resource
- waitlist-telegram-secret: add admin_api_key field
 2026-05-15 00:00:58 +00:00
sirius0xdev
36b1d637f1
feat: add siriusdevops CNPG cluster + waitlist API K8s manifests (#112) 
- New CNPG PostgreSQL cluster (siriusdevops-pgdb) in customer1 namespace
- Single-instance, 20Gi storage, GCS backups to siriusdevops-backups/
- Waitlist database definition (waitlist-db.yaml)
- Scheduled backup manifest
- Waitlist API deployment, service, HTTPRoute, and Telegram secret placeholder
- Kustomization wiring for both siriusdevops-db and waitlist-api
 2026-05-14 19:24:33 -04:00
sirius0xdev
e3b56d397d
Update deployment.yaml 2026-05-12 22:42:41 -04:00
sirius0xdev
9beb9c4887
Update deployment.yaml 2026-05-12 22:40:46 -04:00
sirius0xdev
228f4d5f0e add version to webui 2026-05-11 13:23:43 +00:00
sirius0xdev
f173f4b84e edit config map 2026-05-11 02:02:47 +00:00
sirius0xdev
24d8a7ce65 fix secret name 2026-05-11 01:34:03 +00:00
sirius0xdev
649e69b585 fix deployment 2026-05-11 01:29:31 +00:00
sirius0xdev
fb32fdad1c fix new config map 2026-05-11 01:17:27 +00:00
sirius0xdev
9657d5bc18 add to kustomization 2026-05-11 01:12:22 +00:00
sirius0xdev
3eee3b3ecf fix cm name 2026-05-11 01:09:02 +00:00
sirius0xdev
e1178b3691 create seperate config maps 2026-05-11 01:03:01 +00:00
sirius0xdev
0f8bcf62e9 add webhook env 2026-05-10 17:48:35 +00:00
sirius0xdev
29a36f9776 webhook nightmare 2026-05-10 03:40:24 +00:00
sirius0xdev
1491f0c1b4 fix webhooks and try vllm config 2026-05-10 03:15:57 +00:00
sirius0xdev
bce6e18f2b fix vllm and tini 2026-05-10 01:48:34 +00:00