- Makes rtx6000-brain selectable in the /model provider picker (preferred name)
- Uses short Kubernetes service names (optimal for same-namespace)
- Keeps existing models: section for CLI compatibility
- Will appear as provider options after ArgoCD/Helm rollout + /restart
- Primary 'vllm' entry for easy /model vllm usage
- Uses model from the rtx6000 deployment
- Kept existing qwen-vllm, rtx6000-vllm, and rtx6000-brain for compatibility
Added rtx6000-brain alongside the existing rtx6000-vllm entry.
Uses the exact model name from rtx6000-vllm Deployment and the correct ClusterIP service endpoint.
- Removed cooldownPeriod, scaledownPeriod, paths (all undeclared in http.keda.sh/v1alpha1 CRD)
- Kept pollingInterval:10 for responsiveness + min:0 replicas for true scale-to-zero
- Cleaned both rtx6000 and a100 configs for consistency
Dry-run should now succeed. Scale behavior preserved via polling + low targetPendingRequests (defaults work well for vLLM).
HTTPScaledObject CRD (http.keda.sh/v1alpha1) does not support .spec.paths field.
Scaling now triggers on all HTTP requests to the host (health/liveness ignored via targetPendingRequests:1 + pollingInterval:10s).
Refs https://keda.sh/docs/scalers/http-addon/
- Switch to HTTPScaledObject (matches a100 pattern)
- Probes (/health, /v1/models) now ignored for scaling
- Keeps fast polling (10s), quick cooldown (30s), 15min idle scale-to-0
Health/readiness/startup probes hit pod IP directly (bypass service), so never triggered scaling anyway — but paths ensure only inference traffic scales.
- Rename deployment to rtx6000-brain-vllm to match scaler target ref
- Set deployment replicas to 0 (KEDA controlled)
- Add pollingInterval: 10s and cooldownPeriod: 30s for faster response
This enables scale-to-zero when idle and quick scale-up on first request.
See https://github.com/sirius0xdev/gcloud-lab/tree/master/infrastructure%2Fgpus%2Fbase
- Cilium Gateway API HTTPRoute routes /v1 traffic to Kong service
- Includes placeholders for Gateway/Kong names (update post-merge)
- Added to openclaw kustomization.yaml for FluxCD sync
Refs LLM gateway.md optimizations.
