Enables /v1/models endpoint query for custom providers (fixes qwen model not showing under qwen-vllm).
Changes:
- Added `type: openai` to `rtx6000-brain` and `qwen-vllm` providers
- Standardized base_url to full FQDN: \*.customer1.svc.cluster.local
- Added context_length to qwen-vllm provider
- Updated models.qwen-vllm base_url and context_length for consistency
- Set replicas.min: 0 for scale-to-zero when idle
- Add config.cooldownPeriod: 1200 to KEDA HTTP add-on HelmRelease
(20 min buffer before scaling down from 1 replica)
- Add external HTTPRoute at brain.siriusdevops.com for manual wake-up
via curl from Hermes/Telegram
- Makes rtx6000-brain selectable in the /model provider picker (preferred name)
- Uses short Kubernetes service names (optimal for same-namespace)
- Keeps existing models: section for CLI compatibility
- Will appear as provider options after ArgoCD/Helm rollout + /restart
- Primary 'vllm' entry for easy /model vllm usage
- Uses model from the rtx6000 deployment
- Kept existing qwen-vllm, rtx6000-vllm, and rtx6000-brain for compatibility
Added rtx6000-brain alongside the existing rtx6000-vllm entry.
Uses the exact model name from rtx6000-vllm Deployment and the correct ClusterIP service endpoint.
- Removed cooldownPeriod, scaledownPeriod, paths (all undeclared in http.keda.sh/v1alpha1 CRD)
- Kept pollingInterval:10 for responsiveness + min:0 replicas for true scale-to-zero
- Cleaned both rtx6000 and a100 configs for consistency
Dry-run should now succeed. Scale behavior preserved via polling + low targetPendingRequests (defaults work well for vLLM).
HTTPScaledObject CRD (http.keda.sh/v1alpha1) does not support .spec.paths field.
Scaling now triggers on all HTTP requests to the host (health/liveness ignored via targetPendingRequests:1 + pollingInterval:10s).
Refs https://keda.sh/docs/scalers/http-addon/
- Switch to HTTPScaledObject (matches a100 pattern)
- Probes (/health, /v1/models) now ignored for scaling
- Keeps fast polling (10s), quick cooldown (30s), 15min idle scale-to-0
Health/readiness/startup probes hit pod IP directly (bypass service), so never triggered scaling anyway — but paths ensure only inference traffic scales.
- Rename deployment to rtx6000-brain-vllm to match scaler target ref
- Set deployment replicas to 0 (KEDA controlled)
- Add pollingInterval: 10s and cooldownPeriod: 30s for faster response
This enables scale-to-zero when idle and quick scale-up on first request.
See https://github.com/sirius0xdev/gcloud-lab/tree/master/infrastructure%2Fgpus%2Fbase
