7e7b7d414d
- Switch to HTTPScaledObject (matches a100 pattern) - Probes (/health, /v1/models) now ignored for scaling - Keeps fast polling (10s), quick cooldown (30s), 15min idle scale-to-0 Health/readiness/startup probes hit pod IP directly (bypass service), so never triggered scaling anyway — but paths ensure only inference traffic scales.
22 lines
471 B
YAML
22 lines
471 B
YAML
apiVersion: http.keda.sh/v1alpha1
|
|
kind: HTTPScaledObject
|
|
metadata:
|
|
name: rtx6000-scaling
|
|
namespace: customer1
|
|
spec:
|
|
hosts:
|
|
- rtx6000-brain-service.customer1.svc.cluster.local
|
|
scaleTargetRef:
|
|
name: rtx6000-brain-vllm
|
|
kind: Deployment
|
|
apiVersion: apps/v1
|
|
service: rtx6000-brain-service
|
|
port: 8000
|
|
paths:
|
|
- path: "^/v1/(chat/)?completions"
|
|
replicas:
|
|
min: 0
|
|
max: 1
|
|
pollingInterval: 10
|
|
cooldownPeriod: 30
|
|
scaledownPeriod: 900
|