sirius/gcloud-lab
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #17 from sirius0xdev/fix/vllm-mistral-regex

Browse source 
fix: enforce mistral tokenizer mode
This commit is contained in:
sirius0xdev 2026-04-18 16:11:01 -04:00 • committed by GitHub
commit 518127c711
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
apps/base/customer1/openclaw/vllm-gemma.yaml
View file

@ -26,6 +26,13 @@ spec:
- name: vllm-brain - name: vllm-brain
image: vllm/vllm-openai:latest image: vllm/vllm-openai:latest
command: ["python3", "-m", "vllm.entrypoints.openai.api_server"] command: ["python3", "-m", "vllm.entrypoints.openai.api_server"]
env:
- name: VLLM_ATTENTION_BACKEND
value: "XFORMERS"
- name: HF_TOKEN
value: ""
- name: VLLM_TOKENIZER_MODE
value: "mistral"
args: args:
- "--model" - "--model"
- "dphn/Dolphin-Mistral-24B-Venice-Edition" # Uncensored 24B (Native) - "dphn/Dolphin-Mistral-24B-Venice-Edition" # Uncensored 24B (Native)