fix error
This commit is contained in:
parent
c420d2478c
commit
c7f0fc7262
1 changed files with 1 additions and 0 deletions
|
|
@ -36,6 +36,7 @@ spec:
|
||||||
- --host=0.0.0.0
|
- --host=0.0.0.0
|
||||||
- --port=8000
|
- --port=8000
|
||||||
- --tensor-parallel-size=1
|
- --tensor-parallel-size=1
|
||||||
|
- --tokenizer-mode=hf
|
||||||
- --gpu-memory-utilization=0.95
|
- --gpu-memory-utilization=0.95
|
||||||
- --kv-cache-dtype=fp8
|
- --kv-cache-dtype=fp8
|
||||||
- --max-model-len=49152
|
- --max-model-len=49152
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue