tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-07-04 18:16:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

llama-bench : add defrag-thold, check for invalid ranges (#13487)

Browse Source
This commit is contained in:
Diego Devesa
2025-05-12 15:31:37 -07:00
committed by GitHub
parent f0d46ef157
commit cf0a43bb64
3 changed files with 49 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
tools/llama-bench/README.md
View File

@ -43,12 +43,13 @@ test parameters:
-ub, --ubatch-size <n> (default: 512)
-ctk, --cache-type-k <t> (default: f16)
-ctv, --cache-type-v <t> (default: f16)
-t, --threads <n> (default: 16)
-dt, --defrag-thold <f> (default: -1)
-t, --threads <n> (default: system dependent)
-C, --cpu-mask <hex,hex> (default: 0x0)
--cpu-strict <0|1> (default: 0)
--poll <0...100> (default: 50)
-ngl, --n-gpu-layers <n> (default: 99)
-rpc, --rpc <rpc_servers> (default: )
-rpc, --rpc <rpc_servers> (default: none)
-sm, --split-mode <none|layer|row> (default: layer)
-mg, --main-gpu <i> (default: 0)
-nkvo, --no-kv-offload <0|1> (default: 0)
@ -62,7 +63,7 @@ test parameters:
Multiple values can be given for each parameter by separating them with ','
or by specifying the parameter multiple times. Ranges can be given as
'start-end' or 'start-end+step' or 'start-end*mult'.
'first-last' or 'first-last+step' or 'first-last*mult'.
```
llama-bench can perform three types of tests: