tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-26 11:45:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

llama-bench : add defrag-thold, check for invalid ranges (#13487)

Browse Source
This commit is contained in:
Diego Devesa
2025-05-12 15:31:37 -07:00
committed by GitHub
parent f0d46ef157
commit cf0a43bb64
3 changed files with 49 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
include/llama.h
View File

@ -345,7 +345,7 @@ extern "C" {
float yarn_beta_fast; // YaRN low correction dim
float yarn_beta_slow; // YaRN high correction dim
uint32_t yarn_orig_ctx; // YaRN original context size
float defrag_thold; // defragment the KV cache if holes/size > thold, < 0 disabled (default)
float defrag_thold; // defragment the KV cache if holes/size > thold, <= 0 disabled (default)
ggml_backend_sched_eval_callback cb_eval;
void * cb_eval_user_data;