tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-27 03:55:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

tool-call: support Command R7B (+ return tool_plan "thoughts" in API) (#11585)

Browse Source 
* `tool-call`: support Command R7B (w/ tool_plan return)

* `tool-call`: cleaner preservation of tokens + warn when likely bad chat template override

* `tool-call`: test cleanup / handle lazy grammar triggers
This commit is contained in:
Olivier Chafik
2025-02-02 09:25:38 +00:00
committed by GitHub
parent 69804487e0
commit bfcce4d693
8 changed files with 420 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/server/utils.hpp
View File

@ -662,6 +662,7 @@ static json oaicompat_completion_params_parse(
});
}
llama_params["grammar_triggers"] = grammar_triggers;
llama_params["preserved_tokens"] = chat_params.preserved_tokens;
for (const auto & stop : chat_params.additional_stops) {
llama_params["stop"].push_back(stop);
}