tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-17 05:25:09 -04:00
Code Issues Packages Projects Releases Wiki Activity

llama-chat : Do not throw when tool parsing fails (#14012)

Browse Source 
Currently when a model generates output which looks like a tool call,
but is invalid an exception is thrown and not handled, causing the cli
or llama-server to bail. Instead, handle the chat parser exception and
simply return the generated text in such cases.

Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com>
This commit is contained in:
Piotr
2025-06-14 18:25:15 +02:00
committed by GitHub
parent 2e42be42bd
commit 3cb203c89f
3 changed files with 10 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
common/chat.cpp
View File

@@ -1921,7 +1921,9 @@ common_chat_msg common_chat_parse(const std::string & input, bool is_partial, co
} catch (const common_chat_msg_partial_exception & ex) {
LOG_DBG("Partial parse: %s\n", ex.what());
if (!is_partial) {
throw std::runtime_error(ex.what());
builder.clear_tools();
builder.move_to(0);
common_chat_parse_content_only(builder);
}
}
auto msg = builder.result();