tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-19 14:31:06 -04:00
Code Issues Packages Projects Releases Wiki Activity

rpc : early register backend devices (#11262)

Browse Source 
Early register RPC devices and do not propagate RPC specifics in the
llama model structures.

ref: #10609
This commit is contained in:
Radoslav Gerganov
2025-01-17 10:57:09 +02:00
committed by GitHub
parent a133566d34
commit 667d72846c
10 changed files with 61 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
common/common.h
View File

@@ -246,7 +246,6 @@ struct common_params {
std::string lookup_cache_static = ""; // path of static ngram cache file for lookup decoding // NOLINT
std::string lookup_cache_dynamic = ""; // path of dynamic ngram cache file for lookup decoding // NOLINT
std::string logits_file = ""; // file for saving *all* logits // NOLINT
std::string rpc_servers = ""; // comma separated list of RPC servers // NOLINT
std::vector<std::string> in_files; // all input files
std::vector<std::string> antiprompt; // strings upon which more user input is prompted (a.k.a. reverse prompts)