tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-29 12:35:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

Don't tell users to use a bad number of threads (#243)

Browse Source 
The readme tells people to use the command line option "-t 8", causing 8
threads to be started. On systems with fewer than 8 cores, this causes a
significant slowdown. Remove the option from the example command lines
and use /proc/cpuinfo on Linux to determine a sensible default.
This commit is contained in:
Stephan Walter
2023-03-17 17:47:35 +00:00
committed by GitHub
parent 6b0df5ccf3
commit 367946c668
5 changed files with 19 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
utils.h
View File

@ -14,7 +14,7 @@
struct gpt_params {
int32_t seed = -1; // RNG seed
int32_t n_threads = std::min(4, (int32_t) std::thread::hardware_concurrency());
int32_t n_threads;
int32_t n_predict = 128; // new tokens to predict
int32_t repeat_last_n = 64; // last n tokens to penalize
int32_t n_ctx = 512; //context size