tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-18 14:18:50 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs: fix links in development docs [no ci] (#8481)

Browse Source 
Fixes a few links to within the repo that were broken in the reorganization of the
documentation in #8325.
This commit is contained in:
NikolaiLyssogor
2024-07-15 04:46:39 -07:00
committed by GitHub
parent 16bdfa42ac
commit fc690b018e
2 changed files with 9 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/development/token_generation_performance_tips.md
View File

@@ -1,7 +1,7 @@
# Token generation performance troubleshooting
## Verifying that the model is running on the GPU with CUDA
Make sure you compiled llama with the correct env variables according to [this guide](../README.md#CUDA), so that llama accepts the `-ngl N` (or `--n-gpu-layers N`) flag. When running llama, you may configure `N` to be very large, and llama will offload the maximum possible number of layers to the GPU, even if it's less than the number you configured. For example:
Make sure you compiled llama with the correct env variables according to [this guide](/docs/build.md#cuda), so that llama accepts the `-ngl N` (or `--n-gpu-layers N`) flag. When running llama, you may configure `N` to be very large, and llama will offload the maximum possible number of layers to the GPU, even if it's less than the number you configured. For example:
```shell
./llama-cli -m "path/to/model.gguf" -ngl 200000 -p "Please sir, may I have some "
```