tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-07-02 05:15:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into compilade/bitnet-ternary

Browse Source
This commit is contained in:
Francis Couture-Harpin
2024-08-22 16:42:24 -04:00
parent 35cc5567c8 11b84eb457
commit cb6d9962c4
77 changed files with 4681 additions and 2212 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/quantize/README.md
View File

@ -34,7 +34,7 @@ Run the quantized model:
```bash
# start inference on a gguf model
./llama-cli -m ./models/mymodel/ggml-model-Q4_K_M.gguf -n 128
./llama-cli -m ./models/mymodel/ggml-model-Q4_K_M.gguf -cnv -p "You are a helpful assistant"
```
When running the larger models, make sure you have enough disk space to store all the intermediate files.