tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-27 03:55:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

readme : add some recent perplexity and bpw measurements to READMES, link for k-quants (#3340)

Browse Source 
* Update README.md

* Update README.md

* Update README.md with k-quants bpw measurements
This commit is contained in:
BarfingLemurs
2023-09-27 11:30:36 -04:00
committed by GitHub
parent 99115f3fa6
commit ffe88a36a9
3 changed files with 64 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
examples/quantize/README.md
View File

@ -1,3 +1,44 @@
# quantize
TODO
## Llama 2 7B
Quantization | Bits per Weight (BPW)
-- | --
Q2_K | 3.35
Q3_K_S | 3.50
Q3_K_M | 3.91
Q3_K_L | 4.27
Q4_K_S | 4.58
Q4_K_M | 4.84
Q5_K_S | 5.52
Q5_K_M | 5.68
Q6_K | 6.56
## Llama 2 13B
Quantization | Bits per Weight (BPW)
-- | --
Q2_K | 3.34
Q3_K_S | 3.48
Q3_K_M | 3.89
Q3_K_L | 4.26
Q4_K_S | 4.56
Q4_K_M | 4.83
Q5_K_S | 5.51
Q5_K_M | 5.67
Q6_K | 6.56
# Llama 2 70B
Quantization | Bits per Weight (BPW)
-- | --
Q2_K | 3.40
Q3_K_S | 3.47
Q3_K_M | 3.85
Q3_K_L | 4.19
Q4_K_S | 4.53
Q4_K_M | 4.80
Q5_K_S | 5.50
Q5_K_M | 5.65
Q6_K | 6.56