tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-01 15:09:32 -04:00
Code Issues Packages Projects Releases Wiki Activity

gguf-py : Numpy (de)quantization for TQ1_0 and TQ2_0

Browse Source 
* ggml-quants : use roundf instead of nearest_int for TQ1_0 and TQ2_0

This does not change anything for ternary models,
since their values should never end up being in halfway cases anyway.
This commit is contained in:
Francis Couture-Harpin
2024-08-12 00:06:48 -04:00
parent d911cd1f13
commit 3a0bf17d57
3 changed files with 86 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
gguf-py/tests/test_quants.py
View File

@@ -66,6 +66,7 @@ class GGMLQuants:
for t in (
"q4_0", "q4_1", "q5_0", "q5_1", "q8_0",
"q2_K", "q3_K", "q4_K", "q5_K", "q6_K",
"tq1_0", "tq2_0",
"iq2_xxs", "iq2_xs", "iq2_s", "iq3_xxs", "iq3_s", "iq1_s", "iq1_m",
"iq4_nl", "iq4_xs",
):