tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-19 06:25:15 -04:00
Code Issues Packages Projects Releases Wiki Activity

convert-llama2c-to-ggml : enable conversion of GQA models (#6237)

Browse Source 
* convert-llama2c-to-ggml: enable conversion of multiqueries, #5608

* add test in build action

* Update build.yml

* Update build.yml

* Update build.yml

* gg patch
This commit is contained in:
fraxy-v
2024-03-22 20:49:06 +02:00
committed by GitHub
parent 1d0331c12a
commit 92397d87a4
3 changed files with 193 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/convert-llama2c-to-ggml/README.md
View File

@@ -21,6 +21,8 @@ An example command using a model from [karpathy/tinyllamas](https://huggingface.
`$ ./convert-llama2c-to-ggml --copy-vocab-from-model llama-2-7b-chat.gguf.q2_K.bin --llama2c-model stories42M.bin --llama2c-output-model stories42M.gguf.bin`
Note: The vocabulary for `stories260K.bin` should be its own tokenizer `tok512.bin` found in [karpathy/tinyllamas/stories260K](https://huggingface.co/karpathy/tinyllamas/tree/main/stories260K).
Now you can use the model with a command like:
`$ ./main -m stories42M.gguf.bin -p "One day, Lily met a Shoggoth" -n 500 -c 256`