Georgi Gerganov 83796e62bc llama : refactor unicode stuff (#5992) ...

* llama : refactor unicode stuff

ggml-ci

* unicode : names

* make : fix c++ compiler

* unicode : names

* unicode : straighten tables

* zig : fix build

* unicode : put nfd normalization behind API

ggml-ci

* swift : fix build

* unicode : add BOM

* unicode : add <cstdint>

ggml-ci

* unicode : pass as cpts as const ref

2024-03-11 17:47:47 +02:00

..

.gitignore

tests : gitignore ggml-common.h

2024-03-09 14:17:11 +02:00

CMakeLists.txt

llama : add llama_chat_apply_template() (#5538)

2024-02-19 10:23:37 +02:00

get-model.cpp

…

get-model.h

…

test-autorelease.cpp

ggml : add numa options (#5377)

2024-02-16 11:31:07 +02:00

test-backend-ops.cpp

ggml : remove old quantization functions (#5942)

2024-03-09 15:53:59 +02:00

test-c.c

…

test-chat-template.cpp

Add Gemma chat template (#5665)

2024-02-22 19:10:21 +01:00

test-double-float.cpp

…

test-grad0.cpp

…

test-grammar-parser.cpp

ggml, common, examples, tests : fixed type arguments in printf (#5528)

2024-02-18 18:20:12 +02:00

test-llama-grammar.cpp

ggml, common, examples, tests : fixed type arguments in printf (#5528)

2024-02-18 18:20:12 +02:00

test-model-load-cancel.cpp

ggml : add numa options (#5377)

2024-02-16 11:31:07 +02:00

test-opt.cpp

code : normalize enum names (#5697)

2024-02-25 12:09:09 +02:00

test-quantize-fns.cpp

Adding IQ2_S and IQ2_M to complete coverage of the 2-3 bit quantization range (#5721)

2024-02-26 18:28:38 +02:00

test-quantize-perf.cpp

…

test-rope.cpp

…

test-sampling.cpp

…

test-tokenizer-0-falcon.cpp

ggml : add numa options (#5377)

2024-02-16 11:31:07 +02:00

test-tokenizer-0-falcon.py

…

test-tokenizer-0-llama.cpp

ggml : add numa options (#5377)

2024-02-16 11:31:07 +02:00

test-tokenizer-0-llama.py

…

test-tokenizer-1-bpe.cpp

llama : refactor unicode stuff (#5992)

2024-03-11 17:47:47 +02:00

test-tokenizer-1-llama.cpp

llama : refactor unicode stuff (#5992)

2024-03-11 17:47:47 +02:00