Georgi Gerganov 12d0188c0d kv-cache : refactor + add llama_memory_state_i (#13746) ...

* kv-cache : simplify the "struct llama_kv_cache" interface

ggml-ci

* kv-cache : revert the (n_swa + n_ubatch) change (for next PR)

ggml-ci

* kv-cache : some comments

ggml-ci

* context : fix graph reserve for multiple sequences

ggml-ci

* kv-cache : fix typo [no ci]

* kv-cache : fix find_slot() logic for free slots

ggml-ci

* llama : add TODO for deprecating the defrag API in the future

* kv-cache : improve find_slot() using min/max seq pos info

ggml-ci

* llama : handle aborts and compute errors

ggml-ci

* memory : extract state into llama_memory_state

ggml-ci

* kv-cache : add comments

ggml-ci

* server : update batching logic to reset n_batch on successful decode

* server : upon full re-processing, remove the sequence from the cache

* kv-cache : add TODO for doing split_equal when split_simple fails

ggml-ci

2025-05-31 10:24:04 +03:00

..

batched

common : refactor downloading system, handle mmproj with -hf option (#12694)

2025-04-01 23:44:05 +02:00

batched.swift

llama : refactor llama_context, llama_kv_cache, llm_build_context (#12181)

2025-03-13 12:35:44 +02:00

convert-llama2c-to-ggml

llama : add llama_vocab, functions -> methods, naming (#11110)

2025-01-12 11:32:42 +02:00

deprecation-warning

Update deprecation-warning.cpp (#10619)

2024-12-04 23:19:20 +01:00

embedding

examples : allow extracting embeddings from decoder contexts (#13797)

2025-05-26 14:03:54 +03:00

eval-callback

llama : add llama_vocab, functions -> methods, naming (#11110)

2025-01-12 11:32:42 +02:00

gen-docs

ggml : move AMX to the CPU backend (#10570)

2024-11-29 21:54:58 +01:00

gguf

GGUF: C++ refactor, backend support, misc fixes (#11030)

2025-01-07 18:01:58 +01:00

gguf-hash

GGUF: C++ refactor, backend support, misc fixes (#11030)

2025-01-07 18:01:58 +01:00

gritlm

common : refactor downloading system, handle mmproj with -hf option (#12694)

2025-04-01 23:44:05 +02:00

jeopardy

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

llama.android

cmake : enable curl by default (#12761)

2025-04-07 13:35:19 +02:00

llama.swiftui

llama : refactor llama_context, llama_kv_cache, llm_build_context (#12181)

2025-03-13 12:35:44 +02:00

lookahead

llama : remove llama_kv_cache_view API + remove deprecated (#13653)

2025-05-20 16:13:16 +03:00

lookup

llama : remove llama_kv_cache_view API + remove deprecated (#13653)

2025-05-20 16:13:16 +03:00

parallel

kv-cache : refactor + add llama_memory_state_i (#13746)

2025-05-31 10:24:04 +03:00

passkey

common : refactor downloading system, handle mmproj with -hf option (#12694)

2025-04-01 23:44:05 +02:00

retrieval

examples : allow extracting embeddings from decoder contexts (#13797)

2025-05-26 14:03:54 +03:00

save-load-state

llama : refactor llama_context, llama_kv_cache, llm_build_context (#12181)

2025-03-13 12:35:44 +02:00

simple

fix: check model pointer validity before use (#13631)

2025-05-19 13:25:41 +03:00

simple-chat

kv-cache : simplify the interface (#13660)

2025-05-21 15:11:13 +03:00

simple-cmake-pkg

repo : update links to new url (#11886)

2025-02-15 16:40:57 +02:00

speculative

common : refactor downloading system, handle mmproj with -hf option (#12694)

2025-04-01 23:44:05 +02:00

speculative-simple

common : refactor downloading system, handle mmproj with -hf option (#12694)

2025-04-01 23:44:05 +02:00

sycl

sycl : backend documentation review (#13544)

2025-05-19 14:38:20 +01:00

training

examples/training: Fix file name in README (#13803)

2025-05-26 16:55:24 +02:00

chat-13B.bat

Create chat-13B.bat (#592)

2023-03-29 20:21:09 +03:00

chat-13B.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat-persistent.sh

scripts : fix pattern and get n_tokens in one go (#10221)

2024-11-09 09:06:54 +02:00

chat-vicuna.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

CMakeLists.txt

llama/ggml: add LLM training support (#10544)

2025-05-12 14:44:49 +02:00

convert_legacy_llama.py

metadata: Detailed Dataset Authorship Metadata (#8875)

2024-11-13 21:10:38 +11:00

json_schema_pydantic_example.py

py : type-check all Python scripts with Pyright (#8341)

2024-07-07 15:04:39 -04:00

json_schema_to_grammar.py

grammar : handle maxItems == 0 in JSON schema (#13117)

2025-04-26 10:10:20 +02:00

llama.vim

repo : update links to new url (#11886)

2025-02-15 16:40:57 +02:00

llm.vim

llm.vim : stop generation at multiple linebreaks, bind to <F2> (#2879)

2023-08-30 09:50:55 +03:00

Miku.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

pydantic_models_to_grammar_examples.py

llama : move end-user examples to tools directory (#13249)

2025-05-02 20:27:13 +02:00

pydantic_models_to_grammar.py

pydantic : replace uses of __annotations__ with get_type_hints (#8474)

2024-07-14 19:51:21 -04:00

reason-act.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

regex_to_grammar.py

py : switch to snake_case (#8305)

2024-07-05 07:53:33 +03:00

server_embd.py

llama : fix FA when KV cache is not used (i.e. embeddings) (#12825)

2025-04-08 19:54:51 +03:00

server-llama2-13B.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

ts-type-to-grammar.sh

JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555)

2024-04-12 19:43:38 +01:00