tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-04 16:23:49 -04:00
Code Issues Packages Projects Releases Wiki Activity

llama : add DRY sampler (#9702)

Browse Source 
* sampling : add DRY sampler (post-refactor)

* DRY: Trying to fix coauthors, removed unneeded line

* DRY: Fixed redundant code

* DRY: Fixed crash issue due to DRY being in chain but uninitialized

---------

Co-authored-by: l3utterfly <gc.pthzfoldr@gmail.com>
Co-authored-by: pi6am <34464159+pi6am@users.noreply.github.com>
This commit is contained in:
wwoodsTM
2024-10-25 10:07:34 -06:00
committed by GitHub
parent d80fb71f8b
commit ff252ea48e
17 changed files with 713 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
src/llama-vocab.h
View File

@@ -163,3 +163,8 @@ int32_t llama_detokenize_impl(
int32_t text_len_max,
bool remove_special,
bool unparse_special);
std::string llama_detokenize(
const struct llama_vocab & vocab,
const std::vector<llama_token> & tokens,
bool special);