tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-28 04:15:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

main : add --in-prefix-bos to prefix BOS to user inputs; keep EOS (#2304)

Browse Source 
* add `--in-prefix-bos` to prefix BOS to user inputs; keep EOS

The BOS precedes the string specified by `--in-prefix`.
Model generated EOS is now kept in the context.

It provides a way to strictly following the prompt format used in
Llama-2-chat.

The EOS handling also benefits some existing finetunes that uses
EOS to mark the end of turn.

* examples/common: move input_prefix_bos to other bools
This commit is contained in:
Xiao-Yong Jin
2023-07-25 07:19:11 -05:00
committed by GitHub
parent 1fed755b1f
commit 0c06204fb3
3 changed files with 34 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/common.h
View File

@ -82,6 +82,7 @@ struct gpt_params {
bool interactive_first = false; // wait for user input immediately
bool multiline_input = false; // reverse the usage of `\`
bool input_prefix_bos = false; // prefix BOS to user inputs, preceding input_prefix
bool instruct = false; // instruction mode (used for Alpaca models)
bool penalize_nl = true; // consider newlines as a repeatable token
bool perplexity = false; // compute perplexity over the prompt