tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-26 19:55:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

server : add some missing env variables (#9116)

Browse Source 
* server : add some missing env variables

* add LLAMA_ARG_HOST to server dockerfile

* also add LLAMA_ARG_CONT_BATCHING
This commit is contained in:
Xuan Son Nguyen
2024-08-27 11:07:01 +02:00
committed by GitHub
parent 2e59d61c1b
commit a77feb5d71
7 changed files with 60 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.devops/llama-server-cuda.Dockerfile
View File

@ -24,6 +24,8 @@ ENV CUDA_DOCKER_ARCH=${CUDA_DOCKER_ARCH}
ENV GGML_CUDA=1
# Enable cURL
ENV LLAMA_CURL=1
# Must be set to 0.0.0.0 so it can listen to requests from host machine
ENV LLAMA_ARG_HOST=0.0.0.0
RUN make -j$(nproc) llama-server