tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-01 06:59:13 -04:00
Code Issues Packages Projects Releases Wiki Activity
2,019 Commits 471 Branches 4,075 Tags
a4b07c057a553b1ac253051efc3f040351e2eae1
Commit Graph

2 Commits

Author SHA1 Message Date
Wu Jian Ping
c82d18e863 server : embeddings compatibility for OpenAI (#5190) 2024-01-29 15:48:10 +02:00
Xuan Son Nguyen
48c857aa10 server : refactored the task processing logic (#5065) 
* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
 2024-01-26 14:42:20 +02:00