tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-18 05:56:00 -04:00
Code Issues Packages Projects Releases Wiki Activity

server: health: fix race condition on slots data using tasks queue (#5634)

Browse Source 
* server: health: fix race condition on slots data using tasks queue

* server: health:
    * include_slots only if slots_endpoint
    * fix compile warning task.target_id not initialized.
This commit is contained in:
Pierrick Hymbert
2024-02-21 15:47:48 +01:00
committed by GitHub
parent a00a35cef9
commit 1ecea255eb
3 changed files with 84 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/server/README.md
View File

@@ -140,6 +140,8 @@ node index.js
- 200 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if no slot are currently available.
- 503 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if the query parameter `fail_on_no_slot` is provided and no slot are currently available.
If the query parameter `include_slots` is passed, `slots` field will contain internal slots data except if `--slots-endpoint-disable` is set.
- **POST** `/completion`: Given a `prompt`, it returns the predicted completion.
*Options:*