tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-20 06:36:48 -04:00
Code Issues Packages Projects Releases Wiki Activity

server: metrics: add llamacpp:prompt_seconds_total and llamacpp:tokens_predicted_seconds_total, reset bucket only on /metrics. Fix values cast to int. Add Process-Start-Time-Unix header. (#5937)

Browse Source 
Closes #5850
This commit is contained in:
Pierrick Hymbert
2024-03-08 12:25:04 +01:00
committed by GitHub
parent e457fb3540
commit 76e868821a
3 changed files with 46 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/server/tests/features/server.feature
View File

@@ -29,6 +29,7 @@ Feature: llama.cpp server
And a completion request with no api error
Then <n_predicted> tokens are predicted matching <re_content>
And prometheus metrics are exposed
And metric llamacpp:tokens_predicted is <n_predicted>
Examples: Prompts
| prompt | n_predict | re_content | n_predicted |