tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-07-25 04:02:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

scripts: benchmark for HTTP server throughput (#14668)

Browse Source 
* scripts: benchmark for HTTP server throughput

* fix server connection reset
This commit is contained in:
Johannes Gäßler
2025-07-14 13:14:30 +02:00
committed by GitHub
parent 0f4c6ec0f1
commit 494c5899cb
4 changed files with 218 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tools/server/utils.hpp
View File

@ -11,6 +11,8 @@
// increase max payload length to allow use of larger context size
#define CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH 1048576
// increase backlog size to avoid connection resets for >> 1 slots
#define CPPHTTPLIB_LISTEN_BACKLOG 512
// disable Nagle's algorithm
#define CPPHTTPLIB_TCP_NODELAY true
#include <cpp-httplib/httplib.h>