tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-09-02 05:14:11 -04:00
Code Issues Packages Projects Releases Wiki Activity

metal : fix memory leak (#2762)

Browse Source 
* metal : fix memory leak

* metal : fix encoders memory leak

* metal : clean up more memory resources

* metal : fix more leaks

* metal : reuse dispatch queue + autoreleasepool

* metal : reuse array for command buffers and encoders

* ggml : assert for odd number of blocks on ARM

15M tinyllama is an example
This commit is contained in:
Georgi Gerganov
2023-08-28 10:59:08 +03:00
committed by GitHub
parent ebcee207b6
commit f55538c3cc
3 changed files with 88 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
ggml-metal.h
View File

@@ -24,6 +24,7 @@
// max memory buffers that can be mapped to the device
#define GGML_METAL_MAX_BUFFERS 16
#define GGML_METAL_MAX_COMMAND_BUFFERS 32
struct ggml_tensor;
struct ggml_cgraph;