Jeff Bolz 716bd6dec3 vulkan: optimize mul_mat for small values of N (#10991) ...

Make the mul_mat_vec shaders support N>1 (as a spec constant, NUM_COLS) where
the batch_strides are overloaded to hold the row strides. Put the loads from the
B matrix in the innermost loop because it should cache better.

Share some code for reducing the result values to memory in mul_mat_vec_base.

2024-12-30 18:27:11 +01:00

..

.gitignore

…

CMakeLists.txt

…

get-model.cpp

…

get-model.h

…

run-json-schema-to-grammar.mjs

…

test-arg-parser.cpp

…

test-autorelease.cpp

…

test-backend-ops.cpp

…

test-barrier.cpp

…

test-c.c

…

test-chat-template.cpp

…

test-double-float.cpp

…

test-gguf.cpp

…

test-grammar-integration.cpp

…

test-grammar-parser.cpp

…

test-json-schema-to-grammar.cpp

…

test-llama-grammar.cpp

…

test-log.cpp

…

test-lora-conversion-inference.sh

…

test-model-load-cancel.cpp

…

test-opt.cpp

…

test-quantize-fns.cpp

…

test-quantize-perf.cpp

…

test-rope.cpp

…

test-sampling.cpp

…

test-tokenizer-0.cpp

…

test-tokenizer-0.py

…

test-tokenizer-0.sh

…

test-tokenizer-1-bpe.cpp

…

test-tokenizer-1-spm.cpp

…

test-tokenizer-random.py

…