tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-07-09 13:02:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Dequant improvements rebase (#8255)

Browse Source 
* Single load for half2

* Store scales in local mem

* Vec load quantized values
This commit is contained in:
AidanBeltonS
2024-07-03 02:55:34 +01:00
committed by GitHub
parent a27152b602
commit fadde67135
3 changed files with 30 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
ggml/src/ggml-sycl/common.hpp
View File

@ -351,4 +351,10 @@ static __dpct_inline__ float warp_reduce_max(float x,
return x;
}
// Helper for vec loading aligned data
template <typename Tp, int n>
inline sycl::vec<Tp, n> vec_aligned_load(const Tp* aligned_ptr) {
return *reinterpret_cast<const sycl::vec<Tp, n>*>(aligned_ptr);
}
#endif // GGML_SYCL_COMMON_HPP