tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-06-26 19:55:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

make rms_norm_eps a parameter (#2374)

Browse Source 
* make rms_norm_eps a parameter

* add rms_norm_eps to command line

* fix baby llama, test-grad0

* use scientific notation for eps param in the help

ggml-ci
This commit is contained in:
slaren
2023-07-24 17:57:12 +02:00
committed by GitHub
parent b3f138d058
commit 41c674161f
11 changed files with 89 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/test-grad0.c
View File

@ -850,7 +850,7 @@ int main(int argc, const char ** argv) {
ggml_set_param(ctx0, x[i]);
}
struct ggml_tensor * f = ggml_sum(ctx0, ggml_rms_norm(ctx0, x[0]));
struct ggml_tensor * f = ggml_sum(ctx0, ggml_rms_norm(ctx0, x[0], 1e-6f));
check_gradient("rms_norm", ctx0, x, f, ndims, nargs, 1e-4f, 1.0f, INFINITY);
}