tqcq/llama.cpp
0
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-08-20 06:36:48 -04:00
Code Issues Packages Projects Releases Wiki Activity

Add LLAMA_DEFAULT_RMS_EPS so we can change the default (#2384)

Browse Source 
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow
2023-07-25 18:35:53 +03:00
committed by GitHub
parent 07aaa0f63f
commit eb542d3932
5 changed files with 13 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/train-text-from-scratch/train-text-from-scratch.cpp
View File

@@ -16,7 +16,7 @@
#pragma warning(disable: 4244 4267) // possible loss of data
#endif
static const float rms_norm_eps = 1e-6f;
static const float rms_norm_eps = LLAMA_DEFAULT_RMS_EPS;
struct random_normal_distribution {
std::mt19937 gen;