Danny Lin
ad2fa2bf6f
Fix thread ID getter on Android ARM/AArch64
...
Android's Bionic libc stores the thread ID in TLS slot 1 instead of 0
on 32-bit ARM and AArch64. Slot 0 contains a pointer to the ELF DTV
(Dynamic Thread Vector) instead, which is constant for each loaded DSO.
Because mimalloc uses the thread ID to determine whether operations are
thread-local or cross-thread (atomic), all threads having the same ID
causes internal data structures to get corrupted quickly when multiple
threads are using the allocator:
mimalloc: assertion failed: at "external/mimalloc/src/page.c":563, mi_page_extend_free
assertion: "page->local_free == NULL"
mimalloc: assertion failed: at "external/mimalloc/src/page.c":74, mi_page_is_valid_init
assertion: "page->used <= page->capacity"
mimalloc: assertion failed: at "external/mimalloc/src/page.c":100, mi_page_is_valid_init
assertion: "page->used + free_count == page->capacity"
mimalloc: assertion failed: at "external/mimalloc/src/page.c":74, mi_page_is_valid_init
assertion: "page->used <= page->capacity"
Add support for Android's alternate TLS layout to fix the crashes in
multi-threaded use cases.
Fixes #376 .
2021-04-07 01:59:47 -07:00
Daan Leijen
331491e1e8
build fix for Apple M1 (issue #354 and pr #356 )
2021-02-02 10:46:30 -08:00
Daan Leijen
a7c33a3b0e
fix getting the unique thread id on the Apple M1, see issue #354 .
2021-02-01 15:47:22 -08:00
Daan Leijen
35c1fc2be9
limit memcpy as rep stosb to windows where the cpu supporst FSRM; add mi_memcpy_aligned for machine-word aligned copy. see issue #201 and pr #253
2021-01-30 14:33:46 -08:00
Daan Leijen
92ec493a5d
possible fix for aligment warning (issue #341 )
2021-01-29 16:21:50 -08:00
Daan Leijen
0a06884732
ensure memcpy with rep stosb is only used on windows
2021-01-29 16:09:09 -08:00
Daan
9b966c3492
Merge pull request #253 from haneefmubarak/memcpy-rep-movsb-windows-201
...
resolve #201 with a platform-selective REP MOVSB implementation
2021-01-29 16:00:00 -08:00
Daan Leijen
78ce716e2d
add comment on use of tpidrro_el0 on macOS
2021-01-28 17:36:56 -08:00
Uwe L. Korn
a753084f74
Use APPLE instead of MACH
2021-01-28 11:38:38 +01:00
Uwe L. Korn
88330cfc9f
Use __APPLE__ instead of __MACH__
2021-01-22 17:06:43 +01:00
Uwe L. Korn
ab3dac04c2
Use tpidrro_el0 for thread local storage in macOS-arm64
...
Fixes #343
2020-12-30 21:49:41 +01:00
Daan Leijen
bb386025b5
update override on macOS with interpose of malloc_default_zone (issues #313 )
2020-12-15 16:03:54 -08:00
Daan
5bbe1c0216
Merge pull request #323 from devnexen/dfbsd_build_fix
...
DragonFly support fix (for 5.8.x and forward).
2020-12-10 10:19:05 -08:00
Daan
8b8011b4f0
Merge pull request #322 from Kokokokoka/x32_patch
...
fix for x32 builds
2020-12-10 10:14:04 -08:00
David Carlier
e6c2fd44fc
DragonFly support fix (for 5.8.x and forward).
...
The pthread slot approach is somewhat buggy (pretty visible
with the stress unit test which segfault more or less randomly,
but the stats never show up).
Using the default approach instead, the test passes eventough
it s relatively slow (e.g 1.5 sec on FreeBSD vs 4.5 on DragonFly with same
machine).
2020-10-22 11:15:37 +01:00
Vasya B
cb45e3c6b1
fix for x32 builds
2020-10-19 21:00:16 +00:00
daan
14b8d27386
track pinned memory separately from large os pages
2020-09-08 16:46:03 -07:00
daan
c86459afef
split bitmap code into separate header and source file
2020-09-08 10:14:13 -07:00
daan
30b993ecf3
consolidate bit scan operations
2020-09-08 09:27:57 -07:00
daan
900c97664a
merge from dev-atomic
2020-09-03 09:47:01 -07:00
daan
5805c39916
enable --std=c99 compilation; fix mingw compilation
2020-08-09 17:55:17 -07:00
daan
ef8e5d18a6
replace atomics with C11/C++ atomics with explicit memory order; passes tsan. Issue #130
2020-07-26 18:01:33 -07:00
daan
afe29cb8f5
fix ub on shift, issue #279
2020-07-25 19:33:02 -07:00
daan
c5406f327e
move include 'limits.h' outside of definition
2020-07-21 18:51:25 -07:00
David Carlier
0c550d1626
illumos support/build fix and large page support
2020-07-10 03:26:14 +01:00
Haneef Mubarak
4c45793ec1
fix __movsb typecast error MSVC
2020-05-26 16:16:19 -07:00
Haneef Mubarak
6c92690914
fix REP MOVSB doc comment typo
2020-05-26 16:08:33 -07:00
Haneef Mubarak
429025634e
resolve #201 with a platform-selective REP MOVSB implementation
2020-05-26 16:04:28 -07:00
daan
a7d2bc8ad6
edit warning messages to be more consistent
2020-05-19 10:16:28 -07:00
daan
f2ac272baa
strengthen alignment check for memalign and aligned_alloc
2020-02-17 09:59:11 -08:00
daan
a96e94f940
change TLS slot on OpenBSD
2020-02-02 22:46:38 -08:00
daan
3560e0a867
fix TLS slot number on OSX
2020-02-02 22:15:09 -08:00
daan
f3c47c7c91
improved malloc zone handling on macOSX (not working yet)
2020-02-02 21:03:09 -08:00
daan
757dcc8411
extend interpose for macOSX
2020-02-02 19:07:26 -08:00
daan
12c4108abe
update comments
2020-02-02 16:09:09 -08:00
daan
07fbe4f80f
fixes for dragonfly
2020-02-02 14:31:28 -08:00
daan
8bc20631e4
fixes for freeBSD
2020-02-02 13:25:26 -08:00
daan
d2db9f1fc2
update thread local storage
2020-02-02 13:12:22 -08:00
daan
0989562c2d
add initial fast tls for macOSX
2020-02-01 16:57:00 -08:00
daan
fea903900d
use __thread locals on linux
2020-02-01 14:33:34 -08:00
daan
a169cf0e3f
merge dev-exp; add pthread TLS support for macOSX
2020-02-01 14:10:10 -08:00
daan
edff9d4fbb
merge from dev-win (padding)
2020-02-01 12:32:59 -08:00
daan
aa68b8cbc7
improve encoding of padding canary and buffer overflow detection
2020-02-01 12:16:37 -08:00
daan
40f1e1e07b
byte-precise heap block overflow checking with encoded padding
2020-01-31 23:39:51 -08:00
daan
68112a2751
better padding implementation, more precise statistics
2020-01-31 20:34:24 -08:00
daan
f820fae858
merge from dev; add retired collect on abandon
2020-01-30 07:24:35 -08:00
daan
a7c69ccbea
fix stat accounting of segments with huge blocks
2020-01-30 06:25:42 -08:00
daan
ed1c8a203a
improve performance with tls recursion counter
2020-01-29 23:08:52 -08:00
daan
03b363a1c2
first working tls on macOS using interpose; still slow
2020-01-29 22:46:44 -08:00
daan
e628fc7067
cleanup reclaim logic
2020-01-26 12:39:11 -08:00
daan
b31bc52618
add cache alignment directives for contended variables
2020-01-24 19:02:13 -08:00
daan
4a2a0c2d50
initial abandon based on fine-grained reclamation
2020-01-23 19:50:35 -08:00
daan
c9106e74a8
remove __thread attribute from mimalloc.h
2020-01-23 11:06:25 -08:00
daan
6fb434a99b
use -fvisibility=hidden on clang as well
2020-01-23 10:31:47 -08:00
daan
dc58388968
Add ability to register custom error function called on various error conditions; including ENOMEM
2020-01-17 19:59:55 -08:00
daan
6b7356a10a
make mimalloc compile with the highest warning level on msvc
2020-01-16 18:12:01 -08:00
daan
7a9502973d
rearrange STL allocator code: remove pragma, ifdef for C++11
2020-01-16 15:57:11 -08:00
daan
0099707af9
use delayed free for all pages; reduce size of the page structure for improved address calculation
2020-01-15 17:19:01 -08:00
daan
783e3377f7
add output argument to stat printing
2020-01-15 10:53:54 -08:00
daan
12ef2816ed
fix bug exposed by commit 59fa286 where reclaimed pages could be stuck to NEVER_DELAYED
2020-01-08 19:00:03 -08:00
daan
2b108c8748
increase retire expiration to 4
2020-01-03 21:39:18 -08:00
daan
77134e1ad0
update free list encoding to stronger formula with addition last
2019-12-28 15:17:49 -08:00
daan
e3391d9a53
stronger encoding of free lists using two keys per page
2019-12-28 00:57:42 -08:00
daan
ba87a39d9f
updated random cookie generation using OS primitives and chacha20
2019-12-22 19:37:49 -08:00
daan
a799a19136
fix non-standard line continuation
2019-11-25 14:30:12 -08:00
Daan
fe51d11e61
Merge pull request #178 from carenas/master
...
avoid deadlock with BSD systems that call malloc from the dynamic linker
2019-11-25 14:28:58 -08:00
Carlo Marcelo Arenas Belón
41ef691292
avoid deadlock with BSD systems that call malloc from the dynamic linker
...
extend the exception used for macOS to cover also OpenBSD (tested in 6.4+)
and DragonFlyBSD (tested in 5.6.2)
2019-11-24 10:00:39 -08:00
daan
6568059cc6
merge from dev
2019-11-21 16:28:28 -08:00
daan
74dbfc30be
improved security by encoding NULL values; double free mitigation on by default; more precise free list corruption detection
2019-11-21 15:21:23 -08:00
Daan Leijen
211f1aa519
remove reset delay slots; add reset tracking per page and segment
2019-11-20 14:55:12 -08:00
daan
17ade3fca5
merge from dev
2019-11-13 17:52:20 -08:00
daan
a4ed63d127
Adresses pr #165 and issue #164 by @colesbury:
...
On Mac OS, the thread-local _mi_default_heap may get reset before
_mi_thread_done is called, leaking the default heap on non-main threads.
Now the current default heap is also stored in mi_pthread_key (or mi_fls_key
on Windows). The _mi_thread_done function is called with this value.
2019-11-13 17:22:03 -08:00
Daan Leijen
d01ed42bcb
replace max_numa_nodes by use_numa_nodes (to help with wrong detection of numa nodes on WSL for example)
2019-11-13 13:35:50 -08:00
daan
ef179a6377
avoid allocation at numa node detection on linux
2019-11-12 10:16:59 -08:00
daan
181bef382c
merge dev-exp
2019-11-10 11:39:53 -08:00
daan
378716c467
refactor and improve atomic bitmap usage
2019-11-07 10:26:52 -08:00
daan
829fd872f4
initial delay slots
2019-11-04 11:48:41 -08:00
daan
8afd06b248
use int64 for time (instead of double)
2019-11-04 08:44:40 -08:00
daan
e320488791
add numa nodes to stats
2019-11-03 12:18:32 -08:00
daan
2c12d7f223
optimized numa calls; better Linux support
2019-11-01 22:01:52 -07:00
daan
a6499be074
initial numa support for arenas
2019-11-01 19:53:07 -07:00
daan
d36d04b4a6
add arena for huge page management
2019-10-31 15:35:10 -07:00
daan
b052d3b731
enable double free and heap corruption detection in debug mode
2019-10-28 15:54:33 -07:00
daan
25246070ae
fix double free check in secure = 4 mode; inline _mi_ptr_cookie
2019-10-19 08:34:18 -07:00
daan
2affdbbd2e
stronger secure mode when defining MI_SECURE=4: checks for double free, corrupted free list, and invalid pointer frees. Performance is impacted but not too much -- more perf testing is needed
2019-10-18 18:11:04 -07:00
daan
5de851a84d
update page_flags to have more portable definition
2019-10-17 16:48:16 -07:00
daan
f3a162f095
pick better umul_overflow variant based on size_t size
2019-10-17 09:52:10 -07:00
daan
4609537b8a
pick better umul_overflow variant based on intptr size
2019-10-17 09:47:12 -07:00
daan
24777f6a91
limit aligned allocation to power-of-two alignment
2019-09-11 17:49:28 -07:00
Daan Leijen
1909cfb346
refine mi_os_good_alloc_size and use it for huge pages (to ensure realloc is bounded as in #153
2019-09-10 13:26:51 -07:00
daan
ce81af1119
use mi_is_power_of_two when possible (pr #118 )
2019-09-09 08:12:50 -07:00
daan
7a91368108
pass page to block_zero_init and extend to block_size
2019-09-03 10:11:24 -07:00
daan
d1bd1644d5
support zero-initialized memory detection
2019-09-02 13:16:52 -07:00
daan
d6e35ffd83
Merge branch 'dev' into dev-exp
2019-09-02 10:10:08 -07:00
daan
8ef8ddecc5
Merge branch 'master' into dev
2019-09-02 09:53:34 -07:00
Carlo Marcelo Arenas Belón
0e35ec3879
avoid deadlock with OpenBSD
2019-09-01 01:06:01 -07:00
daan
f35f643508
merge from dev
2019-08-29 09:52:22 -07:00
daan
7bf12c7b5f
make output function configurable; remove <stdio.h> from standard includes
2019-08-29 09:42:50 -07:00
daan
9af51506a6
track commit and is_large status more precisely
2019-08-28 11:58:30 -07:00
daan
db8d443ae6
track more precisely if memory is fixed or committed
2019-08-26 22:45:26 -07:00
