782 Commits

Author	SHA1	Message	Date
daan	2e311f341b	fix msvc compilation in C mode	2020-09-05 09:37:09 -07:00
daan	7c2b79bef0	Merge branch 'dev' into dev-slice	2020-09-05 09:17:59 -07:00
daan	50de0d2358	fix C++ compilation with new atomics	2020-09-05 09:17:42 -07:00
daan	97f56b1e08	merge from dev	2020-09-04 14:21:33 -07:00
daan	f107acb3c8	fix __cplusplus test (pr #287)	2020-09-04 10:40:05 -07:00
daan	f6109765d8	update whitespace and comments	2020-09-03 15:04:40 -07:00
daan	03071dec0f	merge from dev-atomic with new atomic interface	2020-09-03 12:13:09 -07:00
daan	900c97664a	merge from dev-atomic	2020-09-03 09:47:01 -07:00
daan	76a68cd7af	bump version to 1.6.6 with new atomics	2020-09-03 09:45:53 -07:00
daan	b4825372ab	small compilation warning fixes (extra semicolon etc)	2020-08-29 19:30:38 -07:00
daan	8033b62979	allow overriding MI_MAX_ALIGN_SIZE	2020-08-29 09:59:15 -07:00
daan	e4ddc75069	set delayed decommit mask more precisely to only decommit currently committed blocks	2020-08-28 08:46:51 -07:00
daan	2cffc3b851	merge from dev	2020-08-27 22:43:57 -07:00
daan	5805c39916	enable --std=c99 compilation; fix mingw compilation	2020-08-09 17:55:17 -07:00
daan	ac0c121c68	update documentation for 1.6.4 release	2020-07-27 08:55:26 -07:00
daan	ef8e5d18a6	replace atomics with C11/C++ atomics with explicit memory order; passes tsan. Issue #130	2020-07-26 18:01:33 -07:00
daan	a468430772	strengthen memory order of bit operations; insert memory fences	2020-07-26 14:19:30 -07:00
daan	a9f46dc86f	reduce memory order constraints for better efficiency on ARM etc	2020-07-26 11:58:02 -07:00
daan	ebf951e851	extra checks for atomic ptr exchange; extend mi_atomic_yield for win32	2020-07-26 00:15:57 -07:00
daan	95afd0509f	make segment abandoned_next atomic; tsan passes without warnings now (issue #130)	2020-07-25 23:50:22 -07:00
daan	09ade02429	bring inline with C11 atomics; no volatile and cas order of expected/desired	2020-07-25 22:52:27 -07:00
daan	e27422adca	switch to using C++ atomics in MSVC as well	2020-07-25 20:55:45 -07:00
daan	afe29cb8f5	fix ub on shift, issue #279	2020-07-25 19:33:02 -07:00
daan	a9a21f39d8	fix memory order for CAS failure, issue #130	2020-07-23 15:52:02 -07:00
daan	b74caddcc1	fix memory order to acq_rel for atomic and/or, issue #130, thanks @mpoeter!	2020-07-23 15:00:49 -07:00
daan	8aa18d3661	fix TSAN warning for statistics maximum, issue #130	2020-07-22 14:16:18 -07:00
daan	444afa934f	fix memory order for weak CAS, issue #130, thanks @mary3000!	2020-07-22 13:58:00 -07:00
daan	2e1b4f512d	make max update in the stats atomic	2020-07-22 13:45:04 -07:00
daan	c5406f327e	move include 'limits.h' outside of definition	2020-07-21 18:51:25 -07:00
David Carlier	0c550d1626	illumos support/build fix and large page support	2020-07-10 03:26:14 +01:00
Haneef Mubarak	4c45793ec1	fix __movsb typecast error MSVC	2020-05-26 16:16:19 -07:00
Haneef Mubarak	6c92690914	fix REP MOVSB doc comment typo	2020-05-26 16:08:33 -07:00
Haneef Mubarak	429025634e	resolve #201 with a platform-selective REP MOVSB implementation	2020-05-26 16:04:28 -07:00
daan	cb05ef9f2c	merge from dev	2020-05-19 10:43:46 -07:00
daan	a7d2bc8ad6	edit warning messages to be more consistent	2020-05-19 10:16:28 -07:00
daan	53aa46890a	merge from dev	2020-05-05 10:54:59 -07:00
daan	cefc930f72	bump version to 1.6.4 for further development	2020-05-05 10:47:46 -07:00
daan	e5b72cdfe7	reduce segment size and increase cache	2020-05-02 22:22:35 -07:00
daan	69158f2c76	roll back again to new arena cache: previous perf regression was caused due to accidentally making secure mode default	2020-05-02 12:04:36 -07:00
daan	b8846f7a27	fix unprotect of guard pages	2020-05-02 10:51:10 -07:00
daan	dcb3574cf0	fix assertions for huge segment free	2020-05-01 21:14:41 -07:00
daan	5cfdc39ff1	remove on-demand page commit option	2020-04-30 18:23:33 -07:00
daan	dad3be3c64	update comments	2020-04-30 17:21:36 -07:00
daan	0d25493c39	segment size to 16MiB to improve perf on mstress and rptest	2020-04-28 16:50:03 -07:00
daan	1b0de9b4cf	merge from dev	2020-04-28 16:22:38 -07:00
Anton Korobeynikov	079b886feb	Add cmake option to specify whether warnings / errors are enabled by default. ... Currently warnings / errors are enabled by default in debug build. Otherwise they could be enabled only via environmental variable or API option call. Add possibility to specify the default during the build time. This simplifies e.g. integration of the library into bigger projects as no source changes would be required.	2020-04-28 16:44:44 +03:00
daan	3484cda169	bump version to 1.6.3 for further development	2020-04-20 18:01:39 -07:00
daan	d102882bdf	bump version to 1.6.2	2020-04-20 17:59:13 -07:00
daan	69a0846478	add MI_PADDING flag to cmake to supress use of padding in debug mode	2020-04-07 10:01:18 -07:00
daan	0a3e5bb699	further fixes to compile on mingw without warnings (issue #217)	2020-04-06 10:56:53 -07:00
daan	50bfacdb71	fix mingw compilation by using export definitions (issue #217)	2020-04-06 09:53:29 -07:00
daan	cfeee53389	remove accidental include file	2020-03-16 16:44:29 -07:00
daan	1f396e64a0	merge from dev	2020-03-16 16:41:21 -07:00
daan	7745dde8d2	allow retirement for all object sizes (issue #212)	2020-03-16 15:31:37 -07:00
daan	82684042be	bump version to 1.6.1	2020-02-17 10:10:22 -08:00
daan	f2ac272baa	strengthen alignment check for memalign and aligned_alloc	2020-02-17 09:59:11 -08:00
daan	67de2549cf	fix build with clang-cl due to _Check_return_ (issue #200)	2020-02-14 09:40:56 -08:00
daan	8a2a52843d	delete all thread owned heaps when a thread is terminated (issue #202)	2020-02-13 12:15:23 -08:00
daan	f42b8526d0	fix wrong __declspec(restrict) and __attribute__((malloc)) attributes on reallocation functions	2020-02-13 10:36:39 -08:00
daan	04f1c3b1e2	bump version to v1.6.0	2020-02-09 18:53:39 -08:00
daan	a96e94f940	change TLS slot on OpenBSD	2020-02-02 22:46:38 -08:00
daan	3560e0a867	fix TLS slot number on OSX	2020-02-02 22:15:09 -08:00
daan	1c2e0a47ca	fix noexcept attribute on array delete operators	2020-02-02 22:04:53 -08:00
daan	f3c47c7c91	improved malloc zone handling on macOSX (not working yet)	2020-02-02 21:03:09 -08:00
daan	757dcc8411	extend interpose for macOSX	2020-02-02 19:07:26 -08:00
daan	12c4108abe	update comments	2020-02-02 16:09:09 -08:00
daan	07fbe4f80f	fixes for dragonfly	2020-02-02 14:31:28 -08:00
daan	8bc20631e4	fixes for freeBSD	2020-02-02 13:25:26 -08:00
daan	d2db9f1fc2	update thread local storage	2020-02-02 13:12:22 -08:00
daan	bb64634e4e	Merge branch 'dev-exp' into dev-exp-tls	2020-02-01 17:48:57 -08:00
daan	bf2eb55ed1	reformatting	2020-02-01 17:48:26 -08:00
daan	8aba40a972	use default declaration for the STL allocator class	2020-02-01 17:41:04 -08:00
daan	3f17ac287c	add 'nodiscard' attribute to mimalloc interface	2020-02-01 17:29:30 -08:00
daan	0989562c2d	add initial fast tls for macOSX	2020-02-01 16:57:00 -08:00
daan	fea903900d	use __thread locals on linux	2020-02-01 14:33:34 -08:00
daan	a169cf0e3f	merge dev-exp; add pthread TLS support for macOSX	2020-02-01 14:10:10 -08:00
daan	edff9d4fbb	merge from dev-win (padding)	2020-02-01 12:32:59 -08:00
daan	aa68b8cbc7	improve encoding of padding canary and buffer overflow detection	2020-02-01 12:16:37 -08:00
daan	40f1e1e07b	byte-precise heap block overflow checking with encoded padding	2020-01-31 23:39:51 -08:00
daan	68112a2751	better padding implementation, more precise statistics	2020-01-31 20:34:24 -08:00
daan	4531367de2	fix padding check for aligned allocation; improve perf for small aligned allocations	2020-01-31 13:20:02 -08:00
daan	f820fae858	merge from dev; add retired collect on abandon	2020-01-30 07:24:35 -08:00
daan	a7c69ccbea	fix stat accounting of segments with huge blocks	2020-01-30 06:25:42 -08:00
daan	ed1c8a203a	improve performance with tls recursion counter	2020-01-29 23:08:52 -08:00
daan	03b363a1c2	first working tls on macOS using interpose; still slow	2020-01-29 22:46:44 -08:00
daan	5d212d688f	add MI_PADDING build option to add padding to each block to detect heap block overflows	2020-01-29 17:10:57 -08:00
daan	b50bec463d	merge from dev-exp; better abandoned reclamation	2020-01-27 22:12:23 -08:00
daan	e628fc7067	cleanup reclaim logic	2020-01-26 12:39:11 -08:00
daan	5e32d00aab	add visit count to abandoned to limit list length	2020-01-25 12:26:08 -08:00
daan	b31bc52618	add cache alignment directives for contended variables	2020-01-24 19:02:13 -08:00
daan	4a2a0c2d50	initial abandon based on fine-grained reclamation	2020-01-23 19:50:35 -08:00
daan	c9106e74a8	remove __thread attribute from mimalloc.h	2020-01-23 11:06:25 -08:00
daan	6fb434a99b	use -fvisibility=hidden on clang as well	2020-01-23 10:31:47 -08:00
daan	a46d20a681	merge with new atomic macros	2020-01-22 20:53:44 -08:00
daan	e226ebcc97	Merge branch 'dev' into dev-arena	2020-01-22 20:39:33 -08:00
daan	e3b16fe4ef	reduce type casts in atomic operations	2020-01-22 19:44:54 -08:00
daan	0f14f431c5	bump version to 1.5 for further development	2020-01-22 18:33:07 -08:00
Daan Leijen	caa5e51a67	align size of page_t, increase slices per segment	2020-01-22 11:29:32 -08:00
daan	0028272cf4	small fixes, reduced segment size, fix merge conflicts	2020-01-20 22:33:29 -08:00
daan	394a7a92ab	merge from dev	2020-01-20 19:06:08 -08:00
daan	3957b2fd28	add mi_new_realloc(n) to support C++ style reallocation that raises std::bad_alloc on out-of-memory	2020-01-20 15:41:56 -08:00
daan	146899af8a	add missing members to stl allocator (#193)	2020-01-20 15:27:05 -08:00
daan	a33ebb8625	add alloc_align attribute to aligned allocation functions	2020-01-19 22:14:35 -08:00
daan	514b315283	add max_size member to STL allocator	2020-01-19 21:27:46 -08:00
daan	9d7ac76d93	fix compilation under Intel C compiler (icc)	2020-01-19 18:35:45 -08:00
daan	dc58388968	Add ability to register custom error function called on various error conditions; including ENOMEM	2020-01-17 19:59:55 -08:00
daan	3e982a3813	fix STL deallocate passing count (instead of size) to mi_free_size	2020-01-17 15:58:43 -08:00
daan	6dd636d82d	improve STL allocator using mi_new_n and removing unused parameter names; follow up from pr #193 and #188	2020-01-17 15:41:52 -08:00
daan	6b7356a10a	make mimalloc compile with the highest warning level on msvc	2020-01-16 18:12:01 -08:00
daan	a0bee081dd	use proper C++11 check for noexcept attribute	2020-01-16 15:58:48 -08:00
daan	7a9502973d	rearrange STL allocator code: remove pragma, ifdef for C++11	2020-01-16 15:57:11 -08:00
Daan	97bd204c42	Merge branch 'dev' into kile/stl	2020-01-16 15:33:13 -08:00
Kirsten Lee	526bee6843	merge stl and main header	2020-01-16 15:17:15 -08:00
Kirsten Lee	1b2b7404f7	flip the order of includes for ease of use	2020-01-16 14:54:13 -08:00
daan	24f8bcbc8f	add explicit calling convention to registered functions	2020-01-16 14:25:09 -08:00
daan	8d8f355ed0	add option to reset eagerly when a segment is abandoned	2020-01-16 11:25:02 -08:00
daan	7a1e86fc20	merge from dev	2020-01-15 18:03:37 -08:00
daan	0099707af9	use delayed free for all pages; reduce size of the page structure for improved address calculation	2020-01-15 17:19:01 -08:00
daan	202246425b	bump version to 1.4 for further development	2020-01-15 12:16:53 -08:00
daan	c9b5ac80b3	update page reset queue to just do delayed page resets	2020-01-15 12:00:44 -08:00
daan	783e3377f7	add output argument to stat printing	2020-01-15 10:53:54 -08:00
daan	0956a05bf6	add argument pointer to the register deferred free callback	2020-01-15 10:21:32 -08:00
daan	f92a2a7264	add argument pointer to the register output routine	2020-01-15 10:18:32 -08:00
daan	941c55ee42	wip: first implementation of page free list on segments for effecient delayed page reset	2020-01-14 21:47:18 -08:00
daan	88b141cf1f	ensure proper padding for the page structure	2020-01-13 20:48:37 -08:00
daan	4a27ea1643	merge from dev	2020-01-13 18:01:34 -08:00
daan	12ef2816ed	fix bug exposed by commit 59fa286 where reclaimed pages could be stuck to NEVER_DELAYED	2020-01-08 19:00:03 -08:00
daan	683d8998d4	fix potential A-B-A problem with segment abandonment; noticed by Manual Poeter and Sam Gross	2020-01-08 17:45:38 -08:00
Kirsten Lee	743e891738	add stl mimalloc wrapper	2020-01-06 16:18:22 -08:00
daan	2b108c8748	increase retire expiration to 4	2020-01-03 21:39:18 -08:00
daan	eeb623e6af	increase retire limit, collect retired pages	2020-01-03 17:06:41 -08:00
daan	f9ca88f71c	set secure default to 0 again	2020-01-02 17:57:41 -08:00
daan	fc3e537bd4	improve double free detection with faster same page check	2019-12-28 15:28:13 -08:00
daan	77134e1ad0	update free list encoding to stronger formula with addition last	2019-12-28 15:17:49 -08:00
daan	e3391d9a53	stronger encoding of free lists using two keys per page	2019-12-28 00:57:42 -08:00
daan	ba87a39d9f	updated random cookie generation using OS primitives and chacha20	2019-12-22 19:37:49 -08:00
daan	a799a19136	fix non-standard line continuation	2019-11-25 14:30:12 -08:00
Daan	fe51d11e61	Merge pull request #178 from carenas/master ... avoid deadlock with BSD systems that call malloc from the dynamic linker	2019-11-25 14:28:58 -08:00
daan	b5fbdb7180	merge from dev	2019-11-25 11:16:39 -08:00
daan	1d998af854	clean up options; make secure work with eager_page_commit	2019-11-25 10:47:17 -08:00
daan	c6df7a199c	experimental eager page commit option	2019-11-24 22:02:40 -08:00
daan	41af533a34	define commit unit in terms of segment size	2019-11-24 19:17:56 -08:00
daan	ec0005b919	more fine grained commit tracking per MiB	2019-11-24 19:09:15 -08:00
Carlo Marcelo Arenas Belón	41ef691292	avoid deadlock with BSD systems that call malloc from the dynamic linker ... extend the exception used for macOS to cover also OpenBSD (tested in 6.4+) and DragonFlyBSD (tested in 5.6.2)	2019-11-24 10:00:39 -08:00
daan	f45ec667a3	Merge branch 'dev' into dev-arena	2019-11-22 09:29:00 -08:00
daan	24b768363e	bump version to 1.3 for further development	2019-11-22 09:28:48 -08:00
daan	7da00c1220	wip: full decommit delay, for arena cache as well	2019-11-21 20:57:32 -08:00
daan	321e18777e	wip: delayed decommit on segments	2019-11-21 19:53:43 -08:00
daan	1066be1594	merge from dev-exp	2019-11-21 17:03:30 -08:00
daan	50575b12c0	Merge branch 'dev' into dev-exp	2019-11-21 16:29:59 -08:00
daan	41caf6d0f8	set secure default to 0	2019-11-21 16:29:46 -08:00
daan	6568059cc6	merge from dev	2019-11-21 16:28:28 -08:00
daan	d55ab50a84	update version to 1.2	2019-11-21 16:10:04 -08:00
daan	74dbfc30be	improved security by encoding NULL values; double free mitigation on by default; more precise free list corruption detection	2019-11-21 15:21:23 -08:00
Daan Leijen	211f1aa519	remove reset delay slots; add reset tracking per page and segment	2019-11-20 14:55:12 -08:00
Daan Leijen	30e2c54adb	remove delayed reset option (for now)	2019-11-20 14:13:02 -08:00
daan	17ade3fca5	merge from dev	2019-11-13 17:52:20 -08:00
daan	314dcd59c2	Merge branch 'dev' of https://github.com/microsoft/mimalloc into dev	2019-11-13 17:23:54 -08:00
daan	a4ed63d127	Adresses pr #165 and issue #164 by @colesbury: ... On Mac OS, the thread-local _mi_default_heap may get reset before _mi_thread_done is called, leaking the default heap on non-main threads. Now the current default heap is also stored in mi_pthread_key (or mi_fls_key on Windows). The _mi_thread_done function is called with this value.	2019-11-13 17:22:03 -08:00
Daan	9b7a29c5da	Merge pull request #172 from aesophor/dev ... Avoid the use of variable argument list function	2019-11-13 16:28:09 -08:00
Daan Leijen	d01ed42bcb	replace max_numa_nodes by use_numa_nodes (to help with wrong detection of numa nodes on WSL for example)	2019-11-13 13:35:50 -08:00
Marco Wang	29919a938d	Avoid the use of variable argument list function	2019-11-13 13:19:21 +08:00
Daan Leijen	bdb8274819	change max_numa_node to max_numa_nodes option	2019-11-12 12:04:43 -08:00
daan	ef179a6377	avoid allocation at numa node detection on linux	2019-11-12 10:16:59 -08:00
daan	a0958b2da6	enable more reset delay slots	2019-11-11 17:06:16 -08:00
daan	741a80256d	merge from dev-exp	2019-11-11 14:51:16 -08:00
daan	2bb058bd25	remove cache_reset parameter	2019-11-11 14:44:32 -08:00
daan	83a066fd2d	remove reset_decommits option	2019-11-11 09:46:02 -08:00
daan	181bef382c	merge dev-exp	2019-11-10 11:39:53 -08:00
Daan Leijen	b04206a9d3	add os cache to arena	2019-11-10 10:10:10 -08:00
Daan Leijen	fed0068dac	merge from dev-exp; bitmap based arena	2019-11-10 07:56:40 -08:00
daan	27f1a8b3d2	fix avg display; set secure default to 0`	2019-11-07 10:35:30 -08:00
daan	36090dee5c	Merge branch 'dev' into dev-exp	2019-11-07 10:34:06 -08:00
daan	31d11f64d5	fix secure free list extension where a non-empty initial free list was discarded	2019-11-07 10:33:45 -08:00
daan	378716c467	refactor and improve atomic bitmap usage	2019-11-07 10:26:52 -08:00
daan	b09282bc0d	change arena allocator to atomic bitmap as well	2019-11-06 22:49:01 -08:00
daan	00e19cad9a	refactor region code, split out atomic bitmap	2019-11-06 21:37:23 -08:00
daan	829fd872f4	initial delay slots	2019-11-04 11:48:41 -08:00
daan	62df2e2df9	merge from dev-exp	2019-11-04 08:56:42 -08:00
daan	c38af8f7c1	merge d1d65fbc: make max error messages configurable	2019-11-04 08:49:59 -08:00
daan	8afd06b248	use int64 for time (instead of double)	2019-11-04 08:44:40 -08:00
daan	2b005addd3	merge from dev-exp	2019-11-03 13:37:03 -08:00
daan	d1d65fbca4	make max error messages configurable	2019-11-03 13:25:41 -08:00
daan	1a6d150687	merge from dev-exp	2019-11-03 12:21:22 -08:00
daan	e320488791	add numa nodes to stats	2019-11-03 12:18:32 -08:00
daan	e0b8ec7f54	merge with dev-exp	2019-11-02 11:56:19 -07:00
daan	fd9d8c85ae	change numa support on linux to use getcpu	2019-11-02 11:55:03 -07:00
daan	08c4726043	merge from dev-exp	2019-11-01 22:04:20 -07:00
daan	2c12d7f223	optimized numa calls; better Linux support	2019-11-01 22:01:52 -07:00
daan	6b26f0cb17	merge from dev-exp (numa support)	2019-11-01 20:08:56 -07:00
daan	a6499be074	initial numa support for arenas	2019-11-01 19:53:07 -07:00
daan	62984c0a24	merge from dev-exp	2019-10-31 18:44:48 -07:00
daan	d36d04b4a6	add arena for huge page management	2019-10-31 15:35:10 -07:00
daan	6695f8ae91	add allow_decommit option	2019-10-31 10:59:50 -07:00
daan	f7d2c45af3	initial experiment with fixed memory arena and sliced segments	2019-10-31 00:40:41 -07:00
daan	c7ec30ae25	fix secure mode	2019-10-30 15:36:13 -07:00
daan	b73beede34	merge from dev	2019-10-30 15:19:34 -07:00
daan	b052d3b731	enable double free and heap corruption detection in debug mode	2019-10-28 15:54:33 -07:00
daan	081e2d1eb6	fix statistics display	2019-10-28 13:43:42 -07:00
daan	9d4f57abf3	merge from dev-win	2019-10-28 12:33:01 -07:00
daan	25246070ae	fix double free check in secure = 4 mode; inline _mi_ptr_cookie	2019-10-19 08:34:18 -07:00
daan	2affdbbd2e	stronger secure mode when defining MI_SECURE=4: checks for double free, corrupted free list, and invalid pointer frees. Performance is impacted but not too much -- more perf testing is needed	2019-10-18 18:11:04 -07:00
daan	4b15e2ed97	merge from dev	2019-10-17 18:24:35 -07:00
daan	fdfa6ed260	fix warnings at high warning level in msvc	2019-10-17 16:56:57 -07:00
daan	5de851a84d	update page_flags to have more portable definition	2019-10-17 16:48:16 -07:00
daan	26c27fbf58	use uint8_t bit fields, and improve portability of page_flags type	2019-10-17 12:07:26 -07:00
Johannes Schindelin	559688ec64	Suppress warning about unnamed struct ... This prevents MSVC complaining with warning C4201: nonstandard extension used: nameless struct/union The struct might seem unnecessary to the occasional reader (it did seem so to this commit's author), but it is not! It is required to align the fields to a boundary, which is verified by the test suite. Removing that "unnecessary" `struct` results in this failure: 1: Test command: mimalloc-test-api [...] 1: test: malloc-zero... mimalloc: assertion failed: at src/page.c:591, mi_page_init 1: assertion: "!mi_page_has_aligned(page)" Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>	2019-10-17 20:43:18 +02:00
Johannes Schindelin	e747a6f3a6	Use unsigned for bit-field variables ... It is actually non-standard to use `bool` with a bit-field quantifier, and VS 2019 complains about this. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>	2019-10-17 20:43:18 +02:00
daan	f3a162f095	pick better umul_overflow variant based on size_t size	2019-10-17 09:52:10 -07:00
daan	4609537b8a	pick better umul_overflow variant based on intptr size	2019-10-17 09:47:12 -07:00
Daan Leijen	114d05cd76	bump to version 1.1	2019-10-07 09:54:23 -07:00
daan	60efb62155	make secure a build option only	2019-09-17 17:31:42 -07:00
daan	33f4ec4ac6	put new-delete overrides in separate header file	2019-09-15 19:19:00 -07:00
daan	6896408ab1	fix definition of mi_recalloc_tp	2019-09-11 20:49:18 -07:00
daan	24777f6a91	limit aligned allocation to power-of-two alignment	2019-09-11 17:49:28 -07:00
Daan Leijen	1909cfb346	refine mi_os_good_alloc_size and use it for huge pages (to ensure realloc is bounded as in #153	2019-09-10 13:26:51 -07:00
daan	ce81af1119	use mi_is_power_of_two when possible (pr #118)	2019-09-09 08:12:50 -07:00
daan	23155c5d71	add recursion guard to deferred callback	2019-09-09 08:02:41 -07:00
daan	3bbc047ba6	reserve huge pages returns actual number of pages reserved	2019-09-04 12:26:02 -07:00
daan	6c43ae2bde	add later thread-done invokation; add mi_is_redirected call	2019-09-04 12:00:41 -07:00
daan	a6a956c589	add back zero initialized re-allocation	2019-09-03 10:57:39 -07:00
daan	7a91368108	pass page to block_zero_init and extend to block_size	2019-09-03 10:11:24 -07:00
daan	d1bd1644d5	support zero-initialized memory detection	2019-09-02 13:16:52 -07:00
daan	d6e35ffd83	Merge branch 'dev' into dev-exp	2019-09-02 10:10:08 -07:00
daan	8ef8ddecc5	Merge branch 'master' into dev	2019-09-02 09:53:34 -07:00
Carlo Marcelo Arenas Belón	0e35ec3879	avoid deadlock with OpenBSD	2019-09-01 01:06:01 -07:00
daan	f35f643508	merge from dev	2019-08-29 09:52:22 -07:00
daan	7bf12c7b5f	make output function configurable; remove <stdio.h> from standard includes	2019-08-29 09:42:50 -07:00
daan	4819d3f78f	expose mi_stats_merge function	2019-08-29 09:01:00 -07:00
daan	d381fcd9fa	rename lazy to eager_commit_delay	2019-08-28 12:09:23 -07:00
daan	9af51506a6	track commit and is_large status more precisely	2019-08-28 11:58:30 -07:00
daan	23b5593af6	Merge branch 'dev' of https://github.com/microsoft/mimalloc into dev	2019-08-27 16:40:47 -07:00
David Carlier	b72a2d9659	macOS: anonymous page ID make it as env var.	2019-08-27 19:17:41 +01:00
daan	db8d443ae6	track more precisely if memory is fixed or committed	2019-08-26 22:45:26 -07:00
daan	25dca38ef9	merge from dev-win	2019-08-26 12:47:16 -07:00
daan	f0a12699c2	remove atomic_iread	2019-08-26 12:42:25 -07:00
daan	a654732d99	merge from dev new atomics	2019-08-26 12:30:03 -07:00
daan	2c19388bcf	initialize mimalloc options at process load	2019-08-26 11:44:41 -07:00
daan	7ce9c02fd4	make cas weak use release memory order; improve free assembly	2019-08-26 08:20:26 -07:00
daan	5c7c106d62	strengthen some atomic operations for weak memory models	2019-08-26 08:11:58 -07:00
daan	e8664001f7	Use standard _Atomic declarations and clean up atomic operations	2019-08-25 22:59:12 -07:00
daan	b0e38d5697	merge from dev-win	2019-08-25 13:12:57 -07:00
daan	23c35f4aba	merge from dev	2019-08-25 13:07:07 -07:00
daan	b86c851cca	put segment cache behind an option and disable by default	2019-08-25 12:35:14 -07:00
daan	a431d80fc3	better object sizes for large objects	2019-08-25 10:20:03 -07:00
daan	6f5492cef8	enable initial lazy commit and optional decommit to reduce commit charge with many threads	2019-08-24 15:00:55 -07:00
daan	612b2cc9b7	clean up segment slice handling	2019-08-24 12:20:32 -07:00
daan	cce38bc147	more conservative setting to avoid internal fragmentation	2019-08-24 07:32:23 -07:00
daan	082f012a91	merge from dev-win	2019-08-23 21:56:28 -07:00
daan	d3224d0bba	Merge branch 'dev' into dev-win	2019-08-23 21:42:32 -07:00
daan	6c6fcad242	remove threadid from pages and keep page flags separate (cherry picked)	2019-08-23 21:42:24 -07:00
daan	15552eba79	ensure volatile declaration for abandoned_next field	2019-08-23 21:39:27 -07:00
daan	acde83543f	remove threadid from pages and keep page flags separate	2019-08-23 14:08:00 -07:00
daan	b511309709	ensure volatile declaration for abandoned_next field	2019-08-23 13:44:43 -07:00
daan	5c912f16d4	merge from remote	2019-08-21 11:35:09 -07:00
daan	a3c4b1c95b	merge from dev-win	2019-08-21 11:18:05 -07:00
daan	71121a7457	Merge branch 'dev-win' into dev-exp	2019-08-21 11:12:25 -07:00
daan	e7bd8c5d14	merge with dev-win	2019-08-21 11:11:40 -07:00
Daan Leijen	eb25093b13	fix mi_cdecl for older clang versions	2019-08-21 09:40:57 -07:00
Daan Leijen	12257e5bc1	fix thread-id reset on page abandonment	2019-08-21 08:09:28 -07:00
Daan Leijen	cc6d14c55b	fix thread-id reset on page abandonment	2019-08-21 08:06:40 -07:00
daan	cd52d0a6d9	merge dev-win	2019-08-20 17:31:46 -07:00
daan	86d4ded831	Merge branch 'dev' into dev-exp	2019-08-20 08:27:46 -07:00
daan	a5071eb510	Merge branch 'dev' into dev-win	2019-08-20 07:50:03 -07:00
daan	03d994648a	better option interface: option_enable, option_disable, and option_set_enabled	2019-08-20 07:47:50 -07:00
daan	7a2e0df5ea	merge from dev	2019-08-20 07:37:46 -07:00
daan	d52e4039b6	remove the reset_discards option	2019-08-20 07:06:11 -07:00
daan	79487dbedf	add checks for right option order	2019-08-20 06:58:51 -07:00
daan	05631ebfc4	Merge branch 'dev' into dev-win	2019-08-19 21:23:04 -07:00
daan	13ef8a049b	allow larger large objects to better use segment space	2019-08-19 21:22:09 -07:00
daan	770e752578	fix parameter order on mul_overflow to fix static analysis warnings (pr #125)	2019-08-19 19:15:04 -07:00
daan	228c5e5507	allow larger large objects to better use segment space	2019-08-19 18:20:51 -07:00
daan	0e639addb0	add environment option to reserve huge pages upfront	2019-08-19 14:14:50 -07:00
daan	5b2f4184b3	Merge branch 'dev' into dev-exp	2019-08-19 11:37:39 -07:00
daan	42dedb00ec	initial implementation of mi_os_reserve_huge_pages	2019-08-19 11:10:06 -07:00
Daan Leijen	fb12f298ca	merge from dev-win, fix small object size check	2019-08-16 19:14:08 -07:00
Daan Leijen	367fb046ec	Merge branch 'dev' into dev-win	2019-08-16 18:40:56 -07:00
Daan Leijen	069c26dbab	fix small object size check in mi_segment_page_alloc	2019-08-16 18:40:30 -07:00
Daan Leijen	91497e8d2d	whitespace and warning fix	2019-08-16 17:49:49 -07:00
daan	a0b4ac2f66	new segment allocation; good results with Qas service	2019-08-15 23:19:52 -07:00
daan	f2ba95bc64	first working version of new segment allocation	2019-08-15 22:00:42 -07:00
daan	6ee248b012	wip: fixing bugs in new segment allocation	2019-08-15 14:40:15 -07:00
daan	f2bafbc57f	wip: new segment allocation	2019-08-15 11:49:56 -07:00
daan	bbd81bbbd1	wip: new segment allocation with flexible large objects	2019-08-15 00:46:45 -07:00
daan	6d11e59250	fix to avoid potential linear behavior in page collect	2019-08-14 07:46:38 -07:00
daan	f35c2c5201	improve message printing from redirection module	2019-08-11 16:38:58 -07:00
daan	0fd898315c	per thread region search index	2019-08-11 12:15:13 -07:00
daan	2e924150ae	further layout improvement for msvc code generation	2019-08-11 11:30:24 -07:00
daan	7b105c4810	improve layout of page for x64	2019-08-11 10:31:00 -07:00
daan	dbe010f2d4	add huge and giant stats; increase bins	2019-08-10 20:51:37 -07:00
daan	737b05cc25	remove unused fields from OS thread local data	2019-08-10 17:50:27 -07:00
daan	ed785253bf	restructure the page flags to use explicit masks	2019-08-10 17:48:00 -07:00
daan	74a5c17178	update order of options; make eager_region_commit 0 on windows by default	2019-08-10 15:44:55 -07:00
daan	2fee6f98d7	Merge branch 'master' into dev	2019-08-10 15:23:43 -07:00
daan	b1938530af	fix comment	2019-08-09 14:31:45 -07:00
daan	7b16aa9302	remove accidental commit of different size bins experiment	2019-08-09 11:22:38 -07:00
daan	442bad9190	add good-fit for allowing larger blocks in smaller segments	2019-08-09 11:18:38 -07:00
daan	5e56b40fe6	improve page flags handling	2019-08-08 17:18:49 -07:00
daan	6596e970a5	move in_full and has_aligned into page threadid for a single test in mi_free	2019-08-08 15:23:18 -07:00
daan	55778d2fe4	improved stats	2019-08-08 11:36:13 -07:00
daan	189ad0f81d	small optimizations, use bitwise aligne	2019-07-23 15:00:13 -07:00
daan	66b8c37ab3	ensure C++ compilation on windows	2019-07-23 15:00:13 -07:00
daan	219d46ff0c	update test files and overriding	2019-07-23 15:00:13 -07:00
daan	6d9fab5af4	trailing id after #endif	2019-07-23 15:00:13 -07:00
Jim Huang	b5e26bedb1	Enforce strict include-what-you-use policy ... The include-what-you-use (IWYU) policy is beneficial to faster compilation and fewer recompilations. Many build tools, such as GNU make, provide a mechanism for automatically figuring out what .h files a .cc file depends on. These mechanisms typically look at #include lines. When unnecessary #includes are listed, the build system is more likely to recompile in cases where it is not necessary. With the enforcement, header file <include/mimalloc.h> no longer includes <stdlib.h>. Reference: https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/WhyIWYU.md	2019-07-23 14:59:57 -07:00
Jim Huang	8dba36bcec	Use checked unsigned multiplication extension of GCC/Clang ... Most processors have carry flags which they set on addition overflow, so it is a good idea to access them whenever possible. Most of them also have widening multiply instructions that can be used to detect overflow of the non-widening version. Both GCC and Clang offer a way to detect an overflow for security critical applications. Reference: https://clang.llvm.org/docs/LanguageExtensions.html#checked-arithmetic-builtins	2019-07-23 14:59:49 -07:00
Daan	ade118e23d	Merge pull request #88 from jserv/gcc-builtin-umul-overflow ... Use checked unsigned multiplication extension of GCC/Clang	2019-07-22 09:28:38 -07:00
daan	81a7ae33e7	fix order of options	2019-07-19 09:26:22 -07:00
daan	33aa5d92fa	fix to overrides on windows	2019-07-18 21:20:18 -07:00
daan	93c8a5b693	full overriding api supported	2019-07-18 21:12:40 -07:00
daan	306a542336	add mimalloc-override header file and use C++ compilation with msvc	2019-07-18 20:28:29 -07:00
daan	c228ecefd8	update for new dynamic direction on windows 64-bit	2019-07-18 18:59:32 -07:00
daan	f646cc925d	add is_in_heap_region function	2019-07-17 15:37:36 -07:00
daan	72d8608333	avoid thread over-allocation on initial region allocations	2019-07-15 17:35:43 -07:00
daan	7ea9cf8d1f	add region collection	2019-07-15 14:53:03 -07:00
daan	08e0f14c2a	document experimental options	2019-07-15 14:32:54 -07:00
Jim Huang	afd981d008	Use checked unsigned multiplication extension of GCC/Clang ... Most processors have carry flags which they set on addition overflow, so it is a good idea to access them whenever possible. Most of them also have widening multiply instructions that can be used to detect overflow of the non-widening version. Both GCC and Clang offer a way to detect an overflow for security critical applications. Reference: https://clang.llvm.org/docs/LanguageExtensions.html#checked-arithmetic-builtins	2019-07-16 05:28:59 +08:00
Daan	ad45dbf0e5	Merge pull request #32 from jedisct1/wasm ... Port to WebAssembly	2019-07-15 11:03:46 -07:00
daan	08c873c250	Merge branch 'dev' into dev-exp	2019-07-14 20:35:01 -07:00
daan	461b8aed80	more complete C++ support for compliance and performance	2019-07-14 19:56:33 -07:00
Frank Denis	9129ab0f64	Port to WebAssembly	2019-07-15 02:22:41 +02:00
daan	8dc0b5345b	Merge branch 'dev' into dev-exp	2019-07-14 16:50:38 -07:00
daan	49ceb4d018	refactor delayed_free code	2019-07-14 16:48:53 -07:00
daan	5ad2effb39	use standard compliant compilation of the thread_free atomic field	2019-07-14 16:20:27 -07:00
Daan	b09af48a68	Merge pull request #93 from JulianATA/dev-exp ... Fix typo numbes -> number	2019-07-14 14:33:37 -07:00
daan	84a292538c	merge with medium pages	2019-07-12 20:39:29 -07:00
daan	a927c07de9	merge medium pages	2019-07-12 20:27:56 -07:00
daan	42cf5e1580	reintroduce support for medium size pages	2019-07-12 20:11:39 -07:00
daan	8a6b474fa0	merge from dev	2019-07-11 16:32:11 -07:00
daan	a932e43650	experiment with larger small pages	2019-07-11 15:44:37 -07:00
daan	9e9208eae8	Merge branch 'dev' into dev-win	2019-07-11 15:22:09 -07:00
daan	1fdb4b288f	more eager handling of non-local frees	2019-07-11 15:21:57 -07:00
daan	6deea16d2d	add segment count to internal statistics	2019-07-10 19:54:46 -07:00
daan	224951695a	enable non eager commit	2019-07-10 07:24:56 -07:00
daan	46a9e51f74	enable non eager-commit flag	2019-07-10 07:17:21 -07:00
daan	3192ca4682	merge with dev-win	2019-07-09 23:34:17 -07:00
daan	3009a7b1fe	support non- eager_commit; on larson reduces commit charge 10x, but other benchmarks often see no improvement	2019-07-09 21:34:56 -07:00
daan	0314373d93	add reset_decommit, reset_discard options; maintain more statistics	2019-07-09 20:24:00 -07:00
Julian Fang	5928f12b15	Fix typo numbes -> number	2019-07-10 08:38:58 +08:00
daan	35b590de21	Merge branch 'dev' into dev-exp	2019-07-09 15:12:18 -07:00
daan	8ef37e6c0b	update atomic exchange memory model annotations for unix	2019-07-09 14:56:43 -07:00
daan	129149977d	Merge branch 'dev' into dev-exp	2019-07-08 17:45:04 -07:00
daan	8203f3dcfa	add full heap aligned allocation api (issue #52); remove rezalloc (issue #63); add mi_version	2019-07-08 17:17:07 -07:00
daan	493dfc4b82	merge with dev	2019-07-08 15:42:42 -07:00
daan	56bdb2c2f3	add more const declarations (pr #79)	2019-07-08 12:00:59 -07:00
daan	7f0137a617	Fix compilation error with MSVC C++	2019-07-07 21:59:53 -07:00
daan	7e743dfb58	fix C++ compilation warnings with VisualStudio 2017	2019-07-07 18:52:12 -07:00
daan	c3528203b5	fix compilation with C++, fix overrides in C++ to adhere to the spec (issue #26)	2019-07-07 18:11:21 -07:00
daan	dd59a917ce	Expose mi prefixed public standard implementations of various posix, unix, and c++ allocation functions. See issue #75	2019-07-07 13:44:33 -07:00
daan	983c96a238	Merge branch 'dev' into dev-exp	2019-07-04 14:46:53 -07:00
daan	6abdc5f5a3	do not enable verbose by default in the debug build; add trace messages for verbose level 2	2019-07-04 09:28:22 -07:00
daan	663769c512	merge dev branch	2019-07-03 18:12:55 -07:00
daan	5c0ffe9865	merge back changes to OS module from 'dev-exp'	2019-07-02 17:17:03 -07:00
daan	8cb36c62e6	merge with dev branch	2019-07-02 16:44:35 -07:00
daan	aeff1db32b	Fixing interpose on macOS	2019-07-02 12:49:28 -07:00
daan	d35fc6cdc4	PR#66 by kickunderscore to ensure consistent small block alignment	2019-07-02 08:46:50 -07:00
daan	06bcea1761	Initial commit of separate memory region layer and improved large OS pages support, see 'memory.c'	2019-07-02 07:23:24 -07:00
daan	d6901558cd	enable large OS page support on Windows	2019-06-28 22:35:57 -07:00
daan	12d756e7b2	fix page count of heap when absorbing	2019-06-27 13:32:44 -07:00
Julian Fang	c19d3a27a8	Remove underscore prefixes to match c lanuguage standard	2019-06-26 12:57:13 +08:00
daan	fac0d4b409	fix passing void as argument	2019-06-25 20:06:34 -07:00
daan	654bcd7049	Merge branch 'master' into dev	2019-06-25 19:57:45 -07:00
daan	9ba53d3e2b	fix missing prototype for heap_alloc_small	2019-06-25 19:57:21 -07:00
Frank Denis	236cd2e65c	fn() is not a valid C prototype	2019-06-25 12:16:36 +02:00
daan	c1226611c6	Merge branch 'master' into dev	2019-06-24 18:13:04 -07:00
daan	a16d95375f	remove lock code from atomics (as it is unused)	2019-06-24 17:54:41 -07:00
Daan	8e244f6616	Merge pull request #33 from dscho/fix-prototypes ... Use `void` in the signatures of parameter-less functions	2019-06-24 17:49:50 -07:00
Johannes Schindelin	464efa80b0	Fix prototypes of parameter-less functions ... GCC does want to complain about parameter-less functions declared without the recommended `void` as parameter list. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>	2019-06-24 17:03:33 +02:00
daan	cd7aa0ce83	improve handling of huge pages in the cache	2019-06-23 23:15:42 -07:00
daan	43568fa422	ensure huge pages are part of the segment cache to avoid mmap calls	2019-06-23 21:37:43 -07:00
Jim Huang	a667971cbf	Apple rebrands "OS X" as "macOS" ... In 2012, with the release of OS X 10.8 Mountain Lion, the name of the system was shortened from Mac OS X to OS X. In 2016, with the release of macOS 10.12 Sierra, the name was changed from OS X to macOS to streamline it with the branding of Apple's other primary operating systems: iOS, watchOS, and tvOS. Reference: https://en.wikipedia.org/wiki/MacOS	2019-06-24 10:15:50 +08:00
Daan	fab7329c7a	Merge pull request #24 from jserv/clarify-license ... Clarify top-level license file	2019-06-23 10:47:38 -07:00
Daan	902b8ba639	Merge pull request #31 from jserv/fix-typo-again ... Fix typo in comments	2019-06-23 10:46:00 -07:00
Jim Huang	78c7a1f349	Fix typo in comments	2019-06-24 01:40:28 +08:00
Daan	f3cea2c22a	Merge pull request #19 from jserv/arm-aarch64-tls-register ... Support ARM/Aarch64 TLS register fastpath	2019-06-23 10:39:28 -07:00
Jim Huang	b1d299b35b	Clarify top-level license file ... To be consistent with other Microsoft open source projects, each source file points to the file "LICENSE" in top-level directory.	2019-06-23 19:53:34 +08:00
Jim Huang	77991fccee	Implement ARM/Aarch64 atomic_yield fastpath	2019-06-23 16:53:49 +08:00
Jim Huang	3841b32f9d	Support ARM/Aarch64 TLS register fastpath ... Tested with Ubuntu Linux 18.04 LTS running on Marvell/Cavium ThunderX, which consists of Armv8 based processors.	2019-06-23 16:30:20 +08:00
Daan Leijen	79bfe52255	fix accidental delete	2019-06-22 11:50:03 -07:00
Daan Leijen	e52ca3bfa9	possible fix for ARM linux compilation	2019-06-22 11:33:18 -07:00
Sungmann Cho	def97b0fd1	Fix spelling errors ... aligment -> alignment purgable -> purgeable thead -> thread	2019-06-23 00:09:11 +09:00
daan	26a874eb3f	initial checkin	2019-06-19 16:26:12 -07:00