tqcq/leveldb
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e9fd9c5b14
leveldb/port/port_config.h.in

39 lines
1.2 KiB
C
Raw Normal View History

Add CMake build support. Fixes https://github.com/google/leveldb/issues/466 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189425354
2018-03-17 10:10:13 +08:00
// Copyright 2017 The LevelDB Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file. See the AUTHORS file for names of contributors.
#ifndef STORAGE_LEVELDB_PORT_PORT_CONFIG_H_
#define STORAGE_LEVELDB_PORT_PORT_CONFIG_H_
// Define to 1 if you have a definition for fdatasync() in <unistd.h>.
Fix fdatasync() feature detection in opensource build. The CMake feature-detection code used check_symbol_exists(), which invokes the C compiler. However, some glibc versions don't expose the fdatasync() declaration when compiled with -std=c11, but do expose it when compiled with -std=c++11. This most likely comes down to how _POSIX_SOURCE is defined -- it needs to be >= 201112L for <unistd.h> to expose fdatasync(). This CL switches to check_cxx_symbol_exists(), which uses the C++ compiler. Asides from fixing the problem above, this is the right thing to do, because we use <unistd.h> in env_posix.cc, which is compiled with the C++ compiler. This CL also fixes a previously introduced inconsistency, where the macro indicating the fdatasync() feature detection result was referred to as HAVE_FDATASYNC and HAVE_FUNC_FDATASYNC. The former appears to be used in other libraries, so this CL switches all our references to HAVE_FDATASYNC. Fixes https://github.com/google/leveldb/issues/629 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=228392612
2019-01-09 05:49:13 +08:00
#if !defined(HAVE_FDATASYNC)
#cmakedefine01 HAVE_FDATASYNC
#endif // !defined(HAVE_FDATASYNC)
Add CMake build support. Fixes https://github.com/google/leveldb/issues/466 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189425354
2018-03-17 10:10:13 +08:00
leveldb: Fix PosixWritableFile::Sync() on Apple systems. Apple doesn't follow POSIX specifications for fsync(). Instead, fsync() guarantees to flush the buffer cache to the device, which means the data will survive kernel panics, but may not survive power outages. Applications that need stronger guarantees (like databases) need to use fcntl(F_FULLFSYNC). This CL switches PosixWritableFile::Sync() to get the stronger guarantees on Apple systems. The improved implementation follows the same principles as SQLite [1] and node.js [2]. Research for the fcntl() to fsync() fallback strategy: Apple's released source code at https://opensource.apple.com/ shows at least three different error codes being returned when a filesystem does not support F_FULLFSYNC. fcntl() is implemented in xnu-4903.221.2 in bsd/kern/kern_descrip.c, where it delegates to fcntl_nocancel(). The documentation for fcntl_nocancel() mentions error codes for some operations, but does not include F_FULLFSYNC. The F_FULLSYNC branch in fcntl_nocancel() calls VNOP_IOCTL(_, F_FULLSYNC, NULL, 0, _), whose return value sets the error code. VNOP_IOCTL() is implemented in bsd/vfs/kpi_vfs.c and calls the ioctl function in the vnode's operation vector. The per-filesystem function names follow the pattern _vnop_ioctl() for all the instances in opensource code: {hfs,msdosfs,nfs,ntfs,smbfs,webdav,zfs}_vnop_ioctl(). hfs-407.30.1, msdosfs-229.200.3, and nfs in xnu-4903.221.2 handle F_FULLFSYNC. ntfs-94.200.1 and smb-759.40.1 do not handle F_FULLFSYNC, and the default branch returns ENOSUP. webdav-380.200.1 also does not handle F_FULLFSYNC, but the default branch returns EINVAL. zfs-59 also does not handle F_FULLSYNC, and its default branch returns ENOTTY. From a different angle, Apple's ntfs-94.200.1 includes utility code that uses fcntl(F_FULLFSYNC) and falls back to fsync() just like we do, supporting the hypothesis that there is no good way to detect lack of F_FULLFSYNC support. Also, Apple's fcntl() man page [3] does not mention a way to detect lack of F_FULLFSYNC support. [1] https://www.sqlite.org/src/doc/trunk/src/os_unix.c [2] https://github.com/libuv/libuv/blob/master/src/unix/fs.c [3] https://developer.apple.com/library/archive/documentatiVon/System/Conceptual/ManPages_iPhoneOS/man2/fcntl.2.html Tested: https://travis-ci.org/pwnall/leveldb/builds/477318498 TAP global presubmit ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=228593729
2019-01-10 06:53:09 +08:00
// Define to 1 if you have a definition for F_FULLFSYNC in <fcntl.h>.
#if !defined(HAVE_FULLFSYNC)
#cmakedefine01 HAVE_FULLFSYNC
#endif // !defined(HAVE_FULLFSYNC)
Add O_CLOEXEC to open calls. This prevents file descriptors from leaking to child processes. When compiled for older (pre-2.6.23) kernels which lack support for O_CLOEXEC there is no change in behavior. With newer kernels, child processes will no longer inherit leveldb's file handles, which reduces the changes of accidentally corrupting the database. Fixes https://github.com/google/leveldb/issues/623
2018-09-20 07:29:00 +08:00
// Define to 1 if you have a definition for O_CLOEXEC in <fcntl.h>.
#if !defined(HAVE_O_CLOEXEC)
#cmakedefine01 HAVE_O_CLOEXEC
#endif // !defined(HAVE_O_CLOEXEC)
Add CMake build support. Fixes https://github.com/google/leveldb/issues/466 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189425354
2018-03-17 10:10:13 +08:00
// Define to 1 if you have Google CRC32C.
#if !defined(HAVE_CRC32C)
#cmakedefine01 HAVE_CRC32C
#endif // !defined(HAVE_CRC32C)
// Define to 1 if you have Google Snappy.
#if !defined(HAVE_SNAPPY)
#cmakedefine01 HAVE_SNAPPY
#endif // !defined(HAVE_SNAPPY)
Add support for Zstd-based compression in LevelDB. This change implements support for Zstd-based compression in LevelDB. Building up from the Snappy compression (which has been supported since inception), this change adds Zstd as an alternate compression algorithm. We are implementing this to provide alternative options for users who might have different performance and efficiency requirements. For instance, the Zstandard website (https://facebook.github.io/zstd/) claims that the Zstd algorithm can achieve around 30% higher compression ratios than Snappy, with relatively smaller (~10%) slowdowns in de/compression speeds. Benchmarking results: $ blaze-bin/third_party/leveldb/db_bench LevelDB: version 1.23 Date: Thu Feb 2 18:50:06 2023 CPU: 56 * Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz CPUCache: 35840 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 2.613 micros/op; 42.3 MB/s fillsync : 3924.432 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.609 micros/op; 30.7 MB/s overwrite : 4.508 micros/op; 24.5 MB/s readrandom : 6.136 micros/op; (864322 of 1000000 found) readrandom : 5.446 micros/op; (864083 of 1000000 found) readseq : 0.180 micros/op; 613.3 MB/s readreverse : 0.321 micros/op; 344.7 MB/s compact : 827043.000 micros/op; readrandom : 4.603 micros/op; (864105 of 1000000 found) readseq : 0.169 micros/op; 656.3 MB/s readreverse : 0.315 micros/op; 350.8 MB/s fill100K : 854.009 micros/op; 111.7 MB/s (1000 ops) crc32c : 1.227 micros/op; 3184.0 MB/s (4K per op) snappycomp : 3.610 micros/op; 1081.9 MB/s (output: 55.2%) snappyuncomp : 0.691 micros/op; 5656.3 MB/s zstdcomp : 15.731 micros/op; 248.3 MB/s (output: 44.1%) zstduncomp : 4.218 micros/op; 926.2 MB/s PiperOrigin-RevId: 509957778
2023-02-16 08:04:43 +08:00
// Define to 1 if you have Zstd.
#if !defined(HAVE_Zstd)
#cmakedefine01 HAVE_ZSTD
#endif // !defined(HAVE_ZSTD)
#endif // STORAGE_LEVELDB_PORT_PORT_CONFIG_H_
Reference in New Issue Copy Permalink