Commit Graph

26 Commits

Author SHA1 Message Date
leveldb Team
1d6e8d64ee Add support for Zstd-based compression in LevelDB.
This change implements support for Zstd-based compression in LevelDB. Building
up from the Snappy compression (which has been supported since inception), this
change adds Zstd as an alternate compression algorithm.

We are implementing this to provide alternative options for users who might
have different performance and efficiency requirements. For instance, the
Zstandard website (https://facebook.github.io/zstd/) claims that the Zstd
algorithm can achieve around 30% higher compression ratios than Snappy, with
relatively smaller (~10%) slowdowns in de/compression speeds.

Benchmarking results:

$ blaze-bin/third_party/leveldb/db_bench
LevelDB:    version 1.23
Date:       Thu Feb  2 18:50:06 2023
CPU:        56 * Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
CPUCache:   35840 KB
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
RawSize:    110.6 MB (estimated)
FileSize:   62.9 MB (estimated)
------------------------------------------------
fillseq      :       2.613 micros/op;   42.3 MB/s
fillsync     :    3924.432 micros/op;    0.0 MB/s (1000 ops)
fillrandom   :       3.609 micros/op;   30.7 MB/s
overwrite    :       4.508 micros/op;   24.5 MB/s
readrandom   :       6.136 micros/op; (864322 of 1000000 found)
readrandom   :       5.446 micros/op; (864083 of 1000000 found)
readseq      :       0.180 micros/op;  613.3 MB/s
readreverse  :       0.321 micros/op;  344.7 MB/s
compact      :  827043.000 micros/op;
readrandom   :       4.603 micros/op; (864105 of 1000000 found)
readseq      :       0.169 micros/op;  656.3 MB/s
readreverse  :       0.315 micros/op;  350.8 MB/s
fill100K     :     854.009 micros/op;  111.7 MB/s (1000 ops)
crc32c       :       1.227 micros/op; 3184.0 MB/s (4K per op)
snappycomp   :       3.610 micros/op; 1081.9 MB/s (output: 55.2%)
snappyuncomp :       0.691 micros/op; 5656.3 MB/s
zstdcomp     :      15.731 micros/op;  248.3 MB/s (output: 44.1%)
zstduncomp   :       4.218 micros/op;  926.2 MB/s
PiperOrigin-RevId: 509957778
2023-03-28 16:49:13 -07:00
leveldb Team
aa5479bbf4 Fix maintenance text
PiperOrigin-RevId: 461725664
2022-07-18 22:11:43 +00:00
Victor Costan
ca684d00b5 Fix Markdown formatting in README.
PiperOrigin-RevId: 461722304
2022-07-18 21:59:46 +00:00
leveldb Team
7b650f85de Add note on current development state
PiperOrigin-RevId: 461695246
2022-07-18 21:20:02 +00:00
Victor Costan
4fb146810c The master branch was renamed to main.
PiperOrigin-RevId: 422409116
2022-01-17 21:37:53 +00:00
Victor Costan
479a1f4e9b Update contributing guidelines.
* Align CONTRIBUTING.md with the google/new-project template.
* Explain the support story for the CMake config.

PiperOrigin-RevId: 421120645
2022-01-11 22:37:16 +00:00
Victor Costan
5783a79309 Switch CI to GitHub Actions.
PiperOrigin-RevId: 394542401
2021-09-02 21:33:50 +00:00
Dimitris Apostolou
532be85306
Fix insecure links 2020-12-19 20:07:38 +02:00
Victor Costan
1c58902bdc Switch testing harness to googletest.
PiperOrigin-RevId: 281815695
2019-11-21 13:11:40 -08:00
Chris Mumford
046216a7ca Add "leveldb" subdirectory to public include paths.
The documentation (README.md and index.md) referred to the
public headers using an incorrect path - fixing.

PiperOrigin-RevId: 252922925
2019-06-13 15:20:12 -07:00
Chris Mumford
9ee91ac747 Ending sentences with periods in README.md.
This change was submitted in https://github.com/google/leveldb/pull/575
by @prajwalchalla.

This fixes issue #523.

PiperOrigin-RevId: 252912613
2019-06-13 13:48:54 -07:00
Chris Mumford
297e66afc1 Format all files IAW the Google C++ Style Guide.
Use clang-format to correct formatting to be in agreement with the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html). Doing this simplifies the process of accepting changes. Also fixed a few warnings flagged by clang-tidy.

PiperOrigin-RevId: 246350737
2019-05-02 19:04:50 -07:00
Chris Mumford
7b11745190 Changed Windows specific highlighting from bash to cmd.
This makes the syntax highlighting a little nicer on GitHub.

PiperOrigin-RevId: 243426806
2019-04-13 09:20:48 -07:00
Pavel Pimenov
952be04df6 Fix mkdir (windows) 2019-03-31 10:46:31 +03:00
costan
04470825ac Add AppVeyor (Windows CI) badge to README.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237295321
2019-03-07 15:59:36 -08:00
cmumford
c69d33b0ec Added native support for Windows.
This change adds a native Windows port (port_windows.h) and a
Windows Env (WindowsEnv).

Note1: "small" is defined when including <Windows.h> so some
parameters were renamed to avoid conflict.

Note2: leveldb::Env defines the method: "DeleteFile" which is
also a constant defined when including <Windows.h>. The solution
was to ensure this macro is defined in env.h which forces
the function, when compiled, to be either DeleteFileA or
DeleteFileW when building for MBCS or UNICODE respectively.

This resolves #519 on GitHub.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236364778
2019-03-01 18:00:35 -08:00
costan
1cc8b10b82 Document the building process.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192490601
2018-04-11 12:57:42 -07:00
cmumford
d0883b6006 Fixed path to doc file: index.md.
Prior index.html was using rawgit.com which doesn't process
Markdown and therefore only serves the markdown source.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148902180
2017-03-01 10:28:56 -08:00
cmumford
7fa20948d5 Convert documentation to markdown.
Markdown is more readable in a text editor and when hosted
on GitHub is more readable than HTML.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148830423
2017-03-01 09:42:25 -08:00
Lars-Magnus Skog
dd1c3c3572 add travis build badge 2016-01-15 18:29:01 +01:00
Chris Mumford
8ec241a3b0 Merge pull request #317 from falvojr/patch-1
Update README.md
2016-01-12 10:52:33 -08:00
cmumford
4753c9b617 Added a contributors section to README.md
In preparation for accepting GitHub pull requests this new README
section outlines the general criteria that the leveldb project owners
will use when accepting external (and internal) project contributions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=111349899
2016-01-04 13:29:41 -08:00
Venilton FalvoJr
edf2939c0d Update README.md 2015-11-23 16:24:16 -02:00
Paul Irish
0e0f07417c documentation. improved link 2015-02-17 09:55:40 -08:00
Paul Irish
c85addcdf3 readme: improved documentation link
This replaces htmlpreview with [rawgit](http://rawgit.com/).
rawgit is faster and doesnt use JS to render the page, making it SEO friendly.
2015-01-10 17:36:40 -08:00
Chris Mumford
803d69203a Release 1.18
Changes are:

* Update version number to 1.18
* Replace the basic fprintf call with a call to fwrite in order to
  work around the apparent compiler optimization/rewrite failure that we are
  seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8.
* Fix ALL the header guards.
* Createed a README.md with the LevelDB project description.
* A new CONTRIBUTING file.
* Don't implicitly convert uint64_t to size_t or int.  Either preserve it as
  uint64_t, or explicitly cast. This fixes MSVC warnings about possible value
  truncation when compiling this code in Chromium.
* Added a DumpFile() library function that encapsulates the guts of the
  "leveldbutil dump" command. This will allow clients to dump
  data to their log files instead of stdout. It will also allow clients to
  supply their own environment.
* leveldb: Remove unused function 'ConsumeChar'.
* leveldbutil: Remove unused member variables from WriteBatchItemPrinter.
* OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define
  PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes:
   * issue #143
   * issue #198
   * issue #249
* Switch from <cstdatomic> to <atomic>. The former never made it into the
  standard and doesn't exist in modern gcc versions at all.  The later contains
  everything that leveldb was using from the former.
  This problem was noticed when porting to Portable Native Client where no memory
  barrier is defined.  The fact that <cstdatomic> is missing normally goes
  unnoticed since memory barriers are defined for most architectures.
* Make Hash() treat its input as unsigned.  Before this change LevelDB files
  from platforms with different signedness of char were not compatible. This
  change fixes: issue #243
* Verify checksums of index/meta/filter blocks when paranoid_checks set.
* Invoke all tools for iOS with xcrun. (This was causing problems with the new
  XCode 5.1.1 image on pulse.)
* include <sys/stat.h> only once, and fix the following linter warning:
  "Found C system header after C++ system header"
* When encountering a corrupted table file, return Status::Corruption instead of
  Status::InvalidArgument.
* Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188
* Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159
* Fix typos and comments, and address the following two issues:
  * issue #166
  * issue #241
* Add missing db synchronize after "fillseq" in the benchmark.
* Removed unused variable in SeekRandom: value (issue #201)
2014-09-16 14:19:52 -07:00