./devel/snappy, General purpose data compression library

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 1.1.6, Package name: snappy-1.1.6, Maintainer: agc

Snappy is a compression/decompression library. It does not aim for
maximum compression, or compatibility with any other compression
library; instead, it aims for very high speeds and reasonable
compression. For instance, compared to the fastest mode of zlib,
Snappy is an order of magnitude faster for most inputs, but the
resulting compressed files are anywhere from 20% to 100% bigger. On a
single core of a Core i7 processor in 64-bit mode, Snappy compresses
at about 250 MB/sec or more and decompresses at about 500 MB/sec or

Snappy is widely used inside Google, in everything from BigTable and
MapReduce to the internal RPC systems. (Snappy has previously been
referred to as "Zippy" in some presentations and the likes.)

Required to run:

Required to build:
[devel/googletest] [pkgtools/cwrappers]

Master sites:

SHA1: 8d428406c3145edf1d1096cc6eddb995e32e1655
RMD160: 9b71dc2cb330dac502fda4565895cba4f02ae044
Filesize: 1101.941 KB

Version history: (Expand)

CVS history: (Expand)

   2017-07-14 19:27:41 by Adam Ciarcinski | Files touched by this commit (6)
Log message:
Snappy v1.1.6:

This is a re-release of v1.1.5 with proper SONAME / SOVERSION values.

Snappy v1.1.5:

This release has broken SONAME / SOVERSION values. Users of snappy as a shared
library should avoid 1.1.5 and use 1.1.6 instead. SONAME / SOVERSION errors will
manifest as the dynamic library loader complaining that it cannot find snappy's
shared library file (libsnappy.so / libsnappy.dylib), or that the library it
found does not have the required version. 1.1.6 has the same code as 1.1.5, but
carries build configuration fixes for the issues above.

  * Add CMake build support. The autoconf build support is now deprecated, and
    will be removed in the next release.

  * Add AppVeyor configuration, for Windows CI coverage.

  * Small performance improvement on little-endian PowerPC.

  * Small performance improvement on LLVM with position-independent executables.

  * Fix a few issues with various build environments.
   2017-06-24 20:57:28 by Alistair G. Crooks | Files touched by this commit (1)
Log message:
snappy now requires googletest to build, so include the buildlink3.mk file
   2017-05-31 10:52:41 by Jonathan Perkin | Files touched by this commit (2)
Log message:
Apply patch from upstream to fix SSE2 include.
   2017-02-26 09:50:19 by Adam Ciarcinski | Files touched by this commit (1)
Log message:
   2017-02-26 09:41:17 by Adam Ciarcinski | Files touched by this commit (5)
Log message:
Changes 1.1.4:
Fix a 1% performance regression when snappy is used in PIE executables.
Improve compression performance by 5%.
Improve decompression performance by 20%.
   2016-09-19 11:30:36 by Filip Hajny | Files touched by this commit (2) | Package updated
Log message:
Update devel/snappy to 1.1.3.

Snappy v1.1.3, July 6th 2015:

This is the first release to be done from GitHub, which means that
some minor things like the ChangeLog format has changed (git log
format instead of svn log).

  * Add support for Uncompress() from a Source to a Sink.

  * Various minor changes to improve MSVC support; in particular,
    the unit tests now compile and run under MSVC.

Snappy v1.1.2, February 28th 2014:

This is a maintenance release with no changes to the actual
source code.

  * Stop distributing benchmark data files that have unclear
    or unsuitable licensing.

  * Add support for padding chunks in the framing format.
   2015-11-03 04:29:40 by Alistair G. Crooks | Files touched by this commit (1995)
Log message:
Add SHA512 digests for distfiles for devel category

Issues found with existing distfiles:
No changes made to these distinfo files.

Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden).  All existing
SHA1 digests retained for now as an audit trail.
   2013-11-16 20:31:30 by Matthew Sporleder | Files touched by this commit (2) | Package updated
Log message:
Update to 1.1.1

(benchmarks clipped from release notes)

r80 | snappy.mirrorbot@gmail.com | 2013-08-13 14:55:00 +0200 (Tue, 13 Aug 2013) \ 
| 6 lines

Add autoconf tests for size_t and ssize_t. Sort-of resolves public issue 79;
it would solve the problem if MSVC typically used autoconf. However, it gives
a natural place (config.h) to put the typedef even for MSVC.


r79 | snappy.mirrorbot@gmail.com | 2013-07-29 13:06:44 +0200 (Mon, 29 Jul 2013) \ 
| 14 lines

When we compare the number of bytes produced with the offset for a
backreference, make the signedness of the bytes produced clear,
by sticking it into a size_t. This avoids a signed/unsigned compare
warning from MSVC (public issue 71), and also is slightly clearer.

Since the line is now so long the explanatory comment about the -1u
trick has to go somewhere else anyway, I used the opportunity to
explain it in slightly more detail.

This is a purely stylistic change; the emitted assembler from GCC
is identical.


r78 | snappy.mirrorbot@gmail.com | 2013-06-30 21:24:03 +0200 (Sun, 30 Jun 2013) \ 
| 111 lines

In the fast path for decompressing literals, instead of checking
whether there's 16 bytes free and then checking right afterwards
(when having subtracted the literal size) that there are now
5 bytes free, just check once for 21 bytes. This skips a compare
and a branch; although it is easily predictable, it is still
a few cycles on a fast path that we would like to get rid of.

Benchmarking this yields very confusing results. On open-source
GCC 4.8.1 on Haswell, we get exactly the expected results; the
benchmarks where we hit the fast path for literals (in particular
the two HTML benchmarks and the protobuf benchmark) give very nice
speedups, and the others are not really affected.

However, benchmarks with Google's GCC branch on other hardware
is much less clear. It seems that we have a weak loss in some cases
(and the win for the typical win cases are not nearly as clear),
but that it depends on microarchitecture and plain luck in how we run
the benchmark. Looking at the generated assembler, it seems that
the removal of the if causes other large-scale changes in how the
function is laid out, which makes it likely that this is just bad luck.

Thus, we should keep this change, even though its exact current impact is
unclear; it's a sensible change per se, and dropping it on the basis of
microoptimization for a given compiler (or even branch of a compiler)
would seem like a bad strategy in the long run.

r77 | snappy.mirrorbot@gmail.com | 2013-06-14 23:42:26 +0200 (Fri, 14 Jun 2013) \ 
| 92 lines

Make the two IncrementalCopy* functions take in an ssize_t instead of a len,
in order to avoid having to do 32-to-64-bit signed conversions on a hot path
during decompression. (Also fixes some MSVC warnings, mentioned in public
issue 75, but more of those remain.) They cannot be size_t because we expect
them to go negative and test for that.

This saves a few movzwl instructions, yielding ~2% speedup in decompression.

r76 | snappy.mirrorbot@gmail.com | 2013-06-13 18:19:52 +0200 (Thu, 13 Jun 2013) \ 
| 9 lines

Add support for uncompressing to iovecs (scatter I/O).
Windows does not have struct iovec defined anywhere,
so we define our own version that's equal to what UNIX
typically has.

The bulk of this patch was contributed by Mohit Aron.


r75 | snappy.mirrorbot@gmail.com | 2013-06-12 21:51:15 +0200 (Wed, 12 Jun 2013) \ 
| 4 lines

Some code reorganization needed for an internal change.


r74 | snappy.mirrorbot@gmail.com | 2013-04-09 17:33:30 +0200 (Tue, 09 Apr 2013) \ 
| 4 lines

Supports truncated test data in zippy benchmark.


r73 | snappy.mirrorbot@gmail.com | 2013-02-05 15:36:15 +0100 (Tue, 05 Feb 2013) \ 
| 4 lines

Release Snappy 1.1.0.


r72 | snappy.mirrorbot@gmail.com | 2013-02-05 15:30:05 +0100 (Tue, 05 Feb 2013) \ 
| 9 lines

Make ./snappy_unittest pass without "srcdir" being defined.

Previously, snappy_unittests would read from an absolute path /testdata/..;
convert it to use a relative path instead.

Patch from Marc-Antonie Ruel.


r71 | snappy.mirrorbot@gmail.com | 2013-01-18 13:16:36 +0100 (Fri, 18 Jan 2013) \ 
| 287 lines

Increase the Zippy block size from 32 kB to 64 kB, winning ~3% density
while being effectively performance neutral.

The longer story about density is that we win 3-6% density on the benchmarks
where this has any effect at all; many of the benchmarks (cp, c, lsp, man)
are smaller than 32 kB and thus will have no effect. Binary data also seems
to win little or nothing; of course, the already-compressed data wins nothing.
The protobuf benchmark wins as much as ~18% depending on architecture,
but I wouldn't be too sure that this is representative of protobuf data in

As of performance, we lose a tiny amount since we get more tags (e.g., a long
literal might be broken up into literal-copy-literal), but we win it back with
less clearing of the hash table, and more opportunities to skip incompressible
data (e.g. in the jpg benchmark). Decompression seems to get ever so slightly
slower, again due to more tags. The total net change is about as close to zero
as we can get, so the end effect seems to be simply more density and no
real performance change.

The comment about not changing kBlockSize, scary as it is, is not really
relevant, since we're never going to have a block-level decompressor without
explicitly marked blocks. Replace it with something more appropriate.

This affects the framing format, but it's okay to change it since it basically
has no users yet.

r70 | snappy.mirrorbot@gmail.com | 2013-01-06 20:21:26 +0100 (Sun, 06 Jan 2013) \ 
| 6 lines

Adjust the Snappy open-source distribution for the changes in Google's
internal file API.


r69 | snappy.mirrorbot@gmail.com | 2013-01-04 12:54:20 +0100 (Fri, 04 Jan 2013) \ 
| 15 lines

Change a few ORs to additions where they don't matter. This helps the compiler
use the LEA instruction more efficiently, since e.g. a + (b << 2) can be \ 
as one instruction. Even more importantly, it can constant-fold the
COPY_* enums together with the shifted negative constants, which also saves
some instructions. (We don't need it for LITERAL, since it happens to be 0.)

I am unsure why the compiler couldn't do this itself, but the theory is that
it cannot prove that len-1 and len-4 cannot underflow/wrap, and thus can't
do the optimization safely.

The gains are small but measurable; 0.5-1.0% over the BM_Z* benchmarks
(measured on Westmere, Sandy Bridge and Istanbul).


r68 | snappy.mirrorbot@gmail.com | 2012-10-08 13:37:16 +0200 (Mon, 08 Oct 2012) \ 
| 5 lines

Stop giving -Werror to automake, due to an incompatibility between current
versions of libtool and automake on non-GNU platforms (e.g. Mac OS X).


r67 | snappy.mirrorbot@gmail.com | 2012-08-17 15:54:47 +0200 (Fri, 17 Aug 2012) \ 
| 5 lines

Fix public issue 66: Document GetUncompressedLength better, in particular that
it leaves the source in a state that's not appropriate for RawUncompress.


r66 | snappy.mirrorbot@gmail.com | 2012-07-31 13:44:44 +0200 (Tue, 31 Jul 2012) \ 
| 5 lines

Fix public issue 64: Check for <sys/time.h> at configure time,
since MSVC seemingly does not have it.


r65 | snappy.mirrorbot@gmail.com | 2012-07-04 11:34:48 +0200 (Wed, 04 Jul 2012) \ 
| 10 lines

Handle the case where gettimeofday() goes backwards or returns the same value
twice; it could cause division by zero in the unit test framework.
(We already had one fix for this in place, but it was incomplete.)

This could in theory happen on any system, since there are few guarantees
about gettimeofday(), but seems to only happen in practice on GNU/Hurd, where
gettimeofday() is cached and only updated ever so often.


r64 | snappy.mirrorbot@gmail.com | 2012-07-04 11:28:33 +0200 (Wed, 04 Jul 2012) \ 
| 6 lines

Mark ARMv4 as not supporting unaligned accesses (not just ARMv5 and ARMv6);
apparently Debian still targets these by default, giving us segfaults on


r63 | snappy.mirrorbot@gmail.com | 2012-05-22 11:46:05 +0200 (Tue, 22 May 2012) \ 
| 5 lines

Fix public bug #62: Remove an extraneous comma at the end of an enum list,
causing compile errors when embedded in Mozilla on OpenBSD.


r62 | snappy.mirrorbot@gmail.com | 2012-05-22 11:32:50 +0200 (Tue, 22 May 2012) \ 
| 8 lines

Snappy library no longer depends on iostream.

Achieved by moving logging macro definitions to a test-only
header file, and by changing non-test code to use assert,
fprintf, and abort instead of LOG/CHECK macros.