./archivers/py-zstandard, Zstandard bindings for Python

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 0.15.2, Package name: py38-zstandard-0.15.2, Maintainer: pkgsrc-users

This project provides Python bindings for interfacing with the
Zstandard compression library. A C extension and CFFI interface are

The primary goal of the project is to provide a rich interface to
the underlying C API through a Pythonic interface while not sacrificing
performance. This means exposing most of the features and flexibility
of the C API while not sacrificing usability or safety that Python

Required to run:
[devel/py-setuptools] [devel/py-cffi] [archivers/zstd] [lang/python37]

Required to build:

Master sites:

SHA1: 66ffb728d478d94a0d0f16bdfff23373b3b536bc
RMD160: 90309af02d82c8524c4d2b6eabdb5fb2cac46c27
Filesize: 989.125 KB

Version history: (Expand)

CVS history: (Expand)

   2021-02-28 19:35:32 by Adam Ciarcinski | Files touched by this commit (2) | Package updated
Log message:
py-zstandard: updated to 0.15.2


Backwards Compatibility Notes

ZstdCompressor.multi_compress_to_buffer() and
ZstdDecompressor.multi_decompress_to_buffer() are no longer
available when linking against a system zstd library. These
experimental features are only available when building against the
bundled single file zstd C source file distribution.


setup.py now recognizes a ZSTD_EXTRA_COMPILER_ARGS
environment variable to specify additional compiler arguments
to use when compiling the C backend.
PyPy build and test coverage has been added to CI.
Added CI jobs for building against external zstd library.
Wheels supporting macOS ARM/M1 devices are now being produced.
References to Python 2 have been removed from the in-repo Debian packaging
Significant work has been made on a Rust backend. It is currently feature
complete but not yet optimized. We are not yet shipping the backend as part
of the distributed wheels until it is more mature.
The .pyi type annotations file has replaced various default argument
values with ....
   2021-01-03 15:57:53 by Adam Ciarcinski | Files touched by this commit (4) | Package updated
Log message:
py-zstandard: updated to 0.15.1


Bug Fixes

setup.py no longer attempts to build the C backend on PyPy.
<sys/types.h> is now included before <sys/sysctl.h>. This was
the case in releases prior to 0.15.0 and the include order was reversed
as part of running clang-format. The old/working order has been
Include some private zstd C headers so we can build the C extension against
a system library. The previous behavior of referencing these headers is
restored. That behave is rather questionable and undermines the desire to
use the system zstd.


Backwards Compatibility Notes

Support for Python 2.7 has been dropped. Python 3.5 is now the
minimum required Python version.
train_dictionary() now uses the fastcover training mechanism
(as opposed to cover). Some parameter values that worked with the old
mechanism may not work with the new one. e.g. d must be 6 or 8
if it is defined.
train_dictionary() now always calls
ZDICT_optimizeTrainFromBuffer_fastCover() instead of different APIs
depending on which arguments were passed.
The names of various Python modules have been changed. The C extension
is now built as zstandard.backend_c instead of zstd. The
CFFI extension module is now built as zstandard._cffi instead of
_zstd_cffi. The CFFI backend is now zstandard.backend_cffi instead
of zstandard.cffi.
ZstdDecompressionReader.seekable() now returns False instead of
True because not all seek operations are supported and some Python
code in the wild keys off this value to determine if seek() can be
called for all scenarios.
ZstdDecompressionReader.seek() now raises OSError instead of
ValueError when the seek cannot be fulfilled.
ZstdDecompressionReader.readline() and
ZstdDecompressionReader.readlines() now accept an integer argument.
This makes them conform with the IO interface. The methods still raise
ZstdCompressionReader.__enter__ and ZstdDecompressionReader.__enter__
now raise ValueError if the instance was already closed.
The deprecated overlap_size_log attribute on ZstdCompressionParameters
instances has been removed. The overlap_log attribute should be used
The deprecated overlap_size_log argument to ZstdCompressionParameters
has been removed. The overlap_log argument should be used instead.
The deprecated ldm_hash_every_log attribute on
ZstdCompressionParameters instances has been removed. The
ldm_hash_rate_log attribute should be used instead.
The deprecated ldm_hash_every_log argument to
ZstdCompressionParameters has been removed. The ldm_hash_rate_log
argument should be used instead.
The deprecated CompressionParameters type alias to
ZstdCompressionParamaters has been removed. Use
The deprecated aliases ZstdCompressor.read_from() and
ZstdDecompressor.read_from() have been removed. Use the corresponding
read_to_iter() methods instead.
The deprecated aliases ZstdCompressor.write_to() and
ZstdDecompressor.write_to() have been removed. Use the corresponding
stream_writer() methods instead.
ZstdCompressor.copy_stream(), ZstdCompressorIterator.__next__(),
and ZstdDecompressor.copy_stream() now raise the original exception
on error calling the source stream's read() instead of raising
ZstdError. This only affects the C backend.
ZstdDecompressionObj.flush() now returns bytes instead of
None. This makes it behave more similarly to flush() methods
for similar types in the Python standard library.
ZstdCompressionWriter.__exit__() now always calls close().
Previously, close() would not be called if the context manager
raised an exception. The old behavior was inconsistent with other
stream types in this package and with the behavior of Python's
standard library IO types.
Distribution metadata no longer lists cffi as an install_requires
except when running on PyPy. Instead, cffi is listed as an
ZstdCompressor.stream_reader() and ZstdDecompressor.stream_reader()
now default to closing the source stream when the instance is itself
closed. To change this behavior, pass closefd=False.
The CFFI backend now defines
ZstdCompressor.multi_compress_to_buffer() and
ZstdDecompressor.multi_decompress_to_buffer(). However, they
raise NotImplementedError, as they are not yet implemented.
The CFFI backend now exposes the types ZstdCompressionChunker,
ZstdCompressionObj, ZstdCompressionReader,
ZstdCompressionWriter, ZstdDecompressionObj,
ZstdDecompressionReader, and ZstdDecompressionWriter as
symbols on the zstandard module.
The CFFI backend now exposes the types BufferSegment,
BufferSegments, BufferWithSegments, and
BufferWithSegmentsCollection. However, they are not implemented.
ZstdCompressionWriter.flush() now calls flush() on the inner stream
if such a method exists. However, when close() itself calls
self.flush(), flush() is not called on the inner stream.
ZstdDecompressionWriter.close() no longer calls flush() on
the inner stream. However, ZstdDecompressionWriter.flush() still
calls flush() on the inner stream.
ZstdCompressor.stream_writer() and ZstdDecompressor.stream_writer()
now have their write_return_read argument default to True.
This brings the behavior of write() in compliance with the
io.RawIOBase interface by default. The argument may be removed
in a future release.
ZstdCompressionParameters no longer exposes a compression_strategy
property. Its constructor no longer accepts a compression_strategy
argument. Use the strategy property/argument instead.

Bug Fixes

Fix a memory leak in stream_reader decompressor when reader is closed
before reading everything. (Patch by Pierre Fersing.)
The C backend now properly checks for errors after calling IO methods
on inner streams in various methods. ZstdCompressionWriter.write()
now catches exceptions when calling the inner stream's write().
ZstdCompressionWriter.flush() on inner stream's write().
ZstdCompressor.copy_stream() on dest stream's write().
ZstdDecompressionWriter.write() on inner stream's write().
ZstdDecompressor.copy_stream() on dest stream's write().


Bundled zstandard library upgraded from 1.4.5 to 1.4.8.
The bundled zstandard library is now using the single C source file
distribution. The 2 main header files are still present, as these are
needed by CFFI to generate the CFFI bindings.
PyBuffer instances are no longer checked to be C contiguous and
have a single dimension. The former was redundant with what
PyArg_ParseTuple() already did and the latter is not necessary
in practice because very few extension modules create buffers with
more than 1 dimension.
Added Python typing stub file for the zstandard module.
The make_cffi.py script should now respect the CC environment
variable for locating the compiler.
CI now properly uses the cffi backend when running all tests.
train_dictionary() has been rewritten to use the fastcover APIs
and to consistently call ZDICT_optimizeTrainFromBuffer_fastCover()
instead of different C APIs depending on what arguments were passed.
The function also now accepts arguments f, split_point, and
accel, which are parameters unique to fastcover.
CI now tests and builds wheels for Python 3.9.
zstd.c file renamed to c-ext/backend_c.c.
All built/installed Python modules are now in the zstandard
package. Previously, there were modules in other packages.
C source code is now automatically formatted with clang-format.
ZstdCompressor.stream_writer(), ZstdCompressor.stream_reader(),
ZstdDecompressor.stream_writer(), and
ZstdDecompressor.stream_reader() now accept a closefd
argument to control whether the underlying stream should be closed
when the ZstdCompressionWriter, ZstdCompressReader,
ZstdDecompressionWriter, or ZstdDecompressionReader is closed.
There is now a zstandard.open() function for returning a file
object with zstd (de)compression.
The zstandard module now exposes a backend_features
attribute containing a set of strings denoting optional features
present in that backend. This can be used to sniff feature support
by performing a string lookup instead of sniffing for API presence
or behavior.
Python docstrings have been moved from the C backend to the CFFI
backend. Sphinx docs have been updated to generate API documentation
via the CFFI backend. Documentation for Python APIs is now fully
defined via Python docstrings instead of spread across Sphinx ReST
files and source code.
ZstdCompressionParameters now exposes a strategy property.
There are now compress() and decompress() convenience functions
on the zstandard module. These are simply wrappers around the
corresponding APIs on ZstdCompressor and ZstdDecompressor.
   2020-12-06 11:55:42 by Adam Ciarcinski | Files touched by this commit (2) | Package updated
Log message:
py-zstandard: updated to 0.14.1


* Python 3.9 wheels are now provided.
   2020-10-06 12:51:22 by Thomas Klausner | Files touched by this commit (13)
Log message:
*: use py-hypothesis via versioned_dependencies.mk
   2020-06-14 15:53:06 by Adam Ciarcinski | Files touched by this commit (3) | Package updated
Log message:
py-zstandard: updated to 0.14.0

Backwards Compatibility Notes
* This will likely be the final version supporting Python 2.7. Future
  releases will likely only work on Python 3.5+.
* There is a significant possibility that future versions will use
  Rust - instead of C - for compiled code.

Bug Fixes
* Some internal fields of C structs are now explicitly initialized.
* The ``make_cffi.py`` script used to build the CFFI bindings now
  calls ``distutils.sysconfig.customize_compiler()`` so compiler
  customizations (such as honoring the ``CC`` environment variable)
  are performed.
* The ``make_cffi.py`` script now sets ``LC_ALL=C`` when invoking
  the preprocessor in an attempt to normalize output to ASCII.

* Bundled zstandard library upgraded from 1.4.4 to 1.4.5.
* ``setup.py`` is now executable.
* Python code reformatted with black using 80 character line lengths.
   2019-12-29 17:53:13 by Adam Ciarcinski | Files touched by this commit (3) | Package updated
Log message:
py-zstandard: updated to 0.13.0

Changes 0.13.0:
* ``pytest-xdist`` ``pytest`` extension is now installed so tests can be
  run in parallel.
* CI now builds ``manylinux2010`` and ``manylinux2014`` binary wheels
  instead of a mix of ``manylinux2010`` and ``manylinux1``.
* Official support for Python 3.8 has been added.
* Bundled zstandard library upgraded from 1.4.3 to 1.4.4.
* Python code has been reformatted with black.
   2019-12-17 11:50:41 by Leonardo Taccari | Files touched by this commit (1)
Log message:
py-zstandard: Use bundled zstd because not adapted to zstd-1.4.4

macros in 1.4.4 needs a `format' argument but py-zstandard is not
adapted to 1.4.4 yet.

   2019-09-19 18:50:58 by Adam Ciarcinski | Files touched by this commit (6) | Package updated
Log message:
py-zstandard: updated to 0.12.0


Backwards Compatibility Notes
* Support for Python 3.4 has been dropped since Python 3.4 is no longer
  a supported Python version upstream. (But it will likely continue to
  work until Python 2.7 support is dropped and we port to Python 3.5+

Bug Fixes
* Fix ``ZstdDecompressor.__init__`` on 64-bit big-endian systems.
* Fix memory leak in ``ZstdDecompressionReader.seek()``.

* CI transitioned to Azure Pipelines (from AppVeyor and Travis CI).
* Switched to ``pytest`` for running tests (from ``nose``).
* Bundled zstandard library upgraded from 1.3.8 to 1.4.3.