./devel/librdkafka, Apache Kafka C/C++ library

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 1.3.0nb2, Package name: librdkafka-1.3.0nb2, Maintainer: pkgsrc-users

librdkafka is a C library implementation of the Apache Kafka protocol,
containing both Producer and Consumer support.


Required to run:
[security/openssl] [archivers/lz4] [archivers/zstd]

Required to build:
[pkgtools/cwrappers] [lang/python37]

Master sites:

Filesize: 2565.457 KB

Version history: (Expand)


CVS history: (Expand)


   2023-10-25 00:11:51 by Thomas Klausner | Files touched by this commit (2298)
Log message:
*: bump for openssl 3
   2023-08-14 07:25:36 by Thomas Klausner | Files touched by this commit (1247)
Log message:
*: recursive bump for Python 3.11 as new default
   2023-08-11 11:05:24 by Nia Alarie | Files touched by this commit (1)
Log message:
librdkafka: Eliminate -Werror usage
   2021-10-26 12:20:11 by Nia Alarie | Files touched by this commit (3016)
Log message:
archivers: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes

Could not be committed due to merge conflict:
devel/py-traitlets/distinfo

The following distfiles were unfetchable (note: some may be only fetched
conditionally):

./devel/pvs/distinfo pvs-3.2-solaris.tgz
./devel/eclipse/distinfo eclipse-sourceBuild-srcIncluded-3.0.1.zip
   2021-10-07 15:44:44 by Nia Alarie | Files touched by this commit (3017)
Log message:
devel: Remove SHA1 hashes for distfiles
   2020-05-27 21:37:44 by Thomas Klausner | Files touched by this commit (60)
Log message:
*: reset MAINTAINER for fhajny on his request
   2020-02-01 21:49:43 by Adam Ciarcinski | Files touched by this commit (3) | Package updated
Log message:
librdkafka: updated to 1.3.0

librdkafka v1.3.0 release
This is a feature release adding support for KIP-392 Fetch from follower, \ 
allowing a consumer to fetch messages from the closest replica to increase \ 
throughput and reduce cost.

Features
KIP-392 - Fetch messages from closest replica / follower (by @mhowlett)
Added experimental (subject to change or removal) mock broker to make \ 
application and librdkafka development testing easier.

Fixes
Fix consumer_lag in stats when consuming from broker versions <0.11.0.0 \ 
(regression in librdkafka v1.2.0).

librdkafka v1.2.2 release
v1.2.2 fixes the producer performance regression introduced in v1.2.1 which may \ 
affect high-throughput producer applications.

Fixes
Fix producer insert msgq regression in v1.2.1.
Upgrade builtin lz4 to 1.9.2.
Don't trigger error when broker hostname changes.
Less strict message.max.bytes check for individual messages.
Don't call timespec_get() on OSX (since it was removed in recent XCode) by \ 
@maparent .
configure: add --runstatedir for compatibility with autoconf.
LZ4 is available from ProduceRequest 0, not 3.
Address 12 code issues identified by Coverity static code analysis.

Enhancements
Add warnings for inconsistent security configuration.
Optimizations to hdr histogram (stats) rollover.
Reorganized examples and added a cleaner consumer example, added minimal C++ \ 
producer example.
Print compression type per message-set when debug=msg

librdkafka v1.2.1 release
Warning: v1.2.1 has a producer performance regression which may affect \ 
high-throughput producer applications. We recommend such users to stay on \ 
v1.2.0.

v1.2.1 is a maintenance release:
Properly handle new Kafka-framed SASL GSSAPI frame semantics on Windows.
This bug was introduced in v1.2.0 and broke GSSAPI authentication on Windows.
Fix msgq (re)insertion code to avoid O(N^2) insert sort operations on retry
The msgq insert code now properly handles interleaved and overlapping
message range inserts, which may occur during Producer retries for
high-throughput applications.
configure: added --disable-c11threads to avoid using libc-provided C11 threads.
configure: added more autoconf compatibility options to ignore

librdkafka v1.2.0 release

WARNING: There is an issue with SASL GSSAPI authentication on Windows with this \ 
release. Upgrade directly to v1.2.1 which fixes the issue.

v1.2.0 is a feature release making the consumer transaction aware.

Transaction aware consumer (isolation.level=read_committed) implemented by @mhowlett.
Sub-millisecond buffering (linger.ms) on the producer.
Improved authentication errors (KIP-152)

Consumer-side transaction support
This release adds consumer-side support for transactions.
In previous releases, the consumer always delivered all messages to the \ 
application, even those in aborted or not yet committed transactions. In this \ 
release, the consumer will by default skip messages in aborted transactions.
This is controlled through the new isolation.level configuration property which
defaults to read_committed (only read committed messages, filter out aborted and \ 
not-yet committed transactions), to consume all messages, including for aborted \ 
transactions, you may set this property to read_uncommitted to get the behaviour \ 
of previous releases.
For consumers in read_committed mode, the end of a partition is now defined to \ 
be the offset of the last message of a successfully committed transaction \ 
(referred to as the 'Last Stable Offset').
For non-transactional messages there is no change from previous releases, they \ 
will always be read, but a consumer will not advance into a not yet committed \ 
transaction on the partition.

Upgrade considerations
linger.ms default was changed from 0 to 0.5 ms to promote some level of batching \ 
even with default settings.

New configuration properties
Consumer property isolation.level=read_committed ensures the consumer will only \ 
read messages from successfully committed producer transactions. Default is \ 
read_committed. To get the previous behaviour, set the property to \ 
read_uncommitted, which will read all messages produced to a topic, regardless \ 
if the message was part of an aborted or not yet committed transaction.
Enhancements
Offset commit metadata (arbitrary application-specified data) is now returned by \ 
rd_kafka_committed() and rd_kafka_offsets_for_times()
C++: Added Conf::c_ptr*() to retrieve the underlying C config object.
Added on_thread_start() and on_thread_exit() interceptors.
Increase queue.buffering.max.kbytes max to INT_MAX.
Optimize varint decoding, increasing consume performance by ~15%.

Fixes
General:
Rate limit IO-based queue wakeups to linger.ms, this reduces CPU load and lock \ 
contention for high throughput producer applications.
Reduce memory allocations done by rd_kafka_topic_partition_list_new().
Fix socket recv error handling on MSVC (by Jinsu Lee).
Avoid 1s stalls in some scenarios when broker wakeup-fd is triggered.
SSL: Use only hostname (not port) when valid broker hostname (by Hunter Jacksson)
SSL: Ignore OpenSSL cert verification results if \ 
enable.ssl.certificate.verification=false
rdkafka_example_cpp: fix metadata listing mode (@njzcx)
SASL Kerberos/GSSAPI: don't treat kinit ECHILD errors as errors
Fix compare overflows
configure: Add option to disable automagic dependency on zstd (by Thomas Deutschmann)
Documentation updates and fixes by Cedric Cellier and @ngrandem
Set thread name on MacOS X (by Nikhil Benesch)
C++: Fix memory leak in Headers (by Vladimir Sakharuk)
Fix UBSan (undefined behaviour errors)
CONFIGURATION.md: escape || inside markdown table
Refresh broker list metadata even if no topics to refresh

Consumer:
Make rd_kafka_pause|resume_partitions() synchronous, making sure that a \ 
subsequent consumer_poll() will not return messages for the paused partitions.
Fix incorrect toppar destroy in OffsetRequest
Fix message version 1 offset calculation
Defer commit in transport error to avoid consumer_close hang.

Producer:
Messages were not timed out for leader-less partitions.
Improve message timeout granularity to millisecond precision (the smallest \ 
ffective message timeout will still be 1000ms).
message.timeout.ms=0 is now accepted even if linger.ms > 0 (by Jeff Snyder)
Don't track max.poll.interval.ms unless in Consumer mode, this saves quite a few \ 
memory barries for high-performance Producers.
Optimization: avoid atomic fatal error code check when idempotence is disabled.
   2020-01-18 22:51:16 by Jonathan Perkin | Files touched by this commit (1836)
Log message:
*: Recursive revision bump for openssl 1.1.1.