./devel/librdkafka, Apache Kafka C/C++ library

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 0.11.4, Package name: librdkafka-0.11.4, Maintainer: fhajny

librdkafka is a C library implementation of the Apache Kafka protocol,
containing both Producer and Consumer support.

Required to run:

Required to build:
[lang/python27] [pkgtools/cwrappers]

Master sites:

SHA1: 00e5b9c610697a0f32910fd1ef19e9b2c684824e
RMD160: ce9a8261a874183b61b56df3aeecd163567f679f
Filesize: 1855.583 KB

Version history: (Expand)

CVS history: (Expand)

   2018-12-15 22:12:25 by Thomas Klausner | Files touched by this commit (67) | Package updated
Log message:
*: update email for fhajny
   2018-04-05 11:01:16 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
devel/librdkafka: Update to 0.11.4.

Default changes

- socket.max.fails changed to 1 to provide same functionality (fail
  request immediately on error) now when retries are working properly
- fetch.max.bytes (new config property) is automatically adjusted to
  be >= message.max.bytes, and receive.message.max.bytes is
  automatically adjusted to be > fetch.max.bytes.

New features

- Message Headers support
- Java-compatible Murmur2 partitioners
- Add PKCS#12 Keystore support - ssl.keystore.location

Noteworthy fixes

- Formalise and fix Producer retries and retry-ordering
- Ordering is now retained despite retries if max.in.flight=1.
- Behaviour is now documented
- Fix timeouts for retried requests and properly handle retries for
  all request types
- Add and use fetch.max.bytes to limit total Fetch response size.
  Fixes "Invalid response size" issues.


- Added sasl.mechanism and compression.type configuration property
  aliases for conformance with Java client.
- Improved Producer performance
- C++: add c_ptr() to Handle,Topic,Message classes to expose
  underlying librdkafka object
- Honour per-message partition in produce_batch() if MSG_F_PARTITION
- Added on_request_sent() interceptor
- Added experimental flexible producer queuing.strategy=fifo|lifo
- Broker address DNS record round-robin: try to maintain round-robin
  position across resolve calls.
- Set system thread name for internal librdkafka threads
- Added more concise and user-friendly 'consumer' debug context
- Add partitioner (string) topic configuration property to set the
  builtin partitioners
- Generate rdkafka-static.pc (pkg-config) for static linking


- Fix producer memory leak on <0.11 brokers when compressed messageset
  is below copy threshold
- CRC32C - fix unaligned access on ARM
- Fix read after free in buf_write_seek
- Fix broker wake up
- Fix consumer hang when rebalancing during commit
- CMake fixes for Windows
- LeaveGroup was not sent on close when doing final offset commits
- Fix for consumer slowdown/stall on compacted topics where actual
  last offset < MsgSet.LastOffset
- Fix global->topic conf fallthru in C++ API
- Fix infinite loop on LeaveGroup failure
- Fix possible crash on OffsetFetch retry
- Incorporate compressed message count when deciding on fetch backoff
- Fix debug-only crash on Solaris (%s NULL)
- Drain broker ops queue on termination to avoid hang
- cmake: Allow build static library
- Don't store invalid offset as next one when pausing
- use #if instead of #ifdef / defined() for atomics
- fixed .lib paths in nuget packaging
- Fixes strerror_r crash on alpine
- Allow arbitrary lengthed (>255) SASL PLAIN user/pass
- Trigger ApiVersionRequest on reconnect if broker.version.fallback
  supports it
- Read Fetch MsgAttributes as int8
- Portability: stop using typeof in rdkafka_transport.c
- Portability: replace use of #pragma once with header guards
- mklove: add LIBS in reverse order to maintain dependency order
- Fix build when python is not available #1358
   2017-12-09 17:57:04 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
Update devel/librdkafka to 0.11.3.

Default changes
- Change default queue.buffering.max.kbytes and
  queued.max.message.kbytes to 1GB
- win32: Use sasl.kerberos.service.name for broker principal, not

- Default producer message offsets to OFFSET_INVALID rather than 0
- new nuget package layout + debian9 librdkafka build
- Allow for calling rd_kafka_queue_io_event_enable() from the C++
- rdkafka_performance: allow testing latency with different size

- Improved stability on termination
- offsets_for_times() return ERR__TIMED_OUT if brokers did not respond
  in time
- Let list_groups() return ERR__PARTIAL with a partial group list
- Properly handle infinite
- Fix offsets_store() return value when at least one valid partition
- portability: rdendian: add le64toh() alias for older glibc
- Add MIPS build and fix CRC32 to work on big endian CPUs
- osx: fix endian checking for software crc32c
- Fix comparison in rd_list_remove_cmp
- stop calling cnd_timedwait() with a timeout of 0h
- Fix DNS cache logic broker.address.ttl
- Fix broker thread "hang" in CONNECT state
- Reset rkb_blocking_max_ms on broker DOWN to avoid busy-loop during
- Fix memory leak when producev() fails
- Raise cmake minimum version to 3.2
- Do not assume LZ4 worst
- Fix ALL_BROKERS_DOWN re-generation
- rdkafka-performance: busy wait to wait short periods of time
   2017-10-18 12:00:04 by Filip Hajny | Files touched by this commit (2) | Package updated
Log message:
Update devel/librdkafka to 0.11.1.

Noteworthy critical bug fixes
- Fix OpenSSL instability on Windows (fix thread id callback) - the
  bug resulted in SSL connections being torn down for no apparent
- Fetch response fix: read all MessageSets, not just the first one.

- Add api.version.request.timeout.ms
- Point users to documentation when attempting to use Java security

- Adjust log level for partial message reads when debug is enabled
- Allow app metadata() requests regardless of outstanding metadata
- Proper size calculation from flags2str to include nul
- Thread-safe rd_strerror()
- Proper re-sends of ProduceRequests in-transit on connection down
- Producer: invalid offsets reported back when
- Consumer: Message Null Key/Value were messed up
- Fix metadata querying for topics with LEADER_UNAVAIL set
- Treat request __TIMED_OUT as a retryable error
- Let timed out ProduceRequests result in MSG_TIMED_OUT error code for
- Fix crash on leader rejoin when outstanding assignor metadata
- sasl_cyrus: Fix dangling stack ptr to sasl_callback_t
   2017-08-02 17:49:25 by Filip Hajny | Files touched by this commit (4) | Package updated
Log message:
Update devel/librdkafka to 0.11.0.

- Added support for MsgVersion v2 (message format of KIP-98) - message
  format compatible with EOS clients
- Added support for client interceptors
- Added support for dynamically loaded plugins (plugin.library.paths,
  for use with interceptors)
- Added SASL SCRAM support (KIP-84)
- Added builtin SASL PLAIN provider (for Win32, #982)

- Deprecate errno usage, use rd_kafka_last_error() instead.
- Deprecate rd_kafka_wait_destroyed().
- Implemented per-partition Fetch backoffs, previously all partitions
  for the given broker were backed off.
- Added updated Kafka protocol and error enums
- Added rd_kafka_message_latency()
- Added rd_kafka_clusterid() and rd_kafka_type()
- SSL: set default CA verify locations if ssl.ca.location is not
- C++: add yield() method
- Added support for stats as events (#1171)
- Build with system liblz4 if available, else fall back on built-in
  lz4, for improved portability.
- Use SNI when connecting through SSL (@vincentbernat)
- Improve broker thread responsiveness, decreasing internal latency
- Improve OpenSSL config error propagation (#1119)
- Prioritize all relevant user-facing ops (callbacks) over messages on
  poll queue (#1088)
- Added global->topic config fallthru: default topic config properties
- can now be set effortlessly on global config object.
- Log offset commit failures when there is no offset_commit_cb (closes
- Add CRC checking support to consumer (#1056)
- C++: Added seek() support to KafkaConsumer
- Added rd_kafka_conf_dup_filter() to selectively copy a config

- Avoid _ALIGN re-definition on BSD (#1225)
- rdkafka_performance: exit with code 1 if not all messages were
- Fix endianism issues that were causing snappy to compress
  incorrectly (#1219, @rthalley)
- Fix stability on AIX (#1211)
- Document that rd_kafka_message_errstr() must not be used on producer
- Add support for re-queuing half-processed ops to honour yield()
- Handle null Protocol in JoinGroupResponse (#1193)
- Consumer: Proper relative offset handling (#1192, @rthalley)
- OSX: silence libsasl deprecated warnings
- partition count should be per topic in offset request buffer (closes
  #1199, @matthew-d-jones)
- fix build on SmartOS (#1186 by @misterdjules)
- ERR_remove_thread_state OpenSSL version checking
- Don't emit TIMED_OUT_QUEUE for timed out messages (revert)
- producev() default partition should UA, not 0 (#1153)
- Fix SaslHandshakeRequest timeout to 10s
- SASL: fix memory leak: received SASL auth frames were not freed
- Clean up OpenSSL per-thread memory on broker thread exit
- Properly auto-set metadata.max.age.ms when
  metadata.refresh.interval.ms is disabled (closes #1149)
- Fix memory alignment issues (#1150)
- configure: auto add brew openssl pkg-config path
- Fix consumer_lag calculation (don't use cached hi_offset)
- rdkafka_example: fix message_errstr usage, not allowed on producer
- Avoid use of partially destroyed topic object (#1125)
- Improve reconnect delay handling (#1089)
- C++: fix conf->get() allocation (closes #1118)
- Use app_offset to calculate consumer_lag (closes #1112)
- Fix retrybuf memory leak on termination when broker is down
- Fix small memory leak in metadata_leader_query
- Fix use-after-free when log.queue and debug was used
- consumer_example: fix crash on -X dump (closes #841)
- Added rd_kafka_offsets_store() (KafkaConsumer::offsets_store)
  (closes #826)
- Optimize broker id lookups (closes #523)
- Don't log broker failures when an error_cb is registered (closes
- Properly log SSL connection close (closes #1081)
- Win32 SASL GSSAPI: protection level and message size were not sent
- C++: improved error reporting from Conf::set()
- Flush partition fetch buffer on seek (from decide())
- Fix recursive locking on periodic refresh of leader-less partition
  (closes #1311)
   2017-04-24 14:20:37 by Filip Hajny | Files touched by this commit (8)
Log message:
* Use bash for CONFIG_SHELL
* Fix build on NetBSD
* Uses python to build
* Enable tests
   2017-04-20 10:04:11 by Filip Hajny | Files touched by this commit (1)
Log message:
Add librdkafka bl3
   2017-04-20 10:01:19 by Filip Hajny | Files touched by this commit (5)
Log message:
Import librdkafka 0.9.5 as devel/librdkafka.

librdkafka is a C library implementation of the Apache Kafka protocol,
containing both Producer and Consumer support.