Log message:
archivers: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes

Could not be committed due to merge conflict:
devel/py-traitlets/distinfo

The following distfiles were unfetchable (note: some may be only fetched
conditionally):

./devel/pvs/distinfo pvs-3.2-solaris.tgz
./devel/eclipse/distinfo eclipse-sourceBuild-srcIncluded-3.0.1.zip

Log message:
devel: Remove SHA1 hashes for distfiles

Log message:
*: reset MAINTAINER for fhajny on his request

Log message:
all: migrate several HOMEPAGEs to https

pkglint --only "https instead of http" -r -F

With manual adjustments afterwards since pkglint 19.4.4 fixed a few
indentations in unrelated lines.

This mainly affects projects hosted at SourceForce, as well as
freedesktop.org, CTAN and GNU.

Log message:
devel: align variable assignments

pkglint -Wall -F --only aligned --only indent -r

No manual corrections.

Log message:
*: update email for fhajny

Log message:
devel/kafka: Update to 1.1.0.

New Feature

- automatic migration of log dirs to new locations
- KIP-145 - Expose Record Headers in Kafka Connect
- Add the AdminClient in Streams' KafkaClientSupplier
- Support dynamic updates of frequently updated broker configs

Improvement

- KafkaConnect should support regular expression for topics
- Move kafka-streams test fixtures into a published package
- SSL support for Connect REST API
- Grow default heap settings for distributed Connect from 256M to 1G
- Enable access to key in ValueTransformer
- Add "getAllKeys" API for querying windowed KTable stores
- Unify StreamsKafkaClient instances
- Revisit Streams DSL JavaDocs
- Extend Consumer Group Reset Offset tool for Stream Applications
- KIP-175: ConsumerGroupCommand no longer shows output for consumer
  groups which have not committed offsets
- Add a broker metric specifying the number of consumer group
  rebalances in progress
- Use Jackson for serialising to JSON
- KafkaShortnamer should allow for case-insensitive matches
- Improve Util classes
- Gradle 3.0+ is needed on the build
- Adding records deletion operation to the new Admin Client API
- Kafka metrics templates used in document generation should maintain
  order of tags
- Provide for custom error handling when Kafka Streams fails to
  produce
- Make Repartition Topics Transient
- Connect Schema comparison is slow for large schemas
- Add a Validator for NonNull configurations and remove redundant null
  checks on lists
- Have State Stores Restore Before Initializing Toplogy
- Optimize condition in if statement to reduce the number of
  comparisons
- Removed unnecessary null check
- Introduce Incremental FetchRequests to Increase Partition
  Scalability
- SSLTransportLayer should keep reading from socket until either the
  buffer is full or the socket has no more data
- Improve KTable Source state store auto-generated names
- Extend consumer offset reset tool to support deletion (KIP-229)
- Expose Kafka cluster ID in Connect REST API
- Maven artifact for kafka should not depend on log4j
- ConsumerGroupCommand should use the new consumer to query the log
  end offsets.
- Change LogSegment.delete to deleteIfExists and harden log recovery
- Make ProducerConfig and ConsumerConfig constructors public
- Improve synchronization in CachingKeyValueStore methods
- Improve Kafka GZip compression performance
- Improve JavaDoc of SourceTask#poll() to discourage indefinite
  blocking
- Avoid creating dummy checkpoint files with no state stores
- Change log level from ERROR to WARN for not leader for this
  partition exception
- Delay initiating the txn on producers until initializeTopology with
  EOS turned on

Bug

- change log4j to slf4j
- Use RollingFileAppender by default in log4j.properties
- Cached zkVersion not equal to that in zookeeper, broker not
  recovering.
- FileRecords.read doesn't handle size > sizeInBytes when start is not
  zero
- a soft failure in controller may leave a topic partition in an
  inconsistent state
- Cannot truncate to a negative offset (-1) exception at broker
  startup
- automated leader rebalance causes replication downtime for clusters
  with too many partitions
- kafka-run-class has potential to add a leading colon to classpath
- QueryableStateIntegrationTest.concurrentAccess is failing
  occasionally in jenkins builds
- FileStreamSource Connector not working for large files (~ 1GB)
- KeyValueIterator returns null values
- KafkaProducer is not joining its IO thread properly
- Kafka connect: error with special characters in connector name
- Replace StreamsKafkaClient with AdminClient in Kafka Streams
- LogCleaner#cleanSegments should not ignore failures to delete files
- Connect Rest API allows creating connectors with an empty name -
  KIP-212
- KerberosLogin#login should probably be synchronized
- Support replicas movement between log directories (KIP-113)
- Consumer ListOffsets request can starve group heartbeats
- Struct.put() should include the field name if validation fails
- Clarify handling of connector name in config
- Allow user to specify relative path as log directory
- KafkaConsumer should validate topics/TopicPartitions on
  subscribe/assign
- Controller should only update reassignment znode if there is change
  in the reassignment data
- records.lag should use tags for topic and partition rather than
  using metric name.
- KafkaProducer should not wrap InterruptedException in close() with
  KafkaException
- Connect classloader isolation may be broken for JDBC drivers
- JsonConverter generates "Mismatching schema" DataException
- NoSuchElementException in markErrorMeter during
  TransactionsBounceTest
- Make KafkaFuture.Function java 8 lambda compatible
- ThreadCache#sizeBytes() should check overflow
- KafkaFuture timeout fails to fire if a narrow race condition is hit
- DeleteRecordsRequest to a non-leader
- ReplicaFetcherThread should close the ReplicaFetcherBlockingSend
  earlier on shutdown
- NoSuchMethodError when creating ProducerRecord in upgrade system
  tests
- Running tools on Windows fail due to typo in JVM config
- Streams metrics tagged incorrectly
- ducker-ak: add ipaddress and enum34 dependencies to docker image
- Kafka cannot recover after an unclean shutdown on Windows
- Scanning plugin.path needs to support relative symlinks
- Reconnecting to broker does not exponentially backoff
- TaskManager should be type aware
- Major performance issue due to excessive logging during leader
  election
- RecordCollectorImpl should not retry sending
- Restore and global consumer should not use auto.offset.reset
- Global Consumer should handle TimeoutException
- Reduce rebalance time by not checking if created topics are
  available
- VerifiableConsumer with --max-messages doesn't exit
- Transaction markers are sometimes discarded if txns complete
  concurrently
- Simplify StreamsBuilder#addGlobalStore
- JmxReporter can't handle windows style directory paths
- CONSUMER-ID and HOST values are concatenated if the CONSUMER-ID is >
  50 chars
- ClientQuotaManager threads prevent shutdown when encountering an
  error loading logs
- Streams configuration requires consumer. and producer. in order to
  be read
- Timestamp on streams directory contains a colon, which is an illegal
  character
- Add methods in Options classes to keep binary compatibility with
  0.11
- RecordQueue.clear() does not clear MinTimestampTracker's maintained
  list
- Selector memory leak with high likelihood of OOM in case of down
  conversion
- GlobalKTable never finishes restoring when consuming transactional
  messages
- Server crash while deleting segments
- IllegalArgumentException if 1.0.0 is used for
  inter.broker.protocol.version or log.message.format.version
- Using standby replicas with an in memory state store causes Streams
  to crash
- Issues with protocol version when applying a rolling upgrade to
  1.0.0
- Fix system test dependency issues
- Kafka Connect requires permission to create internal topics even if
  they exist
- A metric named 'XX' already exists, can't register another one.
- Improve sink connector topic regex validation
- Flaky Unit test:
  KStreamKTableJoinIntegrationTest.shouldCountClicksPerRegionWithNonZeroByteCache
- Make KafkaStreams.cleanup() clean global state directory
- AbstractCoordinator not clearly handles NULL Exception
- Request logging throws exception if acks=0
- GlobalKTable missing #queryableStoreName()
- KTable state restore fails after rebalance
- Make loadClass thread-safe for class loaders of Connect plugins
- System Test failed: ConnectRestApiTest
- Broken symlink interrupts scanning the plugin path
- NetworkClient should not return internal failed api version
  responses from poll
- Transient failure in
  NetworkClientTest.testConnectionDelayDisconnected
- Line numbers on log messages are incorrect
- Topic can not be recreated after it is deleted
- mBeanName should be removed before returning from
  JmxReporter#removeAttribute()
- Kafka Core should have explicit SLF4J API dependency
- StreamsResetter should return non-zero return code on error
- kafka-acls regression for comma characters (and maybe other
  characters as well)
- Error deleting log for topic, all log dirs failed.
- punctuate with WALL_CLOCK_TIME triggered immediately
- Exclude node groups belonging to global stores in
  InternalTopologyBuilder#makeNodeGroups
- Transient failure in
  \ 
kafka.api.AdminClientIntegrationTest.testLogStartOffsetCheckpointkafka.api.Admin \ 
ClientIntegrationTest.testAlterReplicaLogDirs
- NetworkClient.inFlightRequestCount() is not thread safe, causing
  ConcurrentModificationExceptions when sensors are read
- ConcurrentModificationException during streams state restoration
- Update KStream JavaDoc with regard to KIP-182
- RocksDB segments not removed when store is closed causes
  re-initialization to fail
- auto commit not work since coordinatorUnknown() is always true.
- Fix StateRestoreListener To Use Correct Batch Ending Offset
- NullPointerException on KStream-GlobalKTable leftJoin when
  KeyValueMapper returns null
- StreamThread.shutdown doesn't clean up completely when called before
  StreamThread.start
- Update ZooKeeper to 3.4.11, Gradle and other minor updates
- output from ensure copartitioning is not used for Cluster metadata,
  resulting in partitions without tasks working on them
- Consumer should not block setting initial positions of unavailable
  partitions
- Non-aggregation KTable generation operator does not construct value
  getter correctly
- AdminClient should handle empty or null topic names better
- When enable trace level log in mirror maker, it will throw null
  pointer exception and the mirror maker will shutdown
- Simplify KStreamReduce
- Base64URL encoding under JRE 1.7 is broken due to incorrect padding
  assumption
- ChangeLoggingKeyValueBytesStore.all() returns null
- Fetcher.retrieveOffsetsByTimes() should add all the topics to the
  metadata refresh topics set.
- LogSemgent.truncateTo() should always resize the index file
- Connect: Plugin scan is very slow
- Connect: Some per-task-metrics not working
- Connect header parser incorrectly parses arrays
- Java Producer: Excessive memory usage with compression enabled
- New Connect header support doesn't define 'converter.type' property
  correctly
- ZooKeeperClient holds a lock while waiting for responses, blocking
  shutdown
- Transient failure in
  DynamicBrokerReconfigurationTest.testThreadPoolResize
- Broker leaks memory and file descriptors after sudden client
  disconnects
- Delegation token internals should not impact public interfaces
- Streams quickstart pom.xml is missing versions for a bunch of
  plugins
- Deadlock while processing Controller Events
- Broker doesn't reject Produce request with inconsistent state
- LogCleanerManager.doneDeleting() should check the partition state
  before deleting the in progress partition
- KafkaController.brokerInfo not updated on dynamic update
- Connect standalone SASL file source and sink test fails without
  explanation
- Connect distributed and standalone worker 'main()' methods should
  catch and log all exceptions
- Consumer bytes-fetched and records-fetched metrics are not
  aggregated correctly
- Coordinator disconnect in heartbeat thread can cause commitSync to
  block indefinitely
- Regression in consumer auto-commit backoff behavior
- GroupMetadataManager.loadGroupsAndOffsets decompresses record batch
  needlessly
- log segment deletion could cause a disk to be marked offline
  incorrectly
- Delayed operations may not be completed when there is lock
  contention
- Expression for GlobalKTable is not correct
- System tests do not handle ZK chroot properly with SCRAM
- Fix config initialization in DynamicBrokerConfig
- ReplicaFetcher crashes with "Attempted to complete a transaction
  which was not started"

Test

- Add concurrent tests to exercise all paths in group/transaction
  managers
- Add unit tests for ClusterConnectionStates
- Only delete reassign_partitions znode after reassignment is complete
- KafkaStreamsTest fails in trunk
- SelectorTest may fail with ConcurrentModificationException

Sub-task

- Add capability to create delegation token
- Add authentication based on delegation token.
- Add capability to renew/expire delegation tokens.
- always leave the last surviving member of the ISR in ZK
- handle ZK session expiration properly when a new session can't be
  established
- Streams should not re-throw if suspending/closing tasks fails
- Use async ZookeeperClient in Controller
- Use async ZookeeperClient in SimpleAclAuthorizer
- Use async ZookeeperClient for DynamicConfigManager
- Use async ZookeeperClient for Admin operations
- Trogdor should handle injecting disk faults
- Add process stop faults, round trip workload, partitioned
  produce-consume test
- add the notion of max inflight requests to async ZookeeperClient
- Add workload generation capabilities to Trogdor
- Add ZooKeeperRequestLatencyMs to KafkaZkClient
- Use ZookeeperClient in LogManager
- Use ZookeeperClient in GroupCoordinator and TransactionCoordinator
- Use ZookeeperClient in KafkaApis
- Use ZookeeperClient in ReplicaManager and Partition
- Tests for KafkaZkClient
- Transient failure in
  \ 
kafka.api.SaslScramSslEndToEndAuthorizationTest.testTwoConsumersWithDifferentSas \ 
lCredentials
- minimize the number of triggers enqueuing
  PreferredReplicaLeaderElection events
- Enable dynamic reconfiguration of SSL keystores
- Enable resizing various broker thread pools
- Enable reconfiguration of metrics reporters and their custom configs
- Enable dynamic reconfiguration of log cleaners
- Enable reconfiguration of default topic configs used by brokers
- Enable reconfiguration of listeners and security configs
- Add ProduceBench to Trogdor
- move ZK metrics in KafkaHealthCheck to ZookeeperClient
- Add documentation for delegation token authentication mechanism
- Document dynamic config update
- Extend ConfigCommand to update broker config using new AdminClient
- Add test to verify markPartitionsForTruncation after fetcher thread
  pool resize

Log message:
devel/kafka: Update to 1.0.1.

Improvement
- Kafka metrics templates used in document generation should maintain
  order of tags
- Consolidate MockTime implementations between connect and clients
- Fix repeated words words in JavaDoc and comments.
- Cache lastEntry in TimeIndex to avoid unnecessary disk access
- AbstractIndex should cache index file to avoid unnecessary disk
  access during resize()
- Have State Stores Restore Before Initializing Toplogy
- SSLTransportLayer should keep reading from socket until either the
  buffer is full or the socket has no more data
- Improve KTable Source state store auto-generated names

Bug
- QueryableStateIntegrationTest.concurrentAccess is failing
  occasionally in jenkins builds
- KafkaProducer is not joining its IO thread properly
- Kafka connect: error with special characters in connector name
- Streams metrics tagged incorrectly
- ClassCastException in BigQuery connector
- ClientQuotaManager threads prevent shutdown when encountering an
  error loading logs
- Streams configuration requires consumer. and producer. in order to
  be read
- Timestamp on streams directory contains a colon, which is an illegal
  character
- Add methods in Options classes to keep binary compatibility with
  0.11
- RecordQueue.clear() does not clear MinTimestampTracker's maintained
  list
- Selector memory leak with high likelihood of OOM in case of down
  conversion
- GlobalKTable never finishes restoring when consuming transactional
  messages
- IllegalArgumentException if 1.0.0 is used for
  inter.broker.protocol.version or log.message.format.version
- Using standby replicas with an in memory state store causes Streams
  to crash
- Issues with protocol version when applying a rolling upgrade to
  1.0.0
- A metric named 'XX' already exists, can't register another one.
- Flaky Unit test:
  KStreamKTableJoinIntegrationTest.shouldCountClicksPerRegionWithNonZeroByteCache
- AbstractCoordinator not clearly handles NULL Exception
- Request logging throws exception if acks=0
- KTable state restore fails after rebalance
- Make loadClass thread-safe for class loaders of Connect plugins
- Inconsistent protocol type for empty consumer groups
- Broken symlink interrupts scanning the plugin path
- NetworkClient should not return internal failed api version
  responses from poll
- Topic can not be recreated after it is deleted
- mBeanName should be removed before returning from
  JmxReporter#removeAttribute()
- Connect: Struct equals/hashCode method should use Arrays#deep*
  methods
- kafka-acls regression for comma characters (and maybe other
  characters as well)
- punctuate with WALL_CLOCK_TIME triggered immediately
- Update KStream JavaDoc with regard to KIP-182
- StackOverflowError in kafka-coordinator-heartbeat-thread
- Fix StateRestoreListener To Use Correct Batch Ending Offset
- NullPointerException on KStream-GlobalKTable leftJoin when
  KeyValueMapper returns null
- Non-aggregation KTable generation operator does not construct value
  getter correctly
- When enable trace level log in mirror maker, it will throw null
  pointer exception and the mirror maker will shutdown
- Enforce layout of dependencies within a Connect plugin to be
  deterministic
- Connect: Some per-task-metrics not working
- Broker leaks memory and file descriptors after sudden client
  disconnects

Test
- KafkaStreamsTest fails in trunk
- SelectorTest may fail with ConcurrentModificationException

Log message:
Update devel/kafka to 1.0.0

See upgrade notes:

  http://kafka.apache.org/100/documentation.html#upgrade

New Features

- KIP-72 Allow putting a bound on memory consumed by Incoming requests
- Expose states of active tasks to public API
- Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics
- KIP-182: Reduce Streams DSL overloads and allow easier use of custom
  storage engines
- Add ChangeReplicaDirRequest and DescribeReplicaDirRequest (KIP-113)
- Add cumulative count attribute for all Kafka rate metrics
- Add new metrics to support health checks
- Handle SASL authentication failures as non-retriable exceptions in
  clients

Improvement

- Remove deprecated and slower scala JSON parser
- Add Kafka Connect metrics
- Allow setting of default topic configs via StreamsConfig
- Improve DumpLogSegments offsets-decoder output format
- Improve documentation of configuration
  listeners=PLAINTEXT://0.0.0.0:9092
- StateRestoreCallback begin and end indication
- Update the usage description of clean shutdown file.
- Add a '--group' option to the console consumer
- Support Java 9
- Details of retention.bytes property at Topic level are not clear on
  how they impact partition size
- Handle disk failure for JBOD (KIP-112)
- Improve diagnostics for SASL authentication failures
- Remove usage of java.xml.bind.* classes hidden by default in JDK9
- Make server start script work against Java 9
- Only log invalid user configs and overwrite with correct one
- Changes to punctuate semantics (KIP-138)
- Use per-task converters in Connect
- Consumer perf tool should count rebalance time separately
- Add ability to batch restore and receive restoration stats.
- KIP-162: Enable topic deletion by default
- Fix taskClass() method name in Connector and flush() signature in
  SinkTask
- Add checks in integration tests to verify that threads have been
  shutdown
- Add a new Kafka Streams example IoT oriented
- KIP-168: Add GlobalTopicCount metric per cluster
- Change Source offset commit message to info to match Sink behaviour
- Replace -XX:+DisableExplicitGC with -XX:+ExplicitGCInvokesConcurrent
  in kafka-run-class
- Streams should not suspend tasks twice
- Replace the deprecated 'ConsumerOffsetChecker' in documentation
- Consider removing date formatting from Segments class
- Formatting verifiable producer/consumer output in a similar fashion
- Streams reset tool should have same console output with or without
  dry-run
- Improve Java doc for LeaderEpochFileCache.endOffsetFor()
- Return topic authorization failed if no topic describe access
- Explain that `client.id` is just used as a prefix within Streams
- Using a logPrefix inside the StreamPartitionAssignor
- Add --custom-ducktape flag to ducker-ak
- Console Consumer overrides auto.offset.reset property when provided on
  the command line without warning about it.
- Connect REST API should include the connector type when describing a
  connector
- Test DeleteRecordsRequest in AuthorizerIntegrationTest
- KafkaConsumer.subscribe() overload that takes just Pattern without
  ConsumerRebalanceListener
- All ducktape services should store their files in subdirectories of
  /mnt
- ShellTest: add tests for attempting to run nonexistent program, error
  return
- Elevate log messages for denials to INFO in SimpleAclAuthorizer class
- Refactor Streams to use LogContext
- Refactor Producer to use LogContext
- Refactor AdminClient to use LogContext
- Refactor NetworkClient to use LogContext
- Upgrade ducktape version to 0.7.1, and use new kill_java_processes
- Implement KafkaPrincipalBuilder interface with support for SASL
  (KIP-189)
- Remove unneeded synchronized keyword in StreamThread
- Consistent logging of topic partitions
- Add AdminClient.createPartitions()
- Support unmapping of mapped/direct buffers in Java 9
- Handle SSL authentication failures as non-retriable exceptions in
  clients
- Avoid call fetchPrevious in FlushListeners
- Handle authentication failures from transactional producer and
  KafkaAdminClient
- User StoreListener not available for global stores
- Mention the need to close store iterators
- Enable custom authentication plugins to return error messages to
  clients
- There is something wrong in Kafka Streams document

Bugs

- NullPointerException in client on MetadataRequest
- The kafka-consumer-perf-test.sh script help information print useless
  parameters.
- Remove ConsumerOffsetChecker, deprecated in 0.9, in 0.11
- Make KStreamTestDriver extending from ExternalResource
- Cleanup Kafka Streams builder API
- completedReceives can contain closed channels
- KIP-163: Offset fetch and commit requests use the same permissions
- KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives
  throws exception
- Records from internal repartitioning topics should always use
  RecordMetadataTimestampExtractor
- Augment KStream.print() to allow users pass in extra parameters in the
  printed string
- Calling KafkaProducer.close() from multiple threads may cause spurious
  error
- Kafka batch files does not support path with spaces
- creating a window store with one segment throws division by zero error
- Replace pattern matching with foreach where the case None is unused
- Flaky ResetIntegrationTest
- Kafka Streams keeps restoring state after shutdown is initiated during
  startup
- streams task gets stuck after re-balance due to LockException
- StreamsResetter doesn't allow custom Consumer properties
- Producer buffer pool allocates memory inside a lock.
- ProcessorContext.appConfigs() should return parsed/validated values
- JmxReporter Fetch metrics for kafka.server should not be created when
  client quotas are not enabled
- Using connect-console-sink/source.properties raises an exception
  related to "file" property not found
- Clients get inconsistent connection states when SASL/SSL connection is
  marked CONECTED and DISCONNECTED at the same time
- LogCleaner stopped due to
  org.apache.kafka.common.errors.CorruptRecordException
- StreamsKafkaClient should not use StreamsConfig.POLL_MS_CONFIG
- Created state changelog topics not logged correctly
- ListOffsetResponse isn't logged in the right way with trace level
  enabled
- Refactor kafkatest docker support
- Rolling upgrade test for streams
- bin/kafka-consumer-groups.sh failing to query offsets
- kafka-run-class.sh doesn't check for empty $CLASSPATH
- Documentation for altering topics
- KafkaConsumer: High memory allocation rate when idle
- KafkaConsumer offsetsForTimes should include partitions in result even
  if no offset could be found
- Transformations - tranformations for value broken on tombstone events
- The LastStableOffsetLag metric should be removed when partition is
  deleted
- SchemaBuilder does not validate input.
- KafkaConsumer.commitSync throws IllegalStateException: Attempt to
  retrieve exception from future which hasn't failed
- Instable test
  QueryableStateIntegrationTest.shouldAllowToQueryAfterThreadDied
- With transformations that mutate the topic-partition committing
  offsets should to refer to the original topic-partition
- Support Power platform by updating rocksdb version
- SchemaBuilder.type(Schema.Type) should not allow null.
- Processor got uncaught exception: NullPointerException
- Illegal state in SocketServer; attempt to send with another send in
  progress
- ConsoleConsumer : --new-consumer option as deprecated
- Group loading regression causing stale metadata/offsets cache
- Streams should not abort transaction when closing zombie task
- Improve error handling in SocketServer to reduce illegal states later
- System test failure due to timeout starting Jmx tool
- KafkaApis.handleWriteTxnMarkerRequest can return
  UNSUPPORTED_FOR_MESSAGE_FORMAT error on partition emigration
- Make --new-consumer option as deprecated in all tools
- SerializationException in doSend() masks class cast exception
- ducktape kafka service: do not assume Service contains num_nodes
- Consumer poll loop over the same record after a CorruptRecordException
- Using _DUCKTAPE_OPTIONS has no effect on executing tests
- Transient test failure:
  ResetConsumerGroupOffsetTest.testResetOffsetsToZonedDateTime
- Support bulk attributes request on KafkaMbean where some attributes do
  not exist
- LogDirFailureTest system test fails
- queryable state window store range scan only returns results from one
  store
- Add logging to distinguish between internally and externally initiated
  shutdown of Kafka
- Sort processor node based on name suffix in
  TopologyDescription.toString()
- Unit test shouldTogglePrepareForBulkLoadDuringRestoreCalls fails
- Auto topic creation causes failure with older clusters
- Bulk Restore Should Handle Deletes
- [streams] 'null' values in state stores
- In Jenkins, kafka.api.SaslSslAdminClientIntegrationTest failed with
  org.apache.kafka.common.errors.TimeoutException
- Consumer should ensure consistent ordering between sync and async
  offset commits
- System tests get exception RocksDBException: db has more levels than
  options.num_levels
- Client-ids are not handled consistently by clients and broker
- KafkaAdminClient thread should be daemon
- Partition.makeLeader() should convert HW to OffsetMetadata before
  becoming the leader
- Delete topic and re-create topic immediate will delete the new topic's
  timeindex
- ShellTest.testRunProgramWithErrorReturn fails on macOS
- Synchronization issue on flush
- Reassigning a topic's partitions can adversely impact other topics
- Move merge() from StreamsBuilder to KStream
- Kafka server should halt if IBP < 1.0.0 and there is log directory
  failure
- org.apache.kafka.streams.state.internals.Segments#segments method
  returns incorrect results when segments were added out of order
- WorkerSinkTask failure exception should include cause
- StoreChangeLogReader needs to restore partitions that were added post
  initialization
- SocketServer.processNewResponses should not skip a response if
  exception is thrown
- StoreChangelogReader should be resilient to broker-side metadata not
  available
- ChangeLoggingWindowBytesStore needs to retain duplicates when writing
  to the log
- KafkaStreams state transitions not correct
- Speedup broker startup after unclean shutdown by reducing unnecessary
  snapshot files deletion
- Reset thread interrupt state in case of InterruptedException
- ReassignPartitionsCommand fails if default throttle/timeout used
- Mx4jLoader.maybeLoad should only be executed if kafka_mx4jenable is
  set to true
- Remove ZK dependency from Streams reset tool
- ReplicaFetcherThread should not die due to replica in offline log
  directory
- Expiring batches with idempotence enabled could cause data loss.
- Kafka Connect applications should log info message when starting up
- Transient failure in SslSelectorTest.testMuteOnOOM
- Controller should read the latest IsrChangeNotification znodes when
  handling IsrChangeNotification event
- CompositeReadOnlyWindowStore range fetch doesn't return all values
  when fetching with different start and end times
- Fix minor typos and errors in the Kafka Streams turotial
- Connect classloader isolation may be broken for JDBC drivers
- Failure in Connect system test: ConnectRestApiTest
- StreamBuilder#table and StreamsBuilder#globalTable should use serdes
  from Materialized
- Producer IllegalStateException due to second deallocate after aborting
  a batch
- NPE in NetworkClient
- Producer uses unsupported ProduceRequest version against older brokers
- Ineffective check of negative value in
  CompositeReadOnlyKeyValueStore#approximateNumEntries()
- Remove all broker metrics during shutdown
- Deadlock due to locking of DelayedProduce and group
- RequestChannel.sendReponse records incorrect size for NetworkSend with
  TRACE logging
- kip-120 changed internal streams topic naming
- FailOnInvalidTimestamp does not log error
- Streams State Restoration never completes when logging is disabled
- disableLogging() causes partitions to not be consumed
- Rename AlterReplicaDir to AlterReplicaDirs
- Replication Fetcher thread for a partition with no data fails to start
- Reject JoinGroup request from first member with empty protocol
  type/protocol list
- Kafka Connect: Unsanitized workerID causes exception during startup
- Transient failure:
  MemoryRecordsBuilderTest.convertToV1WithMixedV0AndV2Data
- NoSuchElementException in markErrorMeter during TransactionsBounceTest
- NPE in RecordAccumulator
- Use the idempotent producer in the reassign_partitions_test
- KafkaFuture timeout fails to fire if a narrow race condition is hit
- System test failure: LogDirFailureTest
- Integer overflow in log cleaner cleanable ratio computation
- Kafka Request Handler deadlocks and brings down the cluster.
- NoSuchMethodError when creating ProducerRecord in upgrade system tests
- Running tools on Windows fail due to typo in JVM config
- ducker-ak: add ipaddress and enum34 dependencies to docker image
- Scanning plugin.path needs to support relative symlinks
- replica dir not deleted after topic deletion
- Streams quick start crashes Java on Windows
- Reconnecting to broker does not exponentially backoff
- Major performance issue due to excessive logging during leader
  election
- Silent Data Loss in Kafka011 Transactional Producer
- Transaction markers are sometimes discarded if txns complete
  concurrently
- High memory usage on controller during partition reassignment

Task

- Update Jackson dependencies (from 2.8.5 to 2.9.x)
- Enable PowerMock tests when running on Java 9
- Upgrade RocksDB dependency to legally acceptable version

Test

- Support passing ZK chroot in system tests
- Add unit tests for ClusterConnectionStates

Sub-task

- Replace ControlledShutdown{Request,Response} with
  org.apache.kafka.common.requests equivalent
- Remove temporary mapping to deserialize functions in RequestChannel
- Task migration during rebalance callback process could lead the
  obsoleted task's IllegalStateException
- Improve test coverage of StreamsKafkaClient
- Improve test coverage of RocksDBStore
- Improve test coverage of CompositeReadOnlySessionStore
- Improve test coverage of CompositeReadOnlyKeyValueStore
- Improve test coverage InMemoryKeyValueLoggedStore
- Improve test coverage of CachingKeyValueStore
- Improve test coverage UsePreviousTimeOnInvalidTimestamp
- Optimize RocksDb config for fast recovery/bulk load
- KeyValueStore.put may throw exception unrelated to the current put
  attempt
- Replace zkClient.getChildren method with zkUtils.getChildren
- Options for handling corrupt data during deserialization
- Improve exception handling for state stores
- Distinguish abortable failures in transactional producer
- Unexpected state transition Dead to PendingShutdown
- Replace zkClient.subscribe*Changes method with an equivalent zkUtils
  method
- Replace zkClient.exists method with zkUtils.pathExists
- Replace zkClient.delete* method with an equivalent zkUtils method
- Idempotent producer should not require
  max.in.flight.requests.per.connection=1
- introduce async ZookeeperClient
- Check exception handling in all streams tests
- Autogenerate Producer sender metrics
- Provide a simple way for custom storage engines to use streams wrapped
  stores (KIP-182)
- Add new API methods to KTable
- Add new API methods to KGroupedStream
- Add new API methods to KGroupedTable
- Add Topology and deprecate TopologyBuilder
- Add StreamsBuilder and deprecate KStreamBuilder
- Refactor KeyValueStore hierarchy so that MeteredKeyValueStore is the
  outermost store
- Refactor WindowStore hierarchy so that Metered Store is the outermost
  store
- Add the archetype project along with "write applications" web docs.
- Refactor SessionStore hierarchy
- Add the Trogdor fault injection daemon
- Add ducktape integration for the Trogdor Fault injection daemon
- Tighten up situations where OutOfOrderSequence may be returned
- Fix transient unit test failure in trogdor coordinator shutdown
- Add Printed class and KStream#print(Printed)
- Add Produced class and new to and through overloads to KStream
- Add Serialized class and KStream groupBy and groupByKey overloads
- Add Joined class and relevant KStream join overloads
- Update Docs
- Add Consumed class and overloads to StreamBuilder
- Upgrade Guide doc changes for KIP-130
- Add groupBy(KeyValueMapper, Serialized) to KTable
- Add filter, filterNot, mapValues and Materialized to KTable
- Add WindowedKStream interface and implemenation
- Add Materialized overloads to StreamBuilder
- Transactions system test should check for message order
- Create Connect metrics for connectors
- Create Connect metrics common to source and sink tasks
- Create Connect metrics for source tasks
- Create Connect metrics for sink tasks
- Create Connect metrics for workers
- Create Connect metrics for worker rebalances
- Add `hasOffset` and `hasTimestamp` methods to RecordMetadata to
  indicate whether metadata is available
- Add Materialized overloads to WindowedKStream
- Add SessionWindowedKStream
- Deprecate KTable#to and KTable#through
- Improve ProcessorStateManager exception handling
- User Callback Exceptions need to be handled properly
- Add generated documentation for Connect metrics

Log message:
Update devel/kafka to 0.11.0.0.

New Feature
- KIP-146: Support per-connector/per-task classloaders in Connect
- Add Record Headers
- Add purgeDataBefore() API in AdminClient
- Add KStream.peek(ForeachAction)
- Add a tool to Reset Consumer Group Offsets
- Add internal leave.group.on.close config to consumer
- Add Exactly-Once Semantics to Streams
- Implement Transactional Coordinator
- Range Scan for Windowed State Stores

Improvement
- Extend wire protocol to allow CRC32C
- Cluster collection returning methods should never return null
- Add Prompt to kafka-console-producer
- Allow users greater access to register custom streams metrics
- Exponential backoff for broker reconnect attempts (KIP-144)
- Add JMH module for Benchmarks
- Split the ProducerBatch and resend when received
  RecordTooLargeException
- Allow per stream/table timestamp extractor
- support throttling on request rate
- Replication Quota Tests: Ensure ZK updated before tests start
- REST configuration not visible in connector properties config files
- TopicCommand --describe shows topics marked for deletion as
  under-replicated and unavailable (KIP-137)
- RocksDB checkpoint files lost on kill -9
- Treat null-key records the same way for joins and aggreations
- Per listener security setting overrides (KIP-103)
- Kafka does not offer kafka-configs.bat on Windows box
- Parametrize streams benchmarks to run at scale
- Error message from Struct.validate() should include the name of the
  offending field.
- Add application.id to StreamThread name
- Improve Streams Reset Tool console output
- Add Float serializer, deserializer, serde
- Exploit #peek to implement #print() and other methods
- The Kafka build should run findbugs
- Inner classes which don't need a reference to the outer class should
  be static
- Fix findbugs warnings in kafka-tools
- throw NoOffsetForPartitionException once for all assigned partitions
  from poll
- Stream round-robin scheduler is inneficient
- Fix several FindBugs warnings in Clients and Connect
- SCRAM secret's should be better protected with Zookeeper ACLs
- set internal.leave.group.on.close to false in KafkaStreams
- Add listener tag to socket-server-metrics.connection-... metrics
- Followups from KIP-101
- We shouldn't pass the underlying exception to
  RetriableCommitFailedException when an async offset commit fails.
- Optionally print out metrics after running the perf tests
- ReassignPartitionsCommand should protect against empty replica list
  assignment
- Censor SCRAM config change logging
- DumpLogSegments should not open index files with `rw`
- Add support for reading PID snapshot files to DumpLogSegments
- Improve internal Task APIs
- Trunk compatibility tests should test against 0.10.2
- Improve message for Kafka failed startup with non-Kafka data in
  data.dirs
- Controlled shutdown timeout message improvement
- reassign-partitions to check if broker of ID exists in cluster
- Add option "dry run" to Streams application reset tool
- AdminClient: add controller and clusterId methods to
  DescribeClusterResults
- KIP-153: Include only client traffic in BytesOutPerSec metric
- Application Reset Tool does not need to seek for internal topics
- New Short serializer, deserializer, serde
- Change Default unclean.leader.election.enabled from True to False
- Sticky Assignor should not cache the calculated assignment (KIP-54
  follow-up)
- Support ExtendedDeserializer in Kafka Streams
- Remove Broker ACL resource type
- Request log should log throttle time
- Generate javadoc for AdminClient and show configs in documentation
- Annoying braces showed on log.error using streams

Bug
- Implement generation/term per leader to reconcile messages correctly
- Hold the produce request with ack > 1 in purgatory until replicas'
  HW has larger than the produce offset (KIP-101)
- Delete consumer offsets from kafka once the topic is deleted
- Clean up Controller Object on forced Resignation
- ConsumerGroupCommand throws GroupCoordinatorNotAvailableException
  when describing a non-existent group before the offset topic is
  created
- SASL unit tests dont work with IBM JDK
- Follower Broker cannot start if offsets are already out of range
- Remove deprecated producer configs.
- Kafka default -Xloggc settings should include GC log rotation flags
- Streams is creating two ProducerRecords for each send via
  RecordCollector
- Broker faills to start after ungraceful shutdown due to
  non-monotonically incrementing offsets in logs
- Log should check the return value of dir.mkdirs()
- KIP-115: __consumer_offsets wrong number of replicas at startup
- Issue with processing order of consumer properties in console
  consumer
- Exit Strategy: using exceptions instead of inline invocation of
  exit/halt
- Documentation still refers to AsyncProducer and SyncProducer
- KIP-151: Connect REST API should expose whether each connector is a
  source or sink
- KTable doesn't work with ProcessorTopologyTestDriver in Kafka
  0.10.1.0
- rat license check not running in Jenkins
- When using ProcessorTopologyTestDriver, the combination of map and
  .groupByKey does not produce any result
- Set more conservative default values on RocksDB for memory usage
- Significant startup delays in KStreams app
- Kafka should not require SSL trust store password
- When the destination brokers are down or misconfigured in config,
  Streams should fail fast
- Connect Producer and Consumer ignore ssl parameters configured for
  worker
- Annotate integration tests and provide gradle build targets to run
  subsets of tests
- Controller send thread can't stop when broker change listener event
  trigger for dead brokers
- the argument of shell in doc wrong and command parsed error
- Kafka Streams allows you to provide strings with illegal characters
  for internal topic names
- gradle-generated core eclipse project has incorrect source folder
  structure
- Refresh consumer metadata more frequently for unknown subscribed
  topics
- KIP-154: Connect should create internal topics
- Fix Streams window retention policy
- KIP-100 api changes break Java 8 lambda expressions in some cases
- Python VerifiableConsumer service has thread-safety bug for
  event_handlers
- Avoid unnecessary task movement across threads during rebalance
- OffsetValidationTest fails validation with "Current position greater
  than the total number of consumed records"
- StreamsKafkaClient drops security configs
- Fix Transient failure in
  BrokerApiVersionsCommandTest.checkBrokerApiVersionCommandOutput
- Implement remaining KIP-66 SMTs
- Fix logic for re-checking if internal topic is ready
- connect jars are missing LICENSE/NOTICE files
- Timed out waiting for consumption in
  OffsetValidationTest.test_broker_failure
- Clean up of state directories can possibly remove stores that are
  about to be used by another thread
- KafkaConsumer#commitSync should clone its input
- Fix deadlock issue during MM shutdown
- Remove generic type of class ClientState
- Memory leak in RecordAccumulator.append
- Streams_bounce test failing occassionally
- fix join-time-max and sync-time-max MeasurableStat type
- Streams Simple Benchmark MB/sec calculation is not correct for Join
  operations
- SimpleBenchmark test fails for streams
- Improve NetworkClient trace logging of request details
- Connect WorkerSinkTask is missing checks for NO_TIMESTAMP
- \ 
org.apache.kafka.clients.producer.KafkaProducerTest#testConstructorFailureCloseR \ 
esource
  and Similar Tests are Failing on some Systems (127.0.53.53 Collision
  Warning)
- Implement graceful handling for improperly formed compressed message
  sets
- Kafka client Heartbeat thread use all the cpu.
- Failure in
  kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py
- KIP-128: Blackbox or pass through converter or ByteArrayConverter
  for connect
- Broker level configuration 'log.segment.bytes' not used when
  'segment.bytes' not configured per topic.
- ProcessorTopologyTestDriver does not forward extracted timestamps to
  internal topics
- Kafka cannot recover after a disk full
- Kafka Streams - unable to add state stores when using wildcard
  topics on the source
- Fix some findbugs warnings in Kafka Java client
- Streams State transition ASCII diagrams need fixing and polishing
- KafkaConsumer: ConsumerConfig gets logged twice.
- docker/run_tests.sh should set up /opt/kafka-dev to be the source
  directory
- SchemaBuilder should be more lax about checking that fields are
  unset if they are being set to the same value
- ReplicaFetchThread may fail to create due to existing metric
- ZookeeperLeaderElector not respecting zookeeper.set.acl
- Fix some findbugs warnings in Kafka Streams
- ProcessorTopologyTestDriver does not work when using .through()
- Config validation in Connector plugins need to compare against both
  canonical and simple class names
- NetworkClient should only consider a connection to be fail after
  attempt to connect
- Stream thread getting into deadlock state while trying to get
  rocksdb lock in retryWithBackoff
- SessionStore.fetch(key) is a performance bottleneck
- Struct SchemaBuilder should not allow duplicate fields.
- Querying window store may return unwanted keys
- Kafka Secure Migrator tool doesn't secure all the nodes
- Kafka console consumer property is ignored
- Kafka Connect does not log connector configuration errors
- processstreamwithcachedstatestore and other streams benchmarks fail
  occasionally
- kafka.request.logger TRACE regression
- Fix findbugs "default character set in use" warnings
- Fix findbugs "format string should use %n rather than \n" in tools
- Fix findbugs warnings in kafka-core
- Make ProduceRequest thread-safe
- Utils#delete should correctly handle I/O errors and symlinks
- Remove unused code for reading Shell command stdout and add unit
  test
- Add streams tests with brokers failing
- Document that stores must not be closed when Processors are closed
- Fix findbugs warnings in Kafka-Connect-API
- Add a configurable delay to the initial consumer group rebalance
- KStreamsTestDriver fails with NPE when KStream.to() sinks are used
- Transformation Key/Value type references should be to class name(),
  not canonicalName()
- Batch resetting offsets in Streams' StoreChangelogReader
- Kafka Connect: Offset committing times out before expected
- Fix an "unread field" findbugs warning in streams examples
- Suppress findbugs warnings about machine-generated code in
  jmh-benchmarks
- remove controller concurrent access to non-threadsafe NetworkClient,
  Selector, and SSLEngine
- Delete the kafka to prefix the name of the keystore and truststore
  file will be more suitable
- kafka-connect: fix findbugs issues in connect/runtime
- testReprocessingFromScratch unit test failure
- Fix findbugs warnings in kafka-clients
- Fix remaining findbugs warnings in Kafka Streams
- StreamThread should catch InvalidTopicException
- JoinIntegrationTest fails occasionally
- Fail the build when findbugs fails
- SSL Channel not ready but tcp is established and the server is hung
  will not sending metadata
- running multiple kafka streams instances causes one or more instance
  to get into file contention
- Increase number of Streams producer retries from the default of 0
- InFlightRequests#isEmpty() always returns false
- Add FindCoordinatorRequest RPC stub and update InitPidRequest for
  KIP-98
- NullPointerException while using GlobalKTable in KafkaStreams
- Chroot check should be done for each ZkUtils instance
- Avoid DNS reverse lookup in security-critical TLS code path
- Kafka Streams skipped-records-rate sensor producing nonzero values
  even when FailOnInvalidTimestamp is used as extractor
- Kafka Streams stuck rebalancing after exception thrown in rebalance
  listener
- Defer exception to the next pollOnce() if consumer's fetch position
  has already increased
- PartitionRecords.fetchRecords(...) should defer exception to the
  next call if iterator has already moved across any valid record
- two versions of jackson-annotations-xxx.jar in distribution tgz
- Update topic expiry time in Metadata every time the topic metadata
  is requested
- some spelling error in code comment
- Kafka Streams SessionStore.findSessions javadoc broken
- ThreadCacheTest.cacheOverheadsSmallValues fails intermittently
- Replica Deletion Regression from KIP-101
- ProducerPerformanceService failing due to parsing error
- Remove KafkaController's incrementControllerEpoch method parameter
- Transient test failure
  SocketServerTest.testMetricCollectionAfterShutdown
- Controller Health Metrics (KIP-143)
- Flaky ResetIntegrationTest
- Windows platform does not offer kafka-broker-api-versions.bat
- MinTimestampTracker uses confusing variable names
- LZ4 decompression is 4-5x slower than Snappy on small batches /
  messages
- SetSchemaMetadata does not replace the schemas in structs correctly
- streams task gets stuck after re-balance due to LockException
- KafkaConsumer.close should be idempotent
- KafkaAdminClientIntegration test should wait until metadata is
  propagated to all brokers
- CachingSessionStore doesn't fetchPrevious correctly.
- SASL tests failing with Could not find a 'KafkaServer' or
  'sasl_plaintext.KafkaServer' entry in the JAAS configuration
- RocksDb might stall in environments with 1 core only
- RocksDbStore#openIterators should be synchronized, since it is
  accessed from multiple threads
- Percentilles are calculated incorrectly
- CachingSessionStore doesn't use the default keySerde.
- RocksDBSessionStore doesn't use default aggSerde.
- KafkaConsumer should not skip a corrupted record after throwing an
  exception.
- IllegalStateException in ensureOpenForRecordAppend
- Small JavaDoc fix for AdminClient#describeTopics
- Cached Session/Window store may return error on
  iterator.peekNextKey()
- NullPointerException (NPE) in
  SourceNodeRecordDeserializer.deserialize
- Reflections logs excessive warnings when scanning classpaths
- Recommended values for Connect transformations contain the wrong
  class name
- Kafka broker fails to start if a topic containing dot in its name is
  marked for delete but hasn't been deleted during previous uptime
- Regression in on-disk log size when using Snappy compression with
  0.8.2 log message format
- GlobalKTable does not checkpoint offsets after restoring state
- Tests which delete singleton metrics break subsequent metrics tests
- handleFetchRequest should do down conversion after throttling
- kakfa-clients consume 100% CPU with manual partition assignment when
  network connection is lost
- Move ACLs, Config, NodeVersions classes into org.apache.kafka.common
- Follow-up improvements for consumer offset reset tool (KIP-122)
- kafka-console-consumer: `--value-deserializer` is not working but
  `--property value.deserializer` does
- One StopReplicaRequest will caused two Responses
- Do not apply exponential backoff if users have overridden
  reconnect.backoff.ms
- PlainSaslServerFactory should allow a null Map in getMechanismNames
- FetchRequest doesn't implement toString
- Missing logging information in ReplicaFetcher
- Stores not queryable after one thread died
- Log cleaning can increase message size and cause cleaner to crash
  with buffer overflow
- MemoryRecords.filterTo can return corrupt data if output buffer is
  not large enough
- AdminClient: add close with timeout, fix some timeout bugs
- Connection Lose during Kafka Kerberos Renewal process
- Replica list in the metadata cache on the broker may have different
  order from zookeeper
- rocksdb.config.setter must be a class instance, not a class name
- ListGroup requires Describe on Cluster, but the command-line
  AclCommand tool does not allow this to be set
- There is a Misspell in ResetIntegrationTest
- Change message.timestamp.difference.max.ms back to Long.MaxValue
- Some socket connections not closed after restart of Kafka Streams
- KafkaConsumer occasionally hits IllegalStateException
- Modify Unstable annotations in Streams API
- baseTimestamp should always have a create timestamp
- MirrorMaker not preserving headers
- Down-converted uncompressed batches should respect fetch offset
- Add EOS integration tests for Streams API
- Add EOS system tests for Streams API
- Kafka Streams skipped-records-rate sensor produces nonzero values
  when the timestamps are valid
- ConsoleConsumer prints out object addresses rather than what is
  expected
- AdminClient gets "server returned information about unknown
  correlation ID" when communicating with older brokers
- Transactions: Concurrent transactional consumer loses messages when
  there are broker bounces
- Transient test failure:
  KafkaConsumerTest.testChangingRegexSubscription
- Log recovery can fail if topic names contain one of the index
  suffixes
- Transactional Producer allows batches to expire and commits
  transactions regardless
- KafkaAdminClient#timeoutCallsInFlight does not work as expected
- Distributed Herder Deadlocks on Shutdown
- Add more AdminClient checks to ClientCompatibilityTest
- Log cleaner fails due to large offset in segment file
- Console consumer offset commit regression
- TransactionCoordinator doesn't complete transition to PrepareCommit
  state
- TransactionCoordinator doesn't complete transition to CompleteCommit
- ZkUtils.getAllPartitions() may fail if a topic is marked for
  deletion
- Multiple produce request failures causes invalid state transition in
  TransactionManager
- Transient test failure:
  SaslPlainSslEndToEndAuthorizationTest.testNoProduceWithDescribeAcl
- Streams producer `client.id` are not unique for EOS
- TimestampConverter's "type" config conflicts with the basic
  Transformation "type" config
- Flaky test
  TransactionsTest.testReadCommittedConsumerShouldNotSeeUndecidedData
- Scripts to startup Connect in system tests have too short a timeout
- Update java docs for consumer and producer to be up to date for EOS
- Producer fails with NPE if compressed V0 or V1 record is larger than
  batch size
- MemoryRecordsBuilder.hasRoomfor doesn't take into account the
  headers while computing available space
- Controller incorrectly logs rack information when new brokers are
  added
- Connector validate REST endpoint returning duplicate entries in
  "groups"
- Connector config validation REST API endpoint not including fields
  for transformations
- TransactionalProducer sleeps unnecessarily long during back to back
  transactions
- org.apache.kafka logging should go to server.log
- The ProducerPerformance tool should support transactions
- Connect validation API stops returning recommendations for some
  fields after the right sequence of requests

Task
- Remove deprecated APIs for 0.11.0.0
- Drop support for Scala 2.10 (KIP-119)
- Remove unstable markers from Connect APIs
- KTable materialization and improved semantics

Test
- Transient failure in
  ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade with
  security_protocol = SASL_PLAINTEXT, SSL
- Test with two SASL_SSL listeners with different JAAS contexts
- Implement KIP-98 transactional methods in the MockProducer
- Add cases for concurrent transactional reads and writes in system
  tests
- SyncProducerTest.testReachableServer has become flaky

Sub-task
- KIP-54: Add rebalance with a minimal number of reassignments to
  server-defined strategy list
- ProducerSendTest.testCloseWithZeroTimeoutFromCallerThread Transient
  Failure
- Transient Failure in kafka.api.PlaintextProducerSendTest.testFlush
- Mark the old Scala consumer and related classes as deprecated
- Create Java Admin Client
- Implement KIP-140 RPCs and APIs for creating, altering, and listing
  ACLs
- Describe/Alter Configs protocol, server and client (KIP-133)
- Transient test failure:
  kafka.api.SslProducerSendTest.testSendNonCompressedMessageWithCreateTime
- Unstable test
  KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations
- Transient test failure:
  ConsumerBounceTest.testConsumptionWithBrokerFailures
- Transient failure in QueryableStateIntegrationTest.queryOnRebalance
- Address 2.12 eta-expansion warnings
- Update release process so that Scala 2.12 artifacts are published
- Run tests on travis-ci using docker
- Transient failure in
  \ 
org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvail \ 
able
- QueryableStateIntegrationTest.shouldNotMakeStoreAvailableUntilAllStoresAvailable
  is occasionally failing on jenkins
- Improve test coverage of ProcessorStateManager
- Improve test coverage of StreamsPartitionAssignor
- Improve test coverage of ProcessorTopology
- Improve test coverage AbstractProcessorContext
- Improve test coverage of GlobalStreamThread
- Improve test coverage StreamTask
- Improve test coverage GlobalStateManagerImpl
- Improve test coverage KStreamBuilder
- Improve test coverage MemoryLRUCache
- Improve test coverage of CompositeReadOnlyWindowStore
- Improve test coverage KafkaStreams
- Improve test coverage TopologyBuilder
- Transient test failure:
  ConsumerCoordinatorTest.testHeartbeatThreadClose
- Message format changes for idempotent/transactional producer
- Implement idempotent producer
- Implement transactional clients
- Transient test failure:
  org.apache.kafka.streams.integration.JoinIntegrationTest.shouldCountClicksPerRegion
  (again)
- Consider disabling record level CRC checks for message format V2
- Failure in
  \ 
kafka.admin.DescribeConsumerGroupTest.testDescribeExistingGroupWithNoMembersWith \ 
NewConsumer
- Implementation and unit tests for request handler thread utilization
  quota
- Add network handler thread utilization to request quota calculation
- Add documentation for request quotas
- Transient failure of AdminClientTest.testDeleteRecordsWithException
- Add producer per task support
- Add request/response classes for transactions (KIP-98)
- KeyValueStore.put may throw exception unrelated to the current put
  attempt
- Exactly-once upgrade notes
- Update Message Delivery Semantics section to take into account
  KIP-98
- convert kafka controller to a single-threaded event queue model
- Additional validation in validateMessagesAndAssignOffsets
- Reconsider default retries for idempotent producer
- add controller integration tests
- Load only batch header when rebuilding producer ID map
- Refactor AdminUtils to use zkUtils methods instad of
  zkUtils.zkClient
- remove preferred replica election state from ControllerContext
- Implement transaction index for KIP-98
- shouldInnerLeftJoin unit test fails
- TransactionCoordinator - Check inter broker protocol and message
  format and raise errors if incompatible
- TransactionCoordinator - Add ACL check for each request
- Change InterBrokerSendThread to use a Queue per broker
- WriteTxnMarkers and complete commit/abort on partition immigration
- Abort long running transactions
- Move coordinatorEpoch from WriteTxnMarkerRequest to TxnMarkerEntry
- KafkaProducer's TransactionManager needs a review on synchronization
- Refactor TransactionCoordinator in-memory structure and error
  handling logic
- KIP-98 : broker side handling for the TxnOffsetCommitRequest
- Add a reference to AdminClient to docs/api.html
- TC should not accept empty string transactional id
- Transient failure:
  ControllerIntegrationTest.testPreferredReplicaLeaderElection
- Log connection termination during authentication
- Transient failure:
  ControllerIntegrationTest.testControllerMoveIncrementsControllerEpoch
- Transient failure: RequestQuotaTest.testResponseThrottleTime
- Avoid expensive initialization of producer state when upgrading
- Add Integration tests for transactional producer
- Autogenerate Consumer Fetcher metrics
- LogCleaner should be transaction-aware
- Handle topic deletion for ongoing transactions
- SaslScramSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
- TransactinoCoordinator does not bump epoch when aborting open
  transactions
- Consumer GroupCoordinator should continue to materialize committed
  offsets in offset order even for transactional offset commits
- Remove retention time from TxnOffsetCommit RPC
- Transaction index recovery does not snapshot properly
- Producer should drop queued sends when transaction is aborted
- Fix flaky test LogCleanerTest.testCommitMarkerRemoval
- move all partition and replica state transition rules into their
  states
- TransactionalId authorization should imply ProducerId authorization
- Producer should not send AbortTxn unless transaction has actually
  begun
- TransactionsBounceTest occasionally sees INVALID_TXN_STATE errors
- TransactionBounceTest occasionally fails due to partition errors
- KafkaConsumer.committed() should get latest committed offsets from
  the server
- Review and improve AdminClient Javadoc for the first release
  (KIP-117)
- Review and potentially tweak AdminClient API for the initial release
  (KIP-117)
- TransactionCoordinator must expire transactionalIds
- Protect concurrent access to the cached transaction status
- System tests for KIP-98 / transactions
- Transactions integration test: Use factory methods to keep track of
  open producers and consumers and close them all on tearDown
- Update clients and server code to make sure that epoch and sequence
  numbers wrap around
- AdminClient should not trigger auto creation of topics
- Fix authorization checks in AdminClient
- TC should handle UNSUPPORTED_FOR_MESSAGE_FORMAT in WriteTxnMarker
  response
- reset ControllerContext during resignation
- Update KIP-98 to reflect changes during implementation.
- Update produce/fetch throttle time metrics for any request throttle
- Resolve AddPartitions response error code inconsistency
- Transactions system test with hard broker bounces fails sporadically
- Add additional test cases for batch splitting to ensure
  idempotent/transactional metadata is preserved
- Broker clean bounce test puts the broker into a
  'CONCURRENT_TRANSACTIONS' state permanently
- Broker returns messages beyond "latest stable offset" to
  transactional consumer in read_committed mode
- StackOverFlow error in transaction coordinator
- Producer attempts to send transactional messages before adding
  partitions to transaction
- Fix regression in compressed message iteration affecting magic v0
  and v1
- Transactions: Concurrent transactional consumer reads aborted
  messages
- Last Stable Offset not returned in Fetch request
- Transactional producer cannot find coordinator when trying to abort
  transaction after error
- Transactional producer aborts batches incorrectly in abortable error
  state
- Producer IllegalStateException: Batch has already been completed
- Produce state lost if no snapshot retained
- TransactionalMessageCopier should be force killed on test shutdown
- UnsupportedOperationException in WriteTxnMarkers handler
- Consumer should use last offset from batch to set next fetch offset
- read current brokers from zookeeper upon processing broker change