./databases/elasticsearch, Distributed RESTful Search Engine

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 6.2.0, Package name: elasticsearch-6.2.0, Maintainer: imil

Elasticsearch is a distributed RESTful search engine built for the cloud.
Features include:

* Distributed and Highly Available Search Engine.
* Multi Tenant with Multi Types.
* Various set of APIs
* Document oriented
* Reliable, Asynchronous Write Behind for long term persistency.
* (Near) Real Time Search.
* Built on top of Lucene
* Per operation consistency
* Open Source under Apache 2 License.


Required to run:
[shells/bash] [lang/openjdk8]

Required to build:
[pkgtools/cwrappers]

Master sites:

SHA1: 05f8527be810d45888a00f69edee5c73a13b737c
RMD160: 4ea13b8681c2439cf64fe648615a3530bc8f2693
Filesize: 28366.583 KB

Version history: (Expand)


CVS history: (Expand)


   2018-02-07 17:26:16 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
databases/elasticsearch: Update to 6.2.0.

6.2.0

=== Breaking changes

Aggregations::
- Add a new cluster setting to limit the total number of buckets
  returned by a request

Core::
- Forbid granting the all permission in production

Highlighting::
- Limit the analyzed text for highlighting

Search::
- Introduce limit to the number of terms in Terms Query

=== Breaking Java changes

Java API::
- Remove `operationThreaded` from Java API

Java High Level REST Client::
- REST high-level client: remove index suffix from indices client
  method names

=== Deprecations

Analysis::
- Backport delimited payload filter renaming

Suggesters::
- deprecating `jarowinkler` in favor of `jaro_winkler`
- Deprecating `levenstein` in favor of `levensHtein`

=== New features

Plugin Ingest GeoIp::
- Enable ASN support for Ingest GeoIP plugin.

Plugin Lang Painless::
- Painless: Add spi jar that will be published for extending
  whitelists
- Painless: Add a simple cache for whitelist methods and fields.

Plugins::
- Add the ability to bundle multiple plugins into a meta plugin

Rank Evaluation::
- Backport of ranking evaluation API (#27478)

Recovery::
- Backport for using lastSyncedGlobalCheckpoint in deletion policy

Reindex API::
- Add scroll parameter to _reindex API

=== Enhancements

Allocation::
- Fix cluster.routing.allocation.enable and
  cluster.routing.rebalance.enable case
- Add node id to shard failure message

Analysis::
- Limit the analyzed text for highlighting (#27934)
- Allow TrimFilter to be used in custom normalizers

Circuit Breakers::
- Add accounting circuit breaker and track segment memory usage

Cluster::
- Adds wait_for_no_initializing_shards to cluster health API

Core::
- Introduce elasticsearch-core jar
- Rename core module to server
- Introduce elasticsearch-core jar
- Add Writeable.Reader support to TransportResponseHandler
- Simplify rejected execution exception
- Add node name to thread pool executor name

Discovery::
- Add information when master node left to DiscoveryNodes'
  shortSummary()

Engine::
- Move uid lock into LiveVersionMap
- Optimize version map for append-only indexing

Geo::
- [GEO] Add WKT Support to GeoBoundingBoxQueryBuilder
- [Geo] Add Well Known Text (WKT) Parsing Support to ShapeBuilders

Highlighting::
- Include all sentences smaller than fragment_size in the unified
  highlighter

Ingest::
- Enable convert processor to support Long and Double

Internal::
- Make KeyedLock reentrant
- Make AbstractQueryBuilder.declareStandardFields to be protected
- Tighten the CountedBitSet class
- Avoid doing redundant work when checking for self references.

Java API::
- Add missing delegate methods to NodeIndicesStats
- Java api clean-up : consistency for `shards_acknowledged` getters

Java High Level REST Client::
- add toString implementation for UpdateRequest.
- Add Close Index API to the high level REST client
- Add Open Index API to the high level REST client
- Added Create Index support to high-level REST client
- Add multi get api to the high level rest client
- Add msearch api to high level client

Mapping::
- Allow `_doc` as a type.

Network::
- Add NioGroup for use in different transports
- Add read timeouts to http module
- Implement byte array reusage in `NioTransport`
- Introduce resizable inbound byte buffer
- Decouple nio constructs from the tcp transport

Packaging::
- Extend JVM options to support multiple versions
- Add explicit coreutils dependency
- Detect mktemp from coreutils
- Enable GC logs by default
- Use private directory for temporary files

Percolator::
- also extract match_all queries when indexing percolator queries

Plugin Lang Painless::
- Painless: Add whitelist extensions
- Painless: Modify Loader to Load Classes Directly from Definition
- Clean Up Painless Cast Object
- Painless: Only allow Painless type names to be the same as the
  equivalent Java class.

Plugins::
- Add client actions to action plugin
- Plugins: Add validation to plugin descriptor parsing
- Plugins: Add plugin extension capabilities
- Add support for filtering mappings fields

Rank Evaluation::
- Simplify RankEvalResponse output

Recovery::
- Truncate tlog cli should assign global checkpoint
- Replica starts peer recovery with safe commit
- Primary send safe commit in file-based recovery
- Fail resync-failed shards in subsequent writes
- Introduce promoting index shard state
- Non-peer recovery should set the global checkpoint
- Persist global checkpoint when finalizing a peer recovery
- Rollback a primary before recovering from translog

Rollover::
- Fail rollover if duplicated alias found in templates

Search::
- Use typeName() to check field type in GeoShapeQueryBuilder
- Optimize search_after when sorting in index sort order

Sequence IDs::
- Do not keep 5.x commits when having 6.x commits
- Use lastSyncedGlobalCheckpoint in deletion policy
- Use CountedBitSet in LocalCheckpointTracker
- Only fsync global checkpoint if needed
- Keep commits and translog up to the global checkpoint
- Adjust CombinedDeletionPolicy for multiple commits
- Keeps index commits up to the current global checkpoint
- Dedup translog operations by reading in reverse

Settings::
- Add validation of keystore setting names

Snapshot/Restore::
- Use AmazonS3.doesObjectExist() method in S3BlobContainer
- Remove XContentType auto detection in BlobStoreRepository
- Include include_global_state in Snapshot status API (#22423)

Task Manager::
- Add ability to associate an ID with tasks

Translog::
- Simplify MultiSnapshot#SeqNoset
- Enclose CombinedDeletionPolicy in SnapshotDeletionPolicy

=== Bug fixes

Aggregations::
- Adds metadata to rewritten aggregations
- Fix NPE on composite aggregation with sub-aggregations that need
  scores
- StringTerms.Bucket.getKeyAsNumber detection type
- Fix incorrect results for aggregations nested under a nested
  aggregation
- Fix global aggregation that requires breadth first and scores
- Fix composite aggregation when after term is missing in the shard
- Fix preserving FiltersAggregationBuilder#keyed field on rewrite
- Using DocValueFormat::parseBytesRef for parsing missing value
  parameter
- Fix illegal cast of the "low cardinality" optimization of the
  `terms` aggregation.
- Always include the _index and _id for nested search hits.

Allocation::
- Do not open indices with broken settings

Core::
- Fix lock accounting in releasable lock
- Fixes ByteSizeValue to serialise correctly
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Ensure logging is configured for CLI commands

Engine::
- Replica recovery could go into an endless flushing loop
- Use `_refresh` to shrink the version map on inactivity
- Allow resize version map under lock even if there are pending
  operations
- Reset LiveVersionMap on sync commit

Geo::
- Correct two equality checks on incomparable types
- Handle case where the hole vertex is south of the containing
  polygon(s)

Highlighting::
- Fix highlighting on a keyword field that defines a normalizer

Inner Hits::
- Add version support for inner hits in field collapsing (#27822)

Internal::
- Never return null from Strings.tokenizeToStringArray
- Fallback to TransportMasterNodeAction for cluster health retries
- Retain originalIndex info when rewriting FieldCapabilities requests

Java REST Client::
- Do not use system properties when building the HttpAsyncClient

Mapping::
- Ignore null value for range field (#27845)
- Pass `java.locale.providers=COMPAT` to Java 9 onwards
- Allow update of `eager_global_ordinals` on `_parent`.
- Fix merging of _meta field

Network::
- Only bind loopback addresses when binding to local
- Remove potential nio selector leak
- Fix issue where the incorrect buffers are written
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams

Packaging::
- Allow custom service names when installing on windows

Percolator::
- Avoid TooManyClauses exception if number of terms / ranges is
  exactly equal to 1024

Plugin Analysis ICU::
- Catch InvalidPathException in IcuCollationTokenFilterFactory

Plugin Analysis Phonetic::
- Fix daitch_mokotoff phonetic filter to use the dedicated Lucene
  filter

Plugin Lang Painless::
- Painless: Fix variable scoping issue in lambdas
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository HDFS::
- Fix SecurityException when HDFS Repository used against HA Namenodes

Plugins::
- Make sure that we don't detect files as maven coordinate when
  installing a plugin
- Fix upgrading indices which use a custom similarity plugin.

Recovery::
- Open engine should keep only starting commit
- Allow shrinking of indices from a previous major
- Set global checkpoint before open engine from store
- Check and repair index under the store metadata lock
- Flush old indices on primary promotion and relocation

Rollover::
- Make index rollover action atomic

Scripting::
- Ensure we protect Collections obtained from scripts from
  self-referencing

Scroll::
- Reject scroll query if size is 0 (#22552)
- Fix scroll query with a sort that is a prefix of the index sort

Search::
- Fix simple_query_string on invalid input
- Use the underlying connection version for CCS connections
- Fix synonym phrase query expansion for cross_fields parsing
- Carry forward weights, etc on rescore rewrite
- Fix routing with leading or trailing whitespace

Sequence IDs::
- Recovery from snapshot may leave seq# gaps
- No longer unidle shard during recovery
- Obey translog durability in global checkpoint sync

Settings::
- Settings: Introduce settings updater for a list of settings
- Fix setting notification for complex setting (affixMap settings)
  that could cause transient settings to be ignored
- Fix environment variable substitutions in list setting
- Allow index settings to be reset by wildcards

Snapshot/Restore::
- Consistent updates of IndexShardSnapshotStatus
- Avoid concurrent snapshot finalizations when deleting an INIT
  snapshot
- Do not start snapshots that are deleted during initialization
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Consistent update of stage and failure message in
  IndexShardSnapshotStatus
- Fail restore when the shard allocations max retries count is reached
- Delete shard store files before restoring a snapshot

Stats::
- Fixes DocStats to properly deal with shards that report -1 index
  size
- Include internal refreshes in refresh stats

Term Vectors::
- Fix term vectors generator with keyword and normalizer

=== Upgrades

Core::
- Dependencies: Update joda time to 2.9.9
- upgrade to lucene 7.2.1
- Upgrade jna from 4.4.0-1 to 4.5.1

Ingest::
- update ingest-attachment to use Tika 1.17 and newer deps

6.1.3

Engine::
- Replica recovery could go into an endless flushing loop

Internal::
- Never return null from Strings.tokenizeToStringArray
- Fallback to TransportMasterNodeAction for cluster health retries

Mapping::
- Allow update of `eager_global_ordinals` on `_parent`.

Scripting::
- Ensure we protect Collections obtained from scripts from
  self-referencing

Settings::
- Fix setting notification for complex setting (affixMap settings)
  that could cause transient settings to be ignored
- Fix environment variable substitutions in list setting

Snapshot/Restore::
- Avoid concurrent snapshot finalizations when deleting an INIT
  snapshot
- Do not start snapshots that are deleted during initialization
   2018-01-22 20:27:21 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
Update databases/elasticsearch to 6.1.2

== 6.1.2

=== Enhancements

Internal::
- Make AbstractQueryBuilder.declareStandardFields to be protected

=== Bug fixes

Aggregations::
- Fix incorrect results for aggregations nested under a nested
  aggregation
- Fix composite aggregation when after term is missing in the shard
- Fix preserving FiltersAggregationBuilder#keyed field on rewrite

Engine::
- Use `_refresh` to shrink the version map on inactivity
- Allow resize version map under lock even if there are pending
  operations

Network::
- Only bind loopback addresses when binding to local

Recovery::
- Allow shrinking of indices from a previous major

Search::
- Use the underlying connection version for CCS connections
- Carry forward weights, etc on rescore rewrite

Snapshot/Restore::
- Fail restore when the shard allocations max retries count is reached

Translog::
- Only sync translog when global checkpoint increased

== 6.1.1

=== Enhancements

Snapshot/Restore::
- Use AmazonS3.doesObjectExist() method in S3BlobContainer

=== Bug fixes

Inner Hits::
- Add version support for inner hits in field collapsing

Java REST Client::
- Do not use system properties when building the HttpAsyncClient

Search::
- Fix routing with leading or trailing whitespace

Sequence IDs::
- Recovery from snapshot may leave seq# gaps
- No longer unidle shard during recovery

=== Upgrades

Ingest::
- update ingest-attachment to use Tika 1.17 and newer deps
   2017-12-18 15:27:37 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
Update databases/elasticsearch to 6.1.0.

== 6.1.0 Release Notes

=== Breaking changes

Network::
- Allow only a fixed-size receive predictor

REST::
- Standardize underscore requirements in parameters

Scroll::
- Fail queries with scroll that explicitely set request_cache

Search::
- Add a limit to from + size in top_hits and inner hits.

=== Breaking Java changes

Aggregations::
- Moves deferring code into its own subclass

Core::
- Unify Settings xcontent reading and writing

Settings::
- Return List instead of an array from settings
- Remove `Settings,put(Map<String,String>)`

=== Deprecations

Aggregations::
- Deprecate global_ordinals_hash and global_ordinals_low_cardinality

Allocation::
- Add deprecation warning for negative
  index.unassigned.node_left.delayed_timeout

Analysis::
- Add limits for ngram and shingle settings

Geo::
- [GEO] 6x Deprecate ShapeBuilders and decouple geojson parse logic

Mapping::
- Deprecate the `index_options` parameter for numeric fields

Plugin Repository Azure::
- Azure repository: Move to named configurations as we do for S3
  repository and secure settings

Search::
- doc: deprecate _primary and _replica shard option

=== New features

Aggregations::
- Aggregations: bucket_sort pipeline aggregation
- Add composite aggregator

Analysis::
- Added Bengali Analyzer to Elasticsearch with respect to the lucene
  update

Ingest::
- add URL-Decode Processor to Ingest

Java High Level REST Client::
- Added Delete Index support to high-level REST client

Nested Docs::
- Multi-level Nested Sort with Filters

Query DSL::
- Add terms_set query
- Introduce sorted_after query for sorted index
- Add support for auto_generate_synonyms_phrase_query in match_query,
  multi_match_query, query_string and simple_query_string

Search::
- Expose `fuzzy_transpositions` parameter in fuzzy queries
- Add upper limit for scroll expiry
- Implement adaptive replica selection
- configure distance limit

Similarities::
- Add a scripted similarity.

Suggesters::
- Expose duplicate removal in the completion suggester
- Support must and should for context query in context suggester

=== Enhancements

Aggregations::
- Allow aggregation sorting via nested aggregation

Allocation::
- Tie-break shard path decision based on total number of shards on
  path
- Balance shards for an index more evenly across multiple data paths
- Expand "NO" decision message in NodeVersionAllocationDecider
- _reroute's retry_failed flag should reset failure counter

Analysis::
- Add configurable `max_token_length` parameter to whitespace
  tokenizer

CRUD::
- Add wait_for_active_shards parameter to index open command

Core::
- Fix classes that can exit
- Replace empty index block checks with global block checks in
  template delete/put actions
- Allow Uid#decodeId to decode from a byte array slice
- Use separate searchers for "search visibility" vs "move indexing
  buffer to disk
- Add ability to split shards
- Make circuit breaker mutations debuggable

Dates::
- DateProcessor Locale

Discovery::
- Stop responding to ping requests before master abdication

Engine::
- Ensure external refreshes will also refresh internal searcher to
  minimize segment creation
- Move IndexShard#getWritingBytes() under InternalEngine
- Refactor internal engine

Geo::
- Add ignore_malformed to geo_shape fields

Ingest::
- add json-processor support for non-map json types
- Introduce templating support to timezone/locale in DateProcessor
- Add support for parsing inline script (#23824)
- Consolidate locale parsing.
- Accept ingest simulate params as ints or strings

Internal::
- Avoid uid creation in ParsedDocument
- Upgrade to Lucene 7.1.0 snapshot version
- Remove `_index` fielddata hack if cluster alias is present

Java High Level REST Client::
- Adjust RestHighLevelClient method modifiers
- Decouple BulkProcessor from ThreadPool

Logging::
- Add more information on _failed_to_convert_ exception (#21946)
- Improve shard-failed log messages.

Mapping::
- Allow ip_range to accept CIDR notation
- Deduplicate `_field_names`.
- Throw a better error message for empty field names
- Stricter validation for min/max values for whole numbers
- Make FieldMapper.copyTo() always non-null.

Nested Docs::
- Use the primary_term field to identify parent documents
- Prohibit using `nested_filter`, `nested_path` and new `nested`
  Option at the same time in FieldSortBuilder

Network::
- Remove manual tracking of registered channels
- Remove tcp profile from low level nio channel
- Decouple `ChannelFactory` from Tcp classes

Percolator::
- Use Lucene's CoveringQuery to select percolate candidate matches
- Add support to percolate query to percolate multiple documents
  simultaneously
- Hint what clauses are important in a conjunction query based on
  fields
- Add support for selecting percolator query candidate matches
  containing range queries

Plugin Discovery EC2::
- update AWS SDK for ECS Task IAM support in discovery-ec2

Plugin Lang Painless::
- Painless: Only allow Painless type names to be the same as the
  equivalent Java class.
- Allow for the Painless Definition to have multiple instances for
  white-listing
- Separate Painless Whitelist Loading from the Painless Definition
- Remove Sort enum from Painless Definition

Plugin Repository Azure::
- Add azure storage endpoint suffix #26432
- Support for accessing Azure repositories through a proxy

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Plugins::
- Plugins: Add versionless alias to all security policy codebase
  properties
- Allow plugins to plug rescore implementations

Query DSL::
- Add support for wildcard on `_index`

Reindex API::
- Update by Query is modified to accept short `script` parameter.
- reindex: automatically choose the number of slices

Rollover::
- Add size-based condition to the index rollover API
- Add size-based condition to the index rollover API

Scripting::
- Script: Convert script query to a dedicated script context

Search::
- Make fields optional in multi_match query and rely on
  index.query.default_field by default
- fix unnecessary logger creation
- `ObjectParser` : replace `IllegalStateException` with
  `ParsingException`
- Uses norms for exists query if enabled
- Cross Cluster Search: make remote clusters optional
- Enhances exists queries to reduce need for `_field_names`
- Change ParentFieldSubFetchPhase to create doc values iterator once
  per segment
- Change VersionFetchSubPhase to create doc values iterator once per
  segment
- Change ScriptFieldsFetchSubPhase to create search scripts once per
  segment
- Make sure SortBuilders rewrite inner nested sorts
- Extend testing of build method in ScriptSortBuilder
- Accept an array of field names and boosts in the
  index.query.default_field setting
- Reject IPv6-mapped IPv4 addresses when using the CIDR notation.
- Rewrite range queries with open bounds to exists query

Sequence IDs::
- Only fsync global checkpoint if needed
- Log primary-replica resync failures
- Lazy initialize checkpoint tracker bit sets
- Returns the current primary_term for Get/MultiGet requests

Settings::
- Allow affix settings to specify dependencies
- Represent lists as actual lists inside Settings
- Remove Settings#getAsMap()
- Replace group map settings with affix setting
- Throw exception if setting isn't recognized
- Settings: Move keystore creation to plugin installation

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository
- Snapshot: Migrate TransportRequestHandler to
  TransportMasterNodeAction
- Fix toString of class SnapshotStatus (#26851)

Stats::
- Adds average document size to DocsStats
- Stats to record how often the ClusterState diff mechanism is used
  successfully
- Expose adaptive replica selection stats in /_nodes/stats API
- Add cgroup memory usage/limit to OS stats on Linux
- Add segment attributes to the `_segments` API.

Suggesters::
- Improve error message for parse failures of completion fields
- Support 'AND' operation for context query in context suggester

=== Bug fixes

Aggregations::
- Disable the "low cardinality" optimization of terms aggregations.
- scripted_metric _agg parameter disappears if params are provided

Cluster::
- Properly format IndexGraveyard deletion date as date
-  Remove optimisations to reuse objects when applying a new
   `ClusterState`

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Ensure logging is configured for CLI commands
- Protect shard splitting from illegal target shards
- Avoid NPE when getting build information
- Fix `ShardSplittingQuery` to respect nested documents.
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Geo::
- Correct two equality checks on incomparable types
- [GEO] fix pointsOnly bug for MULTIPOINT

Index Templates::
- Prevent constructing an index template without index patterns

Ingest::
- Add pipeline support for REST API bulk upsert
- Fixing Grok pattern for Apache 2.4

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Internal::
- When checking if key exists in ThreadContextStruct:putHeaders()
  method, should put requestHeaders in map first
- Adding a refresh listener to a recovering shard should be a noop

Java High Level REST Client::
- Register ip_range aggregation with the high level client
- add top hits as a parsed aggregation to the rest high level client

Mapping::
- Fix dynamic mapping update generation.
- Fix merging of _meta field
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids
- Prevent duplicate fields when mixing parent and root nested includes

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams
- Do not set SO_LINGER on server channels
- Do not set SO_LINGER to 0 when not shutting down
- Close TcpTransport on RST in some Spots to Prevent Leaking TIME_WAIT
  Sockets

Packaging::
- Removes minimum master nodes default number
- setgid on /etc/elasticearch on package install

Percolator::
- Avoid TooManyClauses exception if number of terms / ranges is
  exactly equal to 1024

Plugin Analysis ICU::
- Catch InvalidPathException in IcuCollationTokenFilterFactory

Plugin Lang Painless::
- Painless: Fix variable scoping issue in lambdas
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Scroll::
- Fix scroll query with a sort that is a prefix of the index sort

Search::
- Fix profiling naming issues
- Fix max score tracking with field collapsing
- Apply missing request options to the expand phase
- Calculate and cache result when advanceExact is called
- Filter unsupported relation for RangeQueryBuilder
- Handle leniency for phrase query on a field indexed without
  positions

Sequence IDs::
- Obey translog durability in global checkpoint sync
- Fix resync request serialization

Settings::
- Allow index settings to be reset by wildcards

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
- Do not create directory on readonly repository (#21495)

Stats::
- Include internal refreshes in refresh stats
- Make Segment statistics aware of segments hold by internal readers
- Ensure `doc_stats` are changing even if refresh is disabled

=== Upgrades

Core::
- Upgrade to Jackson 2.8.10
- Upgrade to Lucene 7.1

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0

Plugin Lang Painless::
- Upgrade Painless from ANTLR 4.5.1-1 to  ANTLR 4.5.3.

== 6.0.1 Release Notes

=== Breaking changes

Scroll::
- Fail queries with scroll that explicitely set request_cache

=== Enhancements

Core::
- Fix classes that can exit

Discovery::
- Stop responding to ping requests before master abdication

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Search::
- fix unnecessary logger creation

Sequence IDs::
- Log primary-replica resync failures

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository

=== Bug fixes

Cluster::
- Properly format IndexGraveyard deletion date as date

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Avoid NPE when getting build information
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Mapping::
- Fix dynamic mapping update generation.
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams

Plugin Lang Painless::
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Search::
- Fix profiling naming issues

Sequence IDs::
- Fix resync request serialization

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
  (#25359)
- Do not create directory on readonly repository (#21495)

=== Upgrades

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0
   2017-12-13 13:53:45 by Filip Hajny | Files touched by this commit (3) | Package updated
Log message:
Update databases/elasticsearch to 6.0.1

=== Breaking changes

Scroll::
- Fail queries with scroll that explicitely set request_cache

=== Enhancements

Core::
- Fix classes that can exit

Discovery::
- Stop responding to ping requests before master abdication

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Search::
- fix unnecessary logger creation

Sequence IDs::
- Log primary-replica resync failures

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository

=== Bug fixes

Cluster::
- Properly format IndexGraveyard deletion date as date

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Avoid NPE when getting build information
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Mapping::
- Fix dynamic mapping update generation.
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams

Plugin Lang Painless::
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Search::
- Fix profiling naming issues

Sequence IDs::
- Fix resync request serialization

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
- Do not create directory on readonly repository

=== Upgrades

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0
   2017-11-30 18:50:30 by Filip Hajny | Files touched by this commit (4) | Package updated
Log message:
databases/elasticsearch: Commit forgotten updates to patches.
   2017-11-20 16:36:47 by Filip Hajny | Files touched by this commit (5) | Package updated
Log message:
Update databases/elasticsearch to 6.0.0.

=== Breaking changes

Aggregations::
- Change parsing of numeric `to` and `from` parameters in `date_range`
  aggregation

Aliases::
- Wrong behavior deleting alias

Allocation::
- Remove `cluster.routing.allocation.snapshot.relocation_enabled`
  setting

Analysis::
- Do not allow custom analyzers to have the same names as built-in
  analyzers
- Removing query-string parameters in `_analyze` API

CAT API::
- Write -1 on unbounded queue in cat thread pool

CRUD::
- Disallow `VersionType.FORCE` for GetRequest
- Disallow `VersionType.FORCE` versioning for 6.x indices
- If the index does not exist, delete document will not auto create it

Cluster::
- Disallow : in cluster and index/alias names
- No longer allow cluster name in data path

Core::
- Simplify file store
- Make boolean conversion strict
- Remove the `default` store type.
- Remove store throttling.

Geo::
- Remove deprecated geo search features
- Reduce GeoDistance Insanity

Highlighting::
- Remove the postings highlighter and make unified the default
  highlighter choice

Index APIs::
- Remove (deprecated) support for '+' in index expressions
- Delete index API to work only against concrete indices
- Open/Close index api to allow_no_indices by default
- Remove support for controversial `ignore_unavailable` and
  `allow_no_indices` from indices exists api

Index Templates::
- Allows multiple patterns to be specified for index templates

Indexed Scripts/Templates::
- Scripting: Remove search template actions

Ingest::
- update ingest-user-agent regexes.yml
- remove ingest.new_date_format

Inner Hits::
- Return the _source of inner hit nested as is without wrapping it
  into its full path context

Java API::
- Enforce Content-Type requirement on the rest layer and remove
  deprecated methods

Java REST Client::
- Remove deprecated created and found from index, delete and bulk

Mapping::
- Reject out of range numbers for float, double and half_float
- Enforce at most one type.
- Disallow `include_in_all` for 6.0+ indices
- Disable _all by default, disallow configuring _all on 6.0+ indices
- Throw an exception on unrecognized "match_mapping_type"

Network::
- Remove unused Netty-related settings
- Remove blocking TCP clients and servers
- Remove `modules/transport_netty_3` in favor of `netty_4`
- Remove LocalTransport in favor of MockTcpTransport

Packaging::
- Configure heap dump path out of the box
- Remove support for ES_INCLUDE
- Setup: Change default heap to 1G
- Use config directory to find jvm.options
- Remove implicit 32-bit support
- Remove default path settings
- Remove path.conf setting
- Honor masking of systemd-sysctl.service
- Rename CONF_DIR to ES_PATH_CONF
- Remove customization of ES_USER and ES_GROUP

Percolator::
- Remove deprecated percolate and mpercolate apis

Plugin Analysis ICU::
- Upgrade icu4j for the ICU analysis plugin to 59.1
- Upgrade icu4j to latest version

Plugin Delete By Query::
- Require explicit query in _delete_by_query API

Plugin Discovery Azure Classic::
- Remove `discovery.type` BWC layer from the EC2/Azure/GCE plugins

Plugin Discovery EC2::
- Ec2 Discovery: Cleanup deprecated settings
- Discovery EC2: Remove region setting
- AWS Plugins: Remove signer type setting

Plugin Lang JS::
- Remove lang-python and lang-javascript

Plugin Mapper Attachment::
- Remove mapper attachments plugin

Plugin Repository Azure::
- Remove global `repositories.azure` settings
- Remove auto creation of container for azure repository

Plugin Repository GCS::
- GCS Repository: Remove specifying credential file on disk

Plugin Repository S3::
- S3 Repository: Cleanup deprecated settings
- S3 Repository: Remove region setting
- S3 Repository: Remove bucket auto create
- S3 Repository: Remove env var and sysprop credentials support
- Remove deprecated S3 settings

Plugins::
- Make plugin loading stricter

Query DSL::
- Remove deprecated `type` and `slop` field in `match` query
- Remove several parse field deprecations in query builders
- Remove deprecated parameters from `ids_query`
- Refactor QueryStringQuery for 6.0
- Change `split_on_whitespace` default to false
- Remove deprecated template query
- Throw exception in scroll requests using `from`
- Remove deprecated `minimum_number_should_match` in BoolQueryBuilder
- Remove support for empty queries
- Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and
  match_fuzzy
- The `terms` query should always map to a Lucene `TermsQuery`.
- Be strict when parsing values searching for booleans
- Remove collect payloads parameter

REST::
- IndexClosedException to return 400 rather than 403
- Remove comma-separated feature parsing for GetIndicesAction
- Improve REST error handling when endpoint does not support HTTP
  verb, add OPTIONS support
- Remove ldjson support and document ndjson for bulk/msearch
- Enable strict duplicate checks for all XContent types
- Enable strict duplicate checks for JSON content
- Remove lenient stats parsing
- Remove allow unquoted JSON
- Remove FORCE version_type

Scripting::
- remove lang url parameter from stored script requests
- Disallow lang to be used with Stored Scripts
- Remove Deprecated Script Settings
- Scripting: Remove native scripts
- Scripting: Remove file scripts
- Make dates be ReadableDateTimes in scripts
- Remove groovy scripting language
- Remove script access to term statistics

Search::
- Make `index` in TermsLookup mandatory
- Removes FieldStats API
- Remove deprecated fielddata_fields from search request
- Removes deprecated fielddata_fields
- ProfileResult and CollectorResult should print machine readable
  timing information
- Remove indices query
- Remove ignored type parameter in search_shards api

Sequence IDs::
- Change certain replica failures not to fail the replica shard

Settings::
- Settings: Remove shared setting property
- Settings: Remove support for yaml and json config files

Shadow Replicas::
- Remove shadow replicas

Similarities::
- Similarity should accept dynamic settings when possible

=== Breaking Java changes

Aggregations::
- Remove the unused SignificantTerms.compareTerm() method
- Make SignificantTerms.Bucket an interface rather than an abstract
  class
- Fix NPE when `values` is omitted on percentile_ranks agg
- Make Terms.Bucket an interface rather than an abstract class
- Compound order for histogram aggregations

Internal::
- Collapses package structure for some bucket aggs

Java API::
- Remove deprecated IdsQueryBuilder ctor
- Removing unneeded getTookInMillis method
- Java api: ActionRequestBuilder#execute to return a PlainActionFuture

Java High Level REST Client::
- Unify the result interfaces from get and search in Java client
- Allow RestHighLevelClient to use plugins

Java REST Client::
- Rename client artifacts

Network::
- Simplify TransportAddress

Plugin Delete By Query::
- Move DeleteByQuery and Reindex requests into core

Plugins::
- Drop name from TokenizerFactory

Query DSL::
- Remove QueryParseContext
- Remove QueryParseContext from parsing QueryBuilders

REST::
- Return index name and empty map for `/{index}/_alias` with no
  aliases

=== Deprecations

Index APIs::
- Deprecated use of + in index expressions

Index Templates::
- Restore deprecation warning for invalid match_mapping_type values

Indexed Scripts/Templates::
- Scripting: Deprecate stored search template apis

Internal::
- Deprecate XContentType auto detection methods in XContentFactory

Percolator::
- Deprecate percolate query's document_type parameter.

Plugins::
- Plugins: Add backcompat for sha1 checksums

Scripting::
- Scripting: Change keys for inline/stored scripts to source/id
- Scripting: Deprecate native scripts
- Scripting: Deprecate index lookup
- Deprecate Fine Grain Settings for Scripts
- Scripting: Deprecate file script settings
- Scripting: Deprecate file scripts

Settings::
- Settings: Update settings deprecation from yml to yaml
- Deprecate settings in .yml and .json

Tribe Node::
- Deprecate tribe service

=== New features

Aggregations::
- SignificantText aggregation - like significant_terms, but for text

Analysis::
- Expose simplepattern and simplepatternsplit tokenizers
- Parse synonyms with the same analysis chain

Core::
- Enable index-time sorting

Internal::
- Automatically adjust search threadpool queue_size

Mapping::
- Add new ip_range field type

Parent/Child::
- Move parent_id query to the parent-join module
- Introduce ParentJoinFieldMapper, a field mapper that creates
  parent/child relation within documents of the same index

Plugin Analysis ICU::
- Add ICUCollationFieldMapper

Search::
- Automatically early terminate search query based on index sorting

Sequence IDs::
- Add a scheduled translog retention check
- Initialize sequence numbers on a shrunken index
- Initialize primary term for shrunk indices
- Introduce translog size and age based retention policies

Stats::
- Adds nodes usage API to monitor usages of actions

Task Manager::
- Task Management

Upgrade API::
- TemplateUpgraders should be called during rolling restart

=== Enhancements

Aggregations::
- Add strict parsing of aggregation ranges
- Adds rewrite phase to aggregations
- Tweak AggregatorBase.addRequestCircuitBreakerBytes
- Add superset size to Significant Term REST response
- Add document count to Matrix Stats aggregation response
- Adds an implementation of LogLogBeta for the cardinality aggregation
- Support distance units in GeoHashGrid aggregation precision
- Reject multiple methods in `percentiles` aggregation
- Use `global_ordinals_hash` execution mode when sorting by sub
  aggregations.
- Add a specialized deferring collector for terms aggregator
- Agg builder accessibility fixes
- Remove support for the include/pattern syntax.
- Promote longs to doubles when a terms agg mixes decimal and
  non-decimal numbers

Allocation::
- Adjust status on bad allocation explain requests
- Promote replica on the highest version node

Analysis::
- [Analysis] Support normalizer in request param
- Enforce validation for PathHierarchy tokenizer
- [analysis-icu] Allow setting unicodeSetFilter
- Match- and MultiMatchQueryBuilder should only allow setting analyzer
  on string values

Bulk::
- Simplify bulk request execution

CAT API::
- expand `/_cat/nodes` to return information about hard drive

CRUD::
- Added validation for upsert request

Circuit Breakers::
- ScriptService: Replace max compilation per minute setting with max
  compilation rate

Cluster::
- Validate a joining node's version with version of existing cluster
  nodes
- Switch indices read-only if a node runs out of disk space
- Add a cluster block that allows to delete indices that are read-only
- Separate publishing from applying cluster states
- Adds cluster state size to /_cluster/state response

Core::
- Allow `InputStreamStreamInput` array size validation where
  applicable
- Refactor bootstrap check results and error messages
- Add BootstrapContext to expose settings and recovered state to
  bootstrap checks
- Unit testable index creation task on MetaDataCreateIndexService
- Ignore .DS_Store files on macOS
- Add max file size bootstrap check
- Add compatibility versions to main action response
- Index ids in binary form.
- Explicitly reject duplicate data paths
- Use SPI in High Level Rest Client to load XContent parsers
- Upgrade to lucene-7.0.0-snapshot-a0aef2f
- Speed up PK lookups at index time.
- Use Java 9 FilePermission model
- Add friendlier message on bad keystore permissions
- Epoch millis and second formats accept float implicitly
- Remove connect SocketPermissions from core
- Add repository-url module and move URLRepository
- Remove accept SocketPermissions from core
- Move IfConfig.logIfNecessary call into bootstrap
- Remove artificial default processors limit
- Simplify write failure handling
- Improve bootstrap checks error messages

Discovery::
- Allow plugins to validate cluster-state on join

Engine::
- Add refresh stats tracking for realtime get
- Introducing a translog deletion policy
- Fill missing sequence IDs up to max sequence ID when recovering from
  store
- Use sequence numbers to identify out of order delivery in replicas &
  recovery
- Add replica ops with version conflict to translog
- Clarify global checkpoint recovery
- Move the IndexDeletionPolicy to be engine internal

Exceptions::
- IllegalStateException: Only duplicated jar instead of classpath

Highlighting::
- Picks offset source for the unified highlighter directly from the es
  mapping

Index APIs::
- Let primary own its replication group
- Create index request should return the index name

Index Templates::
- Fix error message for a put index template request without
  index_patterns

Ingest::
- Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint
- Port support for commercial GeoIP2 databases from Logstash.
- add `exclude_keys` option to KeyValueProcessor
- Allow removing multiple fields in ingest processor
- Add target_field parameter to ingest processors

Inner Hits::
- Reuse inner hit query weight

Internal::
- TemplateUpgradeService should only run on the master
- Cleanup IndexFieldData visibility
- Bump the min compat version to 5.6.0
- "shard started" should show index and shard ID
- Break out clear scroll logic from TransportClearScrollAction
- Add helper methods to TransportActionProxy to identify proxy actions
  and requests
- Add remote cluster infrastructure to fetch discovery nodes.
- Add the ability to set eager_global_ordinals in the new parent-join
  field
- Disallow multiple parent-join fields per mapping
- Remove the need for _UNRELEASED suffix in versions
- Optimize the order of bytes in uuids for better compression.
- Prevent cluster internal `ClusterState.Custom` impls to leak to a
  client
- Use holder pattern for lazy deprecation loggers
- Allow `ClusterState.Custom` to be created on initial cluster states
- Try to convince the JVM not to lose stacktraces
- Make document write requests immutable
- Add assertions enabled helper

Java API::
- Always Accumulate Transport Exceptions

Java High Level REST Client::
- [DOCS] restructure java clients docs pages
- Use SPI in High Level Rest Client to load XContent parsers
- Add support for clear scroll to high level REST client
- Add search scroll method to high level REST client
- Add search method to high level REST client
- Make RestHighLevelClient Closeable and simplify its creation
- Add info method to High Level Rest client
- Add support for named xcontent parsers to high level REST client
- Add BulkRequest support to High Level Rest client
- Add UpdateRequest support to High Level Rest client
- Add delete API to the High Level Rest Client
- Add Index API to High Level Rest Client
- Add get/exists method to RestHighLevelClient
- Add fromxcontent methods to delete response
- Add REST high level client gradle submodule and first simple method
- Add doc_count to ParsedMatrixStats
- Add fromXContent method to ClearScrollResponse
- ClearScrollRequest to implement ToXContentObject
- SearchScrollRequest to implement ToXContentObject
- Add aggs parsers for high level REST Client

Java REST Client::
- Shade external dependencies in the rest client jar
- RestClient uses system properties and system default SSLContext
- Wrap rest httpclient with doPrivileged blocks

Logging::
- Prevent excessive disk consumption by log files
- Use LRU set to reduce repeat deprecation messages

Mapping::
- More efficient encoding of range fields.
- Don't detect source's XContentType in DocumentParser.parseDocument()
- Better validation of `copy_to`.
- Optimize `terms` queries on `ip` addresses to use a
  `PointInSetQuery` whenever possible.
- Loosen the restrictions on disabling _all in 6.x
- Date detection should not rely on a hardcoded set of characters.
- Identify documents by their `_id`.

Network::
- Add additional low-level logging handler
- Unwrap causes when maybe dying
- Move TransportStats accounting into TcpTransport
- Simplify connection closing and cleanups in TcpTransport
- Disable the Netty recycler in the client
- Remove Netty logging hack
- Isolate SocketPermissions to Netty
- Wrap netty accept/connect ops with doPrivileged
- Replace Socket, ServerSocket, and HttpServer usages in tests with
  mocksocket versions

Packaging::
- Remove memlock suggestion from systemd service
- Set address space limit in systemd service file
- Version option should display if snapshot
- Ignore JVM options before checking Java version
- Also skip JAVA_TOOL_OPTIONS on Windows
- Introduce elasticsearch-env for Windows
- Introduce elasticsearch-env
- Stop exporting HOSTNAME from scripts
- Set number of processes in systemd unit file

Parent/Child::
- Remove ParentJoinFieldSubFetchPhase
- Support parent id being specified as number in the _source

Percolator::
- Store the QueryBuilder's Writable representation instead of its
  XContent representation
- Add support for selecting percolator query candidate matches
  containing wildcard / prefix queries

Plugin Discovery EC2::
- Read ec2 discovery address from aws instance tags

Plugin Lang Painless::
- Allow Custom Whitelists in Painless
- Update Painless to Allow Augmentation from Any Class
- Add Needs Methods to Painless Script Context Factories
- Support Script Context Stateful Factory in Painless
- Generate Painless Factory for Creating Script Instances
- Update Painless to Use New Script Contexts
- Optimize instance creation in LambdaBootstrap
- Make Painless Compiler Use an Instance Per Context
- Make PainlessScript An Interface

Plugin Repository GCS::
- GCS Repository: Add secure storage of credentials

Plugin Repository HDFS::
- Add permission checks before reading from HDFS stream
- Add doPrivilege blocks for socket connect ops in repository-hdfs
- Add Kerberos support for Repo HDFS plugin

Plugin Repository S3::
- S3 Repository: Add back repository level credentials

Plugins::
- Adjust SHA-512 supported format on plugin install
- Move tribe to a module
- Plugins can register pre-configured char filters
- Add purge option to remove plugin CLI
- Allow plugins to register pre-configured tokenizers
- Move ReindexAction class to core
- Make PreConfiguredTokenFilter harder to misuse
- Plugins: Remove leniency for missing plugins dir
- Add doPrivilege blocks for socket connect operations in plugins

Query DSL::
- Make slop optional when parsing `span_near` query
- Require a field when a `seed` is provided to the `random_score`
  function.
- Add support for auto_generate_synonyms_phrase_query in match_query,
  multi_match_query, query_string and simple_query_string

REST::
- Cat shards bytes
- Refactor PathTrie and RestController to use a single trie for all
  methods
- Make ObjectParser support string to boolean conversion

Recovery::
- Introduce a History UUID as a requirement for ops based recovery
- Goodbye, Translog Views
- Disallow multiple concurrent recovery attempts for same target shard
- Live primary-replica resync (no rollback)
- Peer Recovery: remove maxUnsafeAutoIdTimestamp hand off
- Introduce sequence-number-based recovery

Scripting::
- Scripting: Rename SearchScript.needsScores to needs_score
- Scripting: Add optional context parameter to put stored script
  requests
- Add New Security Script Settings
- Add StatefulFactoryType as optional intermediate factory in script
  contexts
- Make contexts available to ScriptEngine construction
- Make ScriptEngine.compile generic on the script context
- Add instance and compiled classes to script contexts

Search::
- Add soft limit on allowed number of script fields in request
- Add a soft limit for the number of requested doc-value fields
- Rewrite search requests on the coordinating nodes
- Ensure query resources are fetched asynchronously during rewrite
- Introduce a new Rewriteable interface to streamline rewriting
- Reduce the scope of `QueryRewriteContext`
- Reduce the overhead of timeouts and low-level search cancellation.
- Reduce profiling overhead.
- Prevent `can_match` requests from sending to incompatible nodes
- Add a shard filter search phase to pre-filter shards based on query
  rewriting
- Ensure we rewrite common queries to `match_none` if possible
- Limit the number of concurrent shard requests per search request
- Add cluster name validation to RemoteClusterConnection
- Speed up sorted scroll when the index sort matches the search sort
- Leverage scorerSupplier when applicable.
- Add Cross Cluster Search support for scroll searches
- Track EWMA[1] of task execution time in search threadpool executor
- Query range fields by doc values when they are expected to be more
  efficient than points
- Search: Fairer balancing when routing searches by session ID
-  Add parsing from xContent to Suggest
- Add parsing from xContent to ShardSearchFailure
- Eliminate array access in tight loops when profiling is enabled.
- Support Multiple Inner Hits on a Field Collapse Request
- Expand cross cluster search indices for search requests to the
  concrete index or to it's aliases

Search Templates::
- Add max concurrent searches to multi template search

Sequence IDs::
- Roll translog generation on primary promotion
- Restoring from snapshot should force generation of a new history
  uuid
- Add global checkpoint tracking on the primary
- Introduce global checkpoint background sync
- Move `UNASSIGNED_SEQ_NO` and `NO_OPS_PERFORMED` to SequenceNumbers`
- Move primary term from ReplicationRequest to ConcreteShardRequest
- Add reason to global checkpoint updates on replica
- Introduce primary/replica mode for GlobalCheckPointTracker
- Throw back replica local checkpoint on new primary
- Update global checkpoint when increasing primary term on replica
- Enable a long translog retention policy by default
- Introduce primary context
- Block older operations on primary term transition
- Block global checkpoint advances when recovering
- Add primary term to doc write response
- Preserve multiple translog generations
- Introduce translog generation rolling
- Replicate write failures
- Introduce sequence-number-aware translog
- Introduce translog no-op
- Tighten sequence numbers recovery
- Add BWC layer to seq no infra and enable BWC tests
- Add internal _primary_term doc values field, fix _seq_no indexing
- Add global checkpoint to translog checkpoints
- Sequence numbers commit data for Lucene uses Iterable interface
- Simplify GlobalCheckpointService and properly hook it for cluster
  state updates
- Fill gaps on primary promotion
- Introduce clean transition on primary promotion
- Guarantee that translog generations are seqNo conflict free
- Inline global checkpoints

Settings::
- Add disk threshold settings validation
- Enable cross-setting validation
- Validate `transport.profiles.*` settings
- Cleanup network / transport related settings
- Emit settings deprecation logging at most once
- IndexMetaData: Introduce internal format index setting
- Persist created keystore on startup unless keystore is present
- Settings: Add keystore.seed auto generated secure setting
- Settings: Add keystore creation to add commands

Snapshot/Restore::
- Fixed references to Multi Index Syntax
- Improves snapshot logging and snapshot deletion error handling
- Enhances get snapshots API to allow retrieving repository index only

Stats::
- Update `IndexShard#refreshMetric` via a
  `ReferenceManager.RefreshListener`
- Expose disk usage estimates in nodes stats

Store::
- Remote support for lucene versions without checksums

Suggesters::
- Remove deprecated _suggest endpoint

Task Manager::
- Add descriptions to bulk tasks

Translog::
- Translog file recovery should not rely on lucene commits

=== Bug fixes

Aggregations::
- Do not delegate a null scorer to LeafBucketCollectors
- Create weights lazily in filter and filters aggregation
- Fix IndexOutOfBoundsException in histograms for NaN doubles (#26787)
- Scripted_metric _agg parameter disappears if params are provided
- Fixes array out of bounds for value count agg
- Aggregations bug: Significant_text fails on arrays of text.
- Check bucket metric ages point to a multi bucket agg
- Terms aggregation should remap global ordinal buckets when a
  sub-aggregator is used to sort the terms
- Correctly set doc_count when MovAvg "predicts" values on existing
  buckets
- DateHistogram: Fix `extended_bounds` with `offset`
- Fix ArrayIndexOutOfBoundsException when no ranges are specified in
  the query

Aliases::
- mget with an alias shouldn't ignore alias routing
- GET aliases should 404 if aliases are missing

Allocation::
- Fix DiskThresholdMonitor flood warning
- Allow wildcards for shard IP filtering

Analysis::
- Pre-configured shingle filter should disable graph analysis
- PatternAnalyzer should lowercase wildcard queries when `lowercase`
  is true.

CAT API::
- Fix NPE for /_cat/indices when no primary shard

CRUD::
- Serialize and expose timeout of acknowledged requests in REST layer
- Fix silent loss of last command to _bulk and _msearch due to missing
  newline

Cache::
- Reduce the default number of cached queries.
- fix bug of weight computation

Circuit Breakers::
- Checks the circuit breaker before allocating bytes for a new big
  array

Cluster::
- Register setting `cluster.indices.tombstones.size`

Core::
- Correctly encode warning headers
- Fix cache compute if absent for expired entries
- Timed runnable should delegate to abstract runnable
- Stop invoking non-existent syscall
- MetaData Builder doesn't properly prevent an alias with the same
  name as an index
- Release operation permit on thread-pool rejection
- Node should start up despite of a lingering `.es_temp_file`
- Fix cache expire after access

Dates::
- Fix typo in date format

Discovery::
- MasterNodeChangePredicate should use the node instance to detect
  master change

Engine::
- Die with dignity while merging
- Engine - do not index operations with seq# lower than the local
  checkpoint into lucene

Geo::
- Fix typo in GeoUtils#isValidLongitude

Highlighting::
- Fix percolator highlight sub fetch phase to not highlight query
  twice
- FastVectorHighlighter should not cache the field query globally
- Higlighters: Fix MultiPhrasePrefixQuery rewriting
- Fix nested query highlighting

Index APIs::
- Shrink API should ignore templates
- Rollover max docs should only count primaries
- Validates updated settings on closed indices

Ingest::
- date processor should not fail if timestamp is specified as json
  number
- date_index_name processor should not fail if timestamp is specified
  as json number
- Sort Processor does not have proper behavior with targetField
- fix grok's pattern parsing to validate pattern names in expression
- Remove support for Visio and potm files
- Fix floating-point error when DateProcessor parses UNIX
- add option for _ingest.timestamp to use new ZonedDateTime (5.x
  backport)

Inner Hits::
- Do not allow inner hits that fetch _source and have a non nested
  object field as parent
- When fetching nested inner hits only access stored fields when
  needed
- If size / offset are out of bounds just do a plain count
- Fix Source filtering in new field collapsing feature

Internal::
- Bump version to 6.0.1
- `IndexShard.routingEntry` should only be updated once all internal
  state is ready
- Catch exceptions and inform handler in
  RemoteClusterConnection#collectNodes
- Internal: Add versionless alias for rest client codebase in policy
  files
- Upgrade Lucene to version 7.0.1
- Fix BytesReferenceStreamInput#skip with offset
- Fix race condition in RemoteClusterConnection node supplier
- Initialise empty lists in BaseTaskResponse constructor
- Extract a common base class for scroll executions
- Obey lock order if working with store to get metadata snapshots
- Fix Version based BWC and set correct minCompatVersion
- Fix `_field_caps` serialization in order to support cross cluster
  search
- Avoid race when shutting down controller processes
- Fix handling of document failure exception in InternalEngine
- Ensure remote cluster is connected before fetching `_field_caps`

Java API::
- BulkProcessor flush runnable preserves the thread context from
  creation time

Java High Level REST Client::
- Make RestHighLevelClient's Request class public
- Forbid direct usage of ContentType.create() methods
- Make ShardSearchTarget optional when parsing ShardSearchFailure

Java REST Client::
- Better message text for ResponseException
- rest-client-sniffer: configurable threadfactory

Logging::
- Allow not configure logging without config

Mapping::
- Allow copying from a field to another field that belongs to the same
  nested object.
- Fixed bug that mapper_parsing_exception is thrown for numeric field
  with ignore_malformed=true when inserting "NaN"
- Coerce decimal strings for whole number types by truncating the
  decimal part
- Fix parsing of ip range queries.
- Disable date field mapping changing
- Correctly enable _all for older 5.x indices
- token_count datatype should handle null value
- keep _parent field while updating child type mapping
- ICUCollationKeywordFieldMapper use SortedSetDocValuesField
- Fix serialization of the `_all` field.

More Like This::
- Pass over _routing value with more_like_this items to be retrieved

NOT CLASSIFIED::
- DocumentMissingException during Logstash scripted upsert
- An assertion trips when master opens an index from before 5.x

Nested Docs::
- In case of a single type the _id field should be added to the nested
  document instead of _uid field
- Inner hits source filtering not working

Network::
- Fixed ByteBuf leaking in
  org.elasticsearch.http.netty4.Netty4HttpRequestHandler
- Check for closed connection while opening
- Ensure pending transport handlers are invoked for all channel
  failures
- Notify onConnectionClosed rather than onNodeDisconnect to prune
  transport handlers
- Release pipelined http responses on close
- Fix error message if an incompatible node connects

Packaging::
- Fix handling of Windows paths containing parentheses
- Exit Windows scripts promptly on failure
- Pass config path as a system property
- ES_HOME needs to be made absolute before attempt at traversal
- Fix elasticsearch-keystore handling of path.conf
- Stop disabling explicit GC
- Avoid failing install if system-sysctl is masked
- Get short path name for native controllers
- When stopping via systemd only kill the JVM, not its control group
- remove remaining references to scripts directory
- Handle parentheses in batch file path
- Detect modified keystore on package removal
- Create keystore on RPM and Debian package install
- Add safer empty variable checking for Windows
- Export HOSTNAME environment variable
- Fix daemonization command status test
- Set RuntimeDirectory in systemd service

Parent/Child::
- The default _parent field should not try to load global ordinals

Percolator::
- Also support query extraction for queries wrapped inside a
  ESToParentBlockJoinQuery
- Fix range queries with date range based on current time in
  percolator queries.

Plugin Analysis Kuromoji::
- Fix kuromoji default stoptags

Plugin Analysis Phonetic::
- Fix beidermorse phonetic token filter for unspecified `languageset`

Plugin Discovery File::
- Fix discovery-file plugin to use custom config path

Plugin Ingest Attachment::
- Add missing mime4j library

Plugin Lang Painless::
- Painless: allow doubles to be casted to longs.

Plugin Repository Azure::
- Azure snapshots can not be restored anymore
- Snapshot : azure module - accelerate the listing of files (used in
  delete snapshot)
- Use Azure upload method instead of our own implementation
- Make calls to CloudBlobContainer#exists privileged

Plugin Repository GCS::
- Ensure that gcs client creation is privileged

Plugin Repository HDFS::
- Add Log4j to SLF4J binding for repository-hdfs
- Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client

Plugin Repository S3::
- Avoid SecurityException in repository-S3 on
  DefaultS3OutputStream.flush()
- Wrap getCredentials() in a doPrivileged() block

Plugins::
- X-Pack plugin download fails on Windows desktop
- Fix plugin installation permissions

Query DSL::
- Fixed incomplete JSON body on count request making
  org.elasticsearch.rest.action.RestActions#parseTopLevelQueryBuilder
go into endless loop
- SpanNearQueryBuilder should return the inner clause when a single
  clause is provided
- Refactor field expansion for match, multi_match and query_string
  query
- WrapperQueryBuilder should also rewrite the parsed query

REST::
- Rest test fixes
- Fix inconsistencies in the rest api specs for cat.snapshots
- Fix inconsistencies in the rest api specs for *_script
- exists template needs a template name
- Fix handling of invalid error trace parameter
- Fix handling of exceptions thrown on HEAD requests
- Fixed NPEs caused by requests without content.
- Fix get mappings HEAD requests

Recovery::
- Close translog view after primary-replica resync

Reindex API::
- Fix update_by_query's default size parameter
- Reindex: don't duplicate _source parameter
- Add qa module that tests reindex-from-remote against pre-5.0
  versions of Elasticsearch

Scroll::
- Fix single shard scroll within a cluster with nodes in version `>=
  5.3` and `<= 5.3`

Search::
- Fail query when a sort is provided in conjunction with rescorers
- Let search phases override max concurrent requests
- Avoid stack overflow on search phases
- Fix search_after with geo distance sorting
- Fix serialization errors when cross cluster search goes to a single
  shard
- Early termination with index sorting should not set terminated_early
  in the response
- Format doc values fields.
- Fix term(s) query for range field
- Caching a MinDocQuery can lead to wrong results.
- Fix random score generation when no seed is provided.
- Merge FunctionScoreQuery and FiltersFunctionScoreQuery
- Respect cluster alias in `_index` aggs and queries
- First increment shard stats before notifying and potentially sending
  response
- Remove assertion about deviation when casting to a float.
- Prevent skipping shards if a suggest builder is present
- Ensure remote cluster alias is preserved in inner hits aggs
- Do not search locally if remote index pattern resolves to no indices
- Adds check for negative search request size
- Make sure range queries are correctly profiled.
- Fix RangeFieldMapper rangeQuery to properly handle relations
- Fix ExpandSearchPhase when response contains no hits
- Refactor simple_query_string to handle text part like multi_match
  and query_string
- Fix `_exists_` in query_string on empty indices.
- Fix script field sort returning Double.MAX_VALUE for all documents
- Compute the took time of the query after the expand phase of field
  collapsing

Sequence IDs::
- Fire global checkpoint sync under system context
- Fix pre-6.0 response to unknown replication actions
- Track local checkpoint on primary immediately
- Initialize max unsafe auto ID timestamp on shrink
- Use correct primary term for replicating NOOPs
- Handle already closed while filling gaps
- TranslogWriter.assertNoSeqNumberConflict failure
- Avoid losing ops in file-based recovery
- Handle primary failure handling replica response

Settings::
- Emit settings deprecation logging on empty update
- Fix filtering for ListSetting
- Fix settings serialization to not serialize secure settings or not
  take the total size into account
- Keystore CLI should use the AddFileKeyStoreCommand for files
- Allow resetting settings that use an IP validator
- Updating an unrecognized setting should error out with that reason
- Settings: Fix setting groups to include secure settings

Similarities::
- Add boolean similarity to built in similarity types

Snapshot/Restore::
- Snapshot/Restore: better handle incorrect chunk_size settings in FS
  repo
- Snapshot/Restore: Ensure that shard failure reasons are correctly
  stored in CS
- Output all empty snapshot info fields if in verbose mode
- Remove redundant and broken MD5 checksum from repository-s3
- Consolidates the logic for cleaning up snapshots on master election
- Removes completed snapshot from cluster state on master change
- Keep snapshot restore state and routing table in sync
- Master failover during snapshotting could leave the snapshot
  incomplete
- Fix inefficient (worst case exponential) loading of snapshot
  repository

Stats::
- Fix RestGetAction name typo
- Keep cumulative elapsed scroll time in microseconds
- _nodes/stats should not fail due to concurrent
  AlreadyClosedException
- Avoid double decrement on current query counter
- Adjust available and free bytes to be non-negative on huge FSes

Suggesters::
- Fix division by zero in phrase suggester that causes assertion to
  fail
- Context suggester should filter doc values field
- Fix context suggester to read values from keyword type field

Templates::
- Tests: Fix FullClusterRestartIT.testSnapshotRestore test failing in
  6.x

Translog::
- Fix Translog.Delete serialization for sequence numbers

Upgrade API::
- Upgrade API: fix excessive logging and unnecessary template updates

=== Regressions

Bulk::
- Only re-parse operation if a mapping update was needed

Highlighting::
- Fix Fast Vector Highlighter NPE on match phrase prefix

Search::
- Always use DisjunctionMaxQuery to build cross fields disjunction

Sequence IDs::
- Indexing performance degradation in 6.0.0-beta1

=== Upgrades

Core::
- Upgrade to Lucene 7.0.0

Logging::
- Upgrade to Log4j 2.9.1

Network::
- Upgrade to Netty 4.1.13.Final

Plugin Ingest Attachment::
- Update to Tika 1.14

Upgrade API::
- Improve stability and logging of TemplateUpgradeServiceIT tests
   2017-10-30 12:27:32 by Filip Hajny | Files touched by this commit (1)
Log message:
databases/elasticsearch: PKG_JAVA_HOME needs quoting on at least Darwin
   2017-10-17 13:27:05 by Filip Hajny | Files touched by this commit (2) | Package updated
Log message:
Update databases/elasticsearch to 5.6.3.

=== Enhancements

Network::
* Add additional low-level logging handler
* Unwrap exception causes when checking if  dying

=== Bug fixes

Aggregations::
* Fix IndexOutOfBoundsException in histograms for NaN doubles

Core::
* MetaData Builder doesn't properly prevent an alias with the same
* name as an index

Packaging::
* Fix default paths for Windows service

Plugin Repository Azure::
* Use Azure upload method instead of our own implementation

Search::
* Fix serialization errors when cross cluster search goes to a single
* shard