Log Message:
Update databases/elasticsearch to 6.1.0.

== 6.1.0 Release Notes

=== Breaking changes

Network::
- Allow only a fixed-size receive predictor

REST::
- Standardize underscore requirements in parameters

Scroll::
- Fail queries with scroll that explicitely set request_cache

Search::
- Add a limit to from + size in top_hits and inner hits.

=== Breaking Java changes

Aggregations::
- Moves deferring code into its own subclass

Core::
- Unify Settings xcontent reading and writing

Settings::
- Return List instead of an array from settings
- Remove `Settings,put(Map<String,String>)`

=== Deprecations

Aggregations::
- Deprecate global_ordinals_hash and global_ordinals_low_cardinality

Allocation::
- Add deprecation warning for negative
  index.unassigned.node_left.delayed_timeout

Analysis::
- Add limits for ngram and shingle settings

Geo::
- [GEO] 6x Deprecate ShapeBuilders and decouple geojson parse logic

Mapping::
- Deprecate the `index_options` parameter for numeric fields

Plugin Repository Azure::
- Azure repository: Move to named configurations as we do for S3
  repository and secure settings

Search::
- doc: deprecate _primary and _replica shard option

=== New features

Aggregations::
- Aggregations: bucket_sort pipeline aggregation
- Add composite aggregator

Analysis::
- Added Bengali Analyzer to Elasticsearch with respect to the lucene
  update

Ingest::
- add URL-Decode Processor to Ingest

Java High Level REST Client::
- Added Delete Index support to high-level REST client

Nested Docs::
- Multi-level Nested Sort with Filters

Query DSL::
- Add terms_set query
- Introduce sorted_after query for sorted index
- Add support for auto_generate_synonyms_phrase_query in match_query,
  multi_match_query, query_string and simple_query_string

Search::
- Expose `fuzzy_transpositions` parameter in fuzzy queries
- Add upper limit for scroll expiry
- Implement adaptive replica selection
- configure distance limit

Similarities::
- Add a scripted similarity.

Suggesters::
- Expose duplicate removal in the completion suggester
- Support must and should for context query in context suggester

=== Enhancements

Aggregations::
- Allow aggregation sorting via nested aggregation

Allocation::
- Tie-break shard path decision based on total number of shards on
  path
- Balance shards for an index more evenly across multiple data paths
- Expand "NO" decision message in NodeVersionAllocationDecider
- _reroute's retry_failed flag should reset failure counter

Analysis::
- Add configurable `max_token_length` parameter to whitespace
  tokenizer

CRUD::
- Add wait_for_active_shards parameter to index open command

Core::
- Fix classes that can exit
- Replace empty index block checks with global block checks in
  template delete/put actions
- Allow Uid#decodeId to decode from a byte array slice
- Use separate searchers for "search visibility" vs "move indexing
  buffer to disk
- Add ability to split shards
- Make circuit breaker mutations debuggable

Dates::
- DateProcessor Locale

Discovery::
- Stop responding to ping requests before master abdication

Engine::
- Ensure external refreshes will also refresh internal searcher to
  minimize segment creation
- Move IndexShard#getWritingBytes() under InternalEngine
- Refactor internal engine

Geo::
- Add ignore_malformed to geo_shape fields

Ingest::
- add json-processor support for non-map json types
- Introduce templating support to timezone/locale in DateProcessor
- Add support for parsing inline script (#23824)
- Consolidate locale parsing.
- Accept ingest simulate params as ints or strings

Internal::
- Avoid uid creation in ParsedDocument
- Upgrade to Lucene 7.1.0 snapshot version
- Remove `_index` fielddata hack if cluster alias is present

Java High Level REST Client::
- Adjust RestHighLevelClient method modifiers
- Decouple BulkProcessor from ThreadPool

Logging::
- Add more information on _failed_to_convert_ exception (#21946)
- Improve shard-failed log messages.

Mapping::
- Allow ip_range to accept CIDR notation
- Deduplicate `_field_names`.
- Throw a better error message for empty field names
- Stricter validation for min/max values for whole numbers
- Make FieldMapper.copyTo() always non-null.

Nested Docs::
- Use the primary_term field to identify parent documents
- Prohibit using `nested_filter`, `nested_path` and new `nested`
  Option at the same time in FieldSortBuilder

Network::
- Remove manual tracking of registered channels
- Remove tcp profile from low level nio channel
- Decouple `ChannelFactory` from Tcp classes

Percolator::
- Use Lucene's CoveringQuery to select percolate candidate matches
- Add support to percolate query to percolate multiple documents
  simultaneously
- Hint what clauses are important in a conjunction query based on
  fields
- Add support for selecting percolator query candidate matches
  containing range queries

Plugin Discovery EC2::
- update AWS SDK for ECS Task IAM support in discovery-ec2

Plugin Lang Painless::
- Painless: Only allow Painless type names to be the same as the
  equivalent Java class.
- Allow for the Painless Definition to have multiple instances for
  white-listing
- Separate Painless Whitelist Loading from the Painless Definition
- Remove Sort enum from Painless Definition

Plugin Repository Azure::
- Add azure storage endpoint suffix #26432
- Support for accessing Azure repositories through a proxy

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Plugins::
- Plugins: Add versionless alias to all security policy codebase
  properties
- Allow plugins to plug rescore implementations

Query DSL::
- Add support for wildcard on `_index`

Reindex API::
- Update by Query is modified to accept short `script` parameter.
- reindex: automatically choose the number of slices

Rollover::
- Add size-based condition to the index rollover API
- Add size-based condition to the index rollover API

Scripting::
- Script: Convert script query to a dedicated script context

Search::
- Make fields optional in multi_match query and rely on
  index.query.default_field by default
- fix unnecessary logger creation
- `ObjectParser` : replace `IllegalStateException` with
  `ParsingException`
- Uses norms for exists query if enabled
- Cross Cluster Search: make remote clusters optional
- Enhances exists queries to reduce need for `_field_names`
- Change ParentFieldSubFetchPhase to create doc values iterator once
  per segment
- Change VersionFetchSubPhase to create doc values iterator once per
  segment
- Change ScriptFieldsFetchSubPhase to create search scripts once per
  segment
- Make sure SortBuilders rewrite inner nested sorts
- Extend testing of build method in ScriptSortBuilder
- Accept an array of field names and boosts in the
  index.query.default_field setting
- Reject IPv6-mapped IPv4 addresses when using the CIDR notation.
- Rewrite range queries with open bounds to exists query

Sequence IDs::
- Only fsync global checkpoint if needed
- Log primary-replica resync failures
- Lazy initialize checkpoint tracker bit sets
- Returns the current primary_term for Get/MultiGet requests

Settings::
- Allow affix settings to specify dependencies
- Represent lists as actual lists inside Settings
- Remove Settings#getAsMap()
- Replace group map settings with affix setting
- Throw exception if setting isn't recognized
- Settings: Move keystore creation to plugin installation

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository
- Snapshot: Migrate TransportRequestHandler to
  TransportMasterNodeAction
- Fix toString of class SnapshotStatus (#26851)

Stats::
- Adds average document size to DocsStats
- Stats to record how often the ClusterState diff mechanism is used
  successfully
- Expose adaptive replica selection stats in /_nodes/stats API
- Add cgroup memory usage/limit to OS stats on Linux
- Add segment attributes to the `_segments` API.

Suggesters::
- Improve error message for parse failures of completion fields
- Support 'AND' operation for context query in context suggester

=== Bug fixes

Aggregations::
- Disable the "low cardinality" optimization of terms aggregations.
- scripted_metric _agg parameter disappears if params are provided

Cluster::
- Properly format IndexGraveyard deletion date as date
-  Remove optimisations to reuse objects when applying a new
   `ClusterState`

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Ensure logging is configured for CLI commands
- Protect shard splitting from illegal target shards
- Avoid NPE when getting build information
- Fix `ShardSplittingQuery` to respect nested documents.
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Geo::
- Correct two equality checks on incomparable types
- [GEO] fix pointsOnly bug for MULTIPOINT

Index Templates::
- Prevent constructing an index template without index patterns

Ingest::
- Add pipeline support for REST API bulk upsert
- Fixing Grok pattern for Apache 2.4

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Internal::
- When checking if key exists in ThreadContextStruct:putHeaders()
  method, should put requestHeaders in map first
- Adding a refresh listener to a recovering shard should be a noop

Java High Level REST Client::
- Register ip_range aggregation with the high level client
- add top hits as a parsed aggregation to the rest high level client

Mapping::
- Fix dynamic mapping update generation.
- Fix merging of _meta field
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids
- Prevent duplicate fields when mixing parent and root nested includes

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams
- Do not set SO_LINGER on server channels
- Do not set SO_LINGER to 0 when not shutting down
- Close TcpTransport on RST in some Spots to Prevent Leaking TIME_WAIT
  Sockets

Packaging::
- Removes minimum master nodes default number
- setgid on /etc/elasticearch on package install

Percolator::
- Avoid TooManyClauses exception if number of terms / ranges is
  exactly equal to 1024

Plugin Analysis ICU::
- Catch InvalidPathException in IcuCollationTokenFilterFactory

Plugin Lang Painless::
- Painless: Fix variable scoping issue in lambdas
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Scroll::
- Fix scroll query with a sort that is a prefix of the index sort

Search::
- Fix profiling naming issues
- Fix max score tracking with field collapsing
- Apply missing request options to the expand phase
- Calculate and cache result when advanceExact is called
- Filter unsupported relation for RangeQueryBuilder
- Handle leniency for phrase query on a field indexed without
  positions

Sequence IDs::
- Obey translog durability in global checkpoint sync
- Fix resync request serialization

Settings::
- Allow index settings to be reset by wildcards

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
- Do not create directory on readonly repository (#21495)

Stats::
- Include internal refreshes in refresh stats
- Make Segment statistics aware of segments hold by internal readers
- Ensure `doc_stats` are changing even if refresh is disabled

=== Upgrades

Core::
- Upgrade to Jackson 2.8.10
- Upgrade to Lucene 7.1

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0

Plugin Lang Painless::
- Upgrade Painless from ANTLR 4.5.1-1 to  ANTLR 4.5.3.

== 6.0.1 Release Notes

=== Breaking changes

Scroll::
- Fail queries with scroll that explicitely set request_cache

=== Enhancements

Core::
- Fix classes that can exit

Discovery::
- Stop responding to ping requests before master abdication

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Search::
- fix unnecessary logger creation

Sequence IDs::
- Log primary-replica resync failures

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository

=== Bug fixes

Cluster::
- Properly format IndexGraveyard deletion date as date

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Avoid NPE when getting build information
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Mapping::
- Fix dynamic mapping update generation.
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams

Plugin Lang Painless::
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Search::
- Fix profiling naming issues

Sequence IDs::
- Fix resync request serialization

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
  (#25359)
- Do not create directory on readonly repository (#21495)

=== Upgrades

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0