Log Message:
mimir: update to 2.15.0.

## 2.15.0

### Grafana Mimir

* [CHANGE] Alertmanager: the following metrics are not exported for a given \ 
`user` when the metric value is zero: #9359
  * `cortex_alertmanager_alerts_received_total`
  * `cortex_alertmanager_alerts_invalid_total`
  * `cortex_alertmanager_partial_state_merges_total`
  * `cortex_alertmanager_partial_state_merges_failed_total`
  * `cortex_alertmanager_state_replication_total`
  * `cortex_alertmanager_state_replication_failed_total`
  * `cortex_alertmanager_alerts`
  * `cortex_alertmanager_silences`
* [CHANGE] Distributor: Drop experimental \ 
`-distributor.direct-otlp-translation-enabled` flag, since direct OTLP \ 
translation is well tested at this point. #9647
* [CHANGE] Ingester: Change `-initial-delay` for circuit breakers to begin when \ 
the first request is received, rather than at breaker activation. #9842
* [CHANGE] Query-frontend: apply query pruning before query sharding instead of \ 
after. #9913
* [CHANGE] Ingester: remove experimental flags \ 
`-ingest-storage.kafka.ongoing-records-per-fetch` and \ 
`-ingest-storage.kafka.startup-records-per-fetch`. They are removed in favour of \ 
`-ingest-storage.kafka.max-buffered-bytes`. #9906
* [CHANGE] Ingester: Replace `cortex_discarded_samples_total` label from \ 
`sample-out-of-bounds` to `sample-timestamp-too-old`. #9885
* [CHANGE] Ruler: the `/prometheus/config/v1/rules` does not return an error \ 
anymore if a rule group is missing in the object storage after been successfully \ 
returned by listing the storage, because it could have been deleted in the \ 
meanwhile. #9936
* [CHANGE] Querier: The `.` pattern in regular expressions in PromQL matches \ 
newline characters. With this change regular expressions like `.*` match strings \ 
that include `\n`. To maintain the old behaviour, you will have to change \ 
regular expressions by replacing all `.` patterns with `[^\n]`, e.g. \ 
`foo[^\n]*`. This upgrades PromQL compatibility from Prometheus 2.0 to 3.0. \ 
#9844
* [CHANGE] Querier: Lookback and range selectors are left open and right closed \ 
(previously left closed and right closed). This change affects queries when the \ 
evaluation time perfectly aligns with the sample timestamps. For example assume \ 
querying a timeseries with evenly spaced samples exactly 1 minute apart. \ 
Previously, a range query with `5m` would usually return 5 samples, or 6 samples \ 
if the query evaluation aligns perfectly with a scrape. Now, queries like this \ 
will always return 5 samples. This upgrades PromQL compatibility from Prometheus \ 
2.0 to 3.0. #9844
* [CHANGE] Querier: promql(native histograms): Introduce exponential \ 
interpolation. #9844
* [CHANGE] Remove deprecated `api.get-request-for-ingester-shutdown-enabled` \ 
setting, which scheduled for removal in 2.15. #10197
* [FEATURE] Querier: add experimental streaming PromQL engine, enabled with \ 
`-querier.query-engine=mimir`. #10067
* [FEATURE] Distributor: Add support for `lz4` OTLP compression. #9763
* [FEATURE] Query-frontend: added experimental configuration options \ 
`query-frontend.cache-errors` and `query-frontend.results-cache-ttl-for-errors` \ 
to allow non-transient responses to be cached. When set to `true` error \ 
responses from hitting limits or bad data are cached for a short TTL. #9028
* [FEATURE] Query-frontend: add middleware to control access to specific PromQL \ 
experimental functions on a per-tenant basis. #9798
* [FEATURE] gRPC: Support S2 compression. #9322
  * `-alertmanager.alertmanager-client.grpc-compression=s2`
  * `-ingester.client.grpc-compression=s2`
  * `-querier.frontend-client.grpc-compression=s2`
  * `-querier.scheduler-client.grpc-compression=s2`
  * `-query-frontend.grpc-client-config.grpc-compression=s2`
  * `-query-scheduler.grpc-client-config.grpc-compression=s2`
  * `-ruler.client.grpc-compression=s2`
  * `-ruler.query-frontend.grpc-client-config.grpc-compression=s2`
* [FEATURE] Alertmanager: limit added for maximum size of the Grafana state \ 
(`-alertmanager.max-grafana-state-size-bytes`). #9475
* [FEATURE] Alertmanager: limit added for maximum size of the Grafana \ 
configuration (`-alertmanager.max-config-size-bytes`). #9402
* [FEATURE] Ingester: Experimental support for ingesting out-of-order native \ 
histograms. This is disabled by default and can be enabled by setting \ 
`-ingester.ooo-native-histograms-ingestion-enabled` to `true`. #7175
* [FEATURE] Distributor: Added `-api.skip-label-count-validation-header-enabled` \ 
option to allow skipping label count validation on the HTTP write path based on \ 
`X-Mimir-SkipLabelCountValidation` header being `true` or not. #9576
* [FEATURE] Ruler: Add experimental support for caching the contents of rule \ 
groups. This is disabled by default and can be enabled by setting \ 
`-ruler-storage.cache.rule-group-enabled`. #9595 #10024
* [FEATURE] PromQL: Add experimental `info` function. Experimental functions are \ 
disabled by default, but can be enabled setting \ 
`-querier.promql-experimental-functions-enabled=true` in the query-frontend and \ 
querier. #9879
* [FEATURE] Distributor: Support promotion of OTel resource attributes to \ 
labels. #8271
* [FEATURE] Querier: Add experimental `double_exponential_smoothing` PromQL \ 
function. Experimental functions are disabled by default, but can be enabled by \ 
setting `-querier.promql-experimental-functions-enabled=true` in the \ 
query-frontend and querier. #9844
* [ENHANCEMENT] Query Frontend: Return server-side `bytes_processed` statistics \ 
following Server-Timing format. #9645 #9985
* [ENHANCEMENT] mimirtool: Adds bearer token support for mimirtool's analyze \ 
ruler/prometheus commands. #9587
* [ENHANCEMENT] Ruler: Support `exclude_alerts` parameter in \ 
`<prometheus-http-prefix>/api/v1/rules` endpoint. #9300
* [ENHANCEMENT] Distributor: add a metric to track tenants who are sending \ 
newlines in their label values called \ 
`cortex_distributor_label_values_with_newlines_total`. #9400
* [ENHANCEMENT] Ingester: improve performance of reading the WAL. #9508
* [ENHANCEMENT] Query-scheduler: improve the errors and traces emitted by \ 
query-schedulers when communicating with queriers. #9519
* [ENHANCEMENT] Compactor: uploaded blocks cannot be bigger than max configured \ 
compactor time range, and cannot cross the boundary for given time range. #9524
* [ENHANCEMENT] The distributor now validates that received label values only \ 
contain allowed characters. #9185
* [ENHANCEMENT] Add SASL plain authentication support to Kafka client used by \ 
the experimental ingest storage. Configure SASL credentials via the following \ 
settings: #9584
  * `-ingest-storage.kafka.sasl-password`
  * `-ingest-storage.kafka.sasl-username`
* [ENHANCEMENT] memberlist: TCP transport write path is now non-blocking, and is \ 
configurable by new flags: #9594
  * `-memberlist.max-concurrent-writes`
  * `-memberlist.acquire-writer-timeout`
* [ENHANCEMENT] memberlist: Notifications can now be processed once per interval \ 
specified by `-memberlist.notify-interval` to reduce notify storm CPU activity \ 
in large clusters. #9594
* [ENHANCEMENT] Query-scheduler: Remove the experimental \ 
`query-scheduler.prioritize-query-components` flag. Request queues always \ 
prioritize query component dequeuing above tenant fairness. #9703
* [ENHANCEMENT] Ingester: Emit traces for block syncing, to join up block-upload \ 
traces. #9656
* [ENHANCEMENT] Querier: Enable the optional querying of additional storage \ 
queryables. #9712
* [ENHANCEMENT] Ingester: Disable the push circuit breaker when ingester is in \ 
read-only mode. #9760
* [ENHANCEMENT] Ingester: Reduced lock contention in the `PostingsForMatchers` \ 
cache. #9773
* [ENHANCEMENT] Storage: Allow HTTP client settings to be tuned for GCS and \ 
Azure backends via an `http` block or corresponding CLI flags. This was already \ 
supported by the S3 backend. #9778
* [ENHANCEMENT] Ruler: Support `group_limit` and `group_next_token` parameters \ 
in the `<prometheus-http-prefix>/api/v1/rules` endpoint. #9563
* [ENHANCEMENT] Ingester: improved lock contention affecting read and write \ 
latencies during TSDB head compaction. #9822
* [ENHANCEMENT] Distributor: when a label value fails validation due to invalid \ 
UTF-8 characters, don't include the invalid characters in the returned error. \ 
#9828
* [ENHANCEMENT] Ingester: when experimental ingest storage is enabled, do not \ 
buffer records in the Kafka client when fetch concurrency is in use. #9838 #9850
* [ENHANCEMENT] Compactor: refresh deletion marks when updating the bucket index \ 
concurrently. This speeds up updating the bucket index by up to 16 times when \ 
there is a lot of blocks churn (thousands of blocks churning every cleanup \ 
cycle). #9881
* [ENHANCEMENT] PromQL: make `sort_by_label` stable. #9879
* [ENHANCEMENT] Distributor: Initialize ha_tracker cache before ha_tracker and \ 
distributor reach running state and begin serving writes. #9826 #9976
* [ENHANCEMENT] Ingester: `-ingest-storage.kafka.max-buffered-bytes` to limit \ 
the memory for buffered records when using concurrent fetching. #9892
* [ENHANCEMENT] Querier: improve performance and memory consumption of queries \ 
that select many series. #9914
* [ENHANCEMENT] Ruler: Support OAuth2 and proxies in Alertmanager client #9945 #10030
* [ENHANCEMENT] Ingester: Add \ 
`-blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samples` to build 24h \ 
blocks for out-of-order data belonging to the previous days instead of building \ 
smaller 2h blocks. This reduces pressure on compactors and ingesters when the \ 
out-of-order samples span multiple days in the past. #9844 #10033 #10035
* [ENHANCEMENT] Distributor: allow a different limit for info series (series \ 
ending in `_info`) label count, via \ 
`-validation.max-label-names-per-info-series`. #10028
* [ENHANCEMENT] Ingester: do not reuse labels, samples and histograms slices in \ 
the write request if there are more entries than 10x the pre-allocated size. \ 
This should help to reduce the in-use memory in case of few requests with a very \ 
large number of labels, samples or histograms. #10040
* [ENHANCEMENT] Query-Frontend: prune `<subquery> and on() (vector(x)==y)` \ 
style queries and stop pruning `<subquery> < -Inf`. Triggered by \ 
https://github.com/prometheus/prometheus/pull/15245. #10026
* [ENHANCEMENT] Query-Frontend: perform request format validation before \ 
processing the request. #10093
* [BUGFIX] Fix issue where functions such as `rate()` over native histograms \ 
could return incorrect values if a float stale marker was present in the \ 
selected range. #9508
* [BUGFIX] Fix issue where negation of native histograms (eg. \ 
`-some_native_histogram_series`) did nothing. #9508
* [BUGFIX] Fix issue where `metric might not be a counter, name does not end in \ 
_total/_sum/_count/_bucket` annotation would be emitted even if `rate` or \ 
`increase` did not have enough samples to compute a result. #9508
* [BUGFIX] Fix issue where sharded queries could return annotations with \ 
incorrect or confusing position information. #9536
* [BUGFIX] Fix issue where downstream consumers may not generate correct cache \ 
keys for experimental error caching. #9644
* [BUGFIX] Fix issue where active series requests error when encountering a \ 
stale posting. #9580
* [BUGFIX] Fix pooling buffer reuse logic when \ 
`-distributor.max-request-pool-buffer-size` is set. #9666
* [BUGFIX] Fix issue when using the experimental \ 
`-ruler.max-independent-rule-evaluation-concurrency` feature, where the ruler \ 
could panic as it updates a running ruleset or shutdowns. #9726
* [BUGFIX] Always return unknown hint for first sample in non-gauge native \ 
histograms chunk to avoid incorrect counter reset hints when merging chunks from \ 
different sources. #10033
* [BUGFIX] Ensure native histograms counter reset hints are corrected when \ 
merging results from different sources. #9909
* [BUGFIX] Ingester: Fix race condition in per-tenant TSDB creation. #9708
* [BUGFIX] Ingester: Fix race condition in exemplar adding. #9765
* [BUGFIX] Ingester: Fix race condition in native histogram appending. #9765
* [BUGFIX] Ingester: Fix bug in concurrent fetching where a failure to list \ 
topics on startup would cause to use an invalid topic ID \ 
(0x00000000000000000000000000000000). #9883
* [BUGFIX] Ingester: Fix data loss bug in the experimental ingest storage when a \ 
Kafka Fetch is split into multiple requests and some of them return an error. \ 
#9963 #9964
* [BUGFIX] PromQL: `round` now removes the metric name again. #9879
* [BUGFIX] Query-Frontend: fix `QueryFrontendCodec` module initialization to set \ 
lookback delta from `-querier.lookback-delta`. #9984
* [BUGFIX] OTLP: Support integer exemplar value type. #9844
* [BUGFIX] Querier: Correct the behaviour of binary operators between native \ 
histograms and floats. #9844
* [BUGFIX] Querier: Fix stddev+stdvar aggregations to always ignore native \ 
histograms. #9844
* [BUGFIX] Querier: Fix stddev+stdvar aggregations to treat Infinity \ 
consistently. #9844
* [BUGFIX] Ingester: Chunks could have one unnecessary zero byte at the end. #9844
* [BUGFIX] OTLP receiver: Preserve colons and combine multiple consecutive \ 
underscores into one when generating metric names in suffix adding mode \ 
(`-distributor.otel-metric-suffixes-enabled`). #10075

### Mixin

* [CHANGE] Remove backwards compatibility for `thanos_memcached_` prefixed \ 
metrics in dashboards and alerts removed in 2.12. #9674 #9758
* [CHANGE] Reworked the alert `MimirIngesterStuckProcessingRecordsFromKafka` to \ 
also work when concurrent fetching is enabled. #9855
* [ENHANCEMENT] Unify ingester autoscaling panels on 'Mimir / Writes' dashboard \ 
to work for both ingest-storage and non-ingest-storage autoscaling. #9617
* [ENHANCEMENT] Alerts: Enable configuring job prefix for alerts to prevent \ 
clashes with metrics from Loki/Tempo. #9659
* [ENHANCEMENT] Dashboards: visualize the age of source blocks in the \ 
"Mimir / Compactor" dashboard. #9697
* [ENHANCEMENT] Dashboards: Include block compaction level on queried blocks in \ 
'Mimir / Queries' dashboard. #9706
* [ENHANCEMENT] Alerts: add `MimirIngesterMissedRecordsFromKafka` to detect gaps \ 
in consumed records in the ingester when using the experimental Kafka-based \ 
storage. #9921 #9972
* [ENHANCEMENT] Dashboards: Add more panels to 'Mimir / Writes' for concurrent \ 
ingestion and fetching when using ingest storage. #10021
* [BUGFIX] Dashboards: Fix autoscaling metrics joins when series churn. #9412 \ 
#9450 #9432
* [BUGFIX] Alerts: Fix autoscaling metrics joins in `MimirAutoscalerNotActive` \ 
when series churn. #9412
* [BUGFIX] Alerts: Exclude failed cache "add" operations from alerting \ 
since failures are expected in normal operation. #9658
* [BUGFIX] Alerts: Exclude read-only replicas from \ 
`IngesterInstanceHasNoTenants` alert. #9843
* [BUGFIX] Alerts: Use resident set memory for the `EtcdAllocatingTooMuchMemory` \ 
alert so that ephemeral file cache memory doesn't cause the alert to misfire. \ 
#9997
* [BUGFIX] Query-frontend: support `X-Read-Consistency-Offsets` on labels \ 
queries too.

### Jsonnet

* [CHANGE] Remove support to set Redis as a cache backend from jsonnet. #9677
* [CHANGE] Rollout-operator now defaults to storing scaling operation metadata \ 
in a Kubernetes ConfigMap. This avoids recursively invoking the admission \ 
webhook in some Kubernetes environments. #9699
* [CHANGE] Update rollout-operator version to 0.20.0. #9995
* [CHANGE] Remove the `track_sizes` feature for Memcached pods since it is \ 
unused. #10032
* [CHANGE] The configuration options `autoscaling_distributor_min_replicas` and \ 
`autoscaling_distributor_max_replicas` has been renamed to \ 
`autoscaling_distributor_min_replicas_per_zone` and \ 
`autoscaling_distributor_max_replicas_per_zone` respectively. #10019
* [FEATURE] Add support to deploy distributors in multi availability zones. #9548
* [FEATURE] Add configuration settings to set the number of Memcached replicas \ 
for each type of cache (`memcached_frontend_replicas`, \ 
`memcached_index_queries_replicas`, `memcached_chunks_replicas`, \ 
`memcached_metadata_replicas`). #9679
* [ENHANCEMENT] Add `ingest_storage_ingester_autoscaling_triggers` option to \ 
specify multiple triggers in ScaledObject created for ingest-store ingester \ 
autoscaling. #9422
* [ENHANCEMENT] Add \ 
`ingest_storage_ingester_autoscaling_scale_up_stabilization_window_seconds` and \ 
`ingest_storage_ingester_autoscaling_scale_down_stabilization_window_seconds` \ 
config options to make stabilization window for ingester autoscaling when using \ 
ingest-storage configurable. #9445
* [ENHANCEMENT] Make label-selector in ReplicaTemplate/ingester-zone-a object \ 
configurable when using ingest-storage. #9480
* [ENHANCEMENT] Add `querier_only_args` option to specify CLI flags that apply \ 
only to queriers but not ruler-queriers. #9503
* [ENHANCEMENT] Validate the Kafka client ID configured when ingest storage is \ 
enabled. #9573
* [ENHANCEMENT] Configure pod anti-affinity and tolerations to run etcd pods \ 
multi-AZ when `_config.multi_zone_etcd_enabled` is set to `true`. #9725

### Mimirtool

### Mimir Continuous Test

### Query-tee

* [FEATURE] Added `-proxy.compare-skip-samples-before` to skip samples before \ 
the given time when comparing responses. The time can be in RFC3339 format (or) \ 
RFC3339 without the timezone and seconds (or) date only. #9515
* [FEATURE] Add `-backend.config-file` for a YAML configuration file for \ 
per-backend options. Currently, it only supports additional HTTP request \ 
headers. #10081
* [ENHANCEMENT] Added human-readable timestamps to comparison failure messages. #9665

### Documentation

* [BUGFIX] Send native histograms: update the migration guide with the corrected \ 
dashboard query for switching between classic and native histograms queries. \ 
#10052

### Tools

* [FEATURE] `splitblocks`: add new tool to split blocks larger than a specified \ 
duration into multiple blocks. #9517, #9779
* [ENHANCEMENT] `copyblocks`: add `--skip-no-compact-block-duration-check`, \ 
which defaults to `false`, to simplify targeting blocks that are not awaiting \ 
compaction. #9439
* [ENHANCEMENT] `copyblocks`: add `--user-mapping` to support copying blocks \ 
between users. #10110
* [ENHANCEMENT] `kafkatool`: add SASL plain authentication support. The \ 
following new CLI flags have been added: #9584
  * `--kafka-sasl-username`
  * `--kafka-sasl-password`
* [ENHANCEMENT] `kafkatool`: add `dump print` command to print the content of \ 
write requests from a dump. #9942
* [ENHANCEMENT] Updated `KubePersistentVolumeFillingUp` runbook, including a \ 
sample command to debug the distroless image. #9802