Grafana Mimir version 2.9 release notes

Grafana Labs is excited to announce version 2.9 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Reduced store-gateway memory utilization on fetching series from long-term storage For queries that include broad label matchers (e.g. datacenter="dc1"), Mimir 2.9 will fetch a reduced volume of index data, which leads to a significant reduction in memory allocations in the store-gateway.
Reduced CPU utilisation for some shuffle sharding scenarios Mimir queriers will now use significantly less CPU in cases where shuffle sharding is enabled for tenants with a shard size that’s large but lower than the total number of ingesters.
Reduced object storage API calls in compactors and rulers Mimir 2.9 comes with optimizations that will reduce the amount of times compactors and rulers need to access rules stored in object storage.
- This release adds experimental support for a ruler storage cache. This cache should reduce the number of “list objects” API calls issued to the object storage when there are 2+ ruler replicas running in a Mimir cluster. The cache can be configured by setting the -ruler-storage.cache.* CLI flags or their respective YAML config options.
- We also introduced a new feature to trigger a synchronization of tenant’s rule groups as soon as changes to the rule configuration are made via API. This synchronization is in addition of the periodic syncing done every -ruler.poll-interval, which has then been relaxed from every 1m to every 10m. The new behaviour is enabled globally by default but can be disabled with -ruler.sync-rules-on-changes-enabled=false or tuned at a per-tenant level.
Experimental support for streaming chunks from ingester to querier This is expected to greatly reduce querier memory consumption when evaluating queries that select a large number of series, because chunks streamed from the querier can now be read into memory as needed.

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.9 we have removed the following previously deprecated or experimental metrics:

cortex_bucket_store_chunk_pool_requested_bytes_total
cortex_bucket_store_chunk_pool_returned_bytes_total

The following configuration options are deprecated and will be removed in Grafana Mimir 2.11:

The CLI flag -querier.query-ingesters-within. This configuration is moved to per-tenant overrides.
The CLI flag -blocks-storage.bucket-store.bucket-index.enabled.
The CLI flags -blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes, -blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes and -blocks-storage.bucket-store.max-chunk-pool-bytes.
The CLI flags -querier.iterators and -query.batch-iterators.

The following configuration options that were deprecated in 2.7 are removed:

The CLI flag -blocks-storage.bucket-store.chunks-cache.subrange-size. A fixed value of 16000 is now always used.
The CLI flag -blocks-storage.bucket-store.consistency-delay.
The CLI flag -compactor.consistency-delay.
The CLI flag -ingester.ring.readiness-check-ring-health.

The following experimental options and features are now stable:

The CLI flag -query-frontend.query-sharding-max-regexp-size-bytes.
The CLI flag -query-scheduler.max-used-instances.
The CLI flags -(alertmanager|blocks|ruler)-storage.storage-prefix.
The CLI flag -compactor.first-level-compaction-wait-period.
The CLI flags -usage-stats.enabled and -usage-stats.installation-mode.
The CLI flag -query-frontend.query-sharding-target-series-per-shard.

The following configuration option defaults were changed:

The default value for the CLI flag -query-frontend.query-sharding-max-regexp-size-bytes was changed from 0 to 4096. As a result, queries with regex matchers exceeding this limit will not be sharded by default.
The default value for the CLI flag -compactor.partial-block-deletion-delay was changed from 0s to 1d. As a result, partial blocks resulting from a failed block upload or deletion will be cleaned up automatically.
The default value for the CLI flag -ruler.poll-interval was changed from 1m to 10m.

Bug fixes

Store-gateway: Detect collisions in the postings cache. PR 4770
Store-gateway: Fix panic caused by cached LabelValues responses with more than 655360 values. PR 5021