Skip to main content

Release Notes

CES 1.0 is based on quincy 17.2.7.

These release notes are divided into two sections, upstream backports and downstream only patches. Upstream backports are a set of patches that didn't make it to quincy 17.2.7 but feel are vital to operating Ceph. Downstream only patches on the other hand are configuration changes we highly recommend to operators or changes we have yet to submit to the upstream Ceph community.

Downstream Patches​

common/options: Reduce rgw_thread_pool_size. ​

  • 512 to 128

Short term patch until https://github.com/ceph/ceph/pull/57167 can be backported.

Increase default bucket shard count. ​

  • from 11 to 31

Improves per-bucket PUT performance fairly dramatically on initial ingest. Pushes off bucket resharding. Does have some impact on list performance for small buckets, however small buckets generally already list fairly quickly and this does not hurt large bucket listing performance that will eventually have more shards anyway.

common/options: Increase objecter inflight thresholds. ​

  • Max in-flight data in bytes: 100_M to 1_G
  • Max in-flight operations: 1_K to 8_K

Increase performance of RGW workloads and is a common optimization. Corresponding op threshold increase allows 8192 128K objects, which is a similar object size target relative to stock upstream defaults

ceph-objectstore-tool: make 'rm-omap' command support remove many keys ​

  • rm-omap can remove multiple keys

Based on: https://github.com/ceph/ceph/pull/22379

Fixes: https://tracker.ceph.com/issues/38215

osd: increase osd_max_pg_per_osd_hard_ratio to 10 ​

  • A hard ratio of 3 is hit too often during normal maintinance tasks such as adding a new host. In that case, OSDs prevent pg peering and many PGs are stuck in activating state. The OSDs log this warning: "maybe_wait_for_max_pg withhold creation of pg"

mgr/balancer: set upmap_max_deviation to 1​

  • default upmax_max_deviation 5 is not effective to reach well a balanced cluster. This is especially on evident on clusters with many pools, because the deviation of 5 is per pool, so the deviations can pile up, and OSDs can vary by a lot.

osd: default osd_op_type wpq​

  • mclock still unstable in corner cases in quincy. Setting to wpq fixes some issues for now.

rgw: Increase default bucket shard count.​

Dashboard​

  • Update logo on create cluster page
  • CES dashboard branding (logo, constants, favicon, login page links)

Images​

Change from quay.io to harbor.clyso.com default container images references.

Upstream Backports​

os/bluestore: get rid off resulting lba alignment in allocators ​

Fixes: https://tracker.ceph.com/issues/63618

Fixes: https://tracker.ceph.com/issues/62815

Based on: https://github.com/ceph/ceph/pull/54877

osd/scrub: increasing max_osd_scrubs from 1 to 3 ​

Current default value of '1' is too low: the cluster is susceptible to scrub scheduling delays and issues stemming from local software/networking/hardware problems, even if affecting a very small number of OSDs.

(cherry picked from commit cc7b4af)

(cherry picked from commit 1ee030e)

cmake/modules: Fix Debian/Ubuntu RocksDB Performance Issues ​

  • setting CXXFLAGS environmental variable, should help with the performance in the bluestore, where RocksDB is used.

install-deps.sh, do_cmake.sh changes ​

  • add support for rocky linux container

osd/ECTransaction: Remove incorrect asserts in generate_transactions ​

  • Back in PR #11701 when EC Overwrites were added, there was significant churn in the ECTransaction code. Several asserts were added in generate_transactions, but the code changed several times and it appears some of those asserts don't make sense in the final version of the PR.

Fixes: https://tracker.ceph.com/issues/65509 (cherry picked from commit c6eb35b)

common/options: Set LZ4 compression for bluestore RocksDB. ​

In the fall of 2022, we tested LZ4 RocksDB compression in bluestore on NVMe backed OSDs here:

https://ceph.io/en/news/blog/2022/rocksdb-tuning-deep-dive/

Since then, we've gotten feedback from users in the field testing compression with extremely positive results. Clyso has also worked with a customer that has a large RGW deployment that has seen extremely positive results.

(cherry picked from commit 17840db)

common/options: Update RocksDB CF Tuning ​

  • max_write_buffer_number from 128 to 64
  • min_write_buffer_number_to_merge from 16 to 6
  • write_buffer_size from 8388608 to 16777216
  • removed ttl
  • L=min_write_buffer_number_to_merge=32
  • P=min_write_buffer_number_to_merge=32

(cherry picked from commit ea92ee7)

common/options: increase mds_cache_trim_threshold 2x ​

  • from 256K to 512K The MDS maintains its LRU cache size by periodically trimming entries. It trims up to mds_cache_trim_threshold entries per tick. By increasing this option to 512k, it will trim the LRU more actively, keeping the cache size under the configured limit.

mgr/dashboard: disable dashboard v3 in quincy ​

Fixes: https://tracker.ceph.com/issues/63357

quincy: cephadm: disable ms_bind_ipv4 if we will enable ms_bind_ipv6​

  • While bootstrapping an ipv6 cluster with an ipv6 initial mon, cephadm correctly enables ms_bind_ipv6=true. However it leaves ms_bind_ipv4 as it's default (true). Fix by disabling ms_bind_ipv4 in the case ipv6 is desired.

Fixes: https://tracker.ceph.com/issues/66436

mgr/k8sevents: update V1Events to CoreV1Events​

Fixes: https://tracker.ceph.com/issues/65627

Fixes: https://tracker.ceph.com/issues/64981

mgr/prometheus: s/pkg_resources.packaging/packaging/​

Fixes https://tracker.ceph.com/issues/66201

pybind/mgr/mgr_util: convert certs to bytes before loading them​

common: fix compilation warnings in numa.cc​