Ceph Umbrella (v21)
This document lists known critical bugs affecting Ceph Umbrella (v21) releases. Find your version in the first table, then read the details of each bug below.
Bugs by Version
| Version | Known bugs |
|---|---|
| Every Umbrella release | Monitor cluster log replay |
| 21.2.0 | BlueFS overwrite by deferred replay, Optimized EC returns zeros, Cephadm uppercase hostnames, RGW multisite upgrade inconsistency, LRC EIO after upgrade, OSD start timeout during upgrade, Active MGR memory leak |
Summary
| Bug | Severity | Affected | Fixed in |
|---|---|---|---|
| Monitors replay months of cluster log after a restart | High | 21.2.0 | Not yet released |
| BlueFS overwritten by deferred replay | Critical | All | Not yet released |
| Optimized EC returns zeros after unaligned writes | Critical | 21.2.0, pools with EC optimizations | Not yet released |
| RGW multisite inconsistency during rolling upgrade | High | 21.2.0 multisite | Not yet released |
| Cephadm marks uppercase hostnames as unknown after upgrade | High | Upgrades to 21.2.0 | Not yet released |
| LRC pools return EIO on degraded reads after upgrade | High | LRC pools created before Tentacle | Not yet released |
| OSDs hit the start timeout during upgrade | Medium | Upgrades under load | Not yet released |
| Active MGR memory leak | Medium | 21.2.0 | Not yet released |
Bugs that link to the Tentacle page are described there.
BlueFS Overwritten by Deferred Replay
Severity: Critical Affected Versions: All Bug Report: https://tracker.ceph.com/issues/82617
Description
During OSD startup, _deferred_replay can write over BlueFS extents and corrupt
the OSD's RocksDB. It can hit on the first OSD start after an upgrade.
Recommendation
- The Umbrella backport is PR #73848 (under review). No released version has it yet.
Optimized EC Returns Zeros After Unaligned Writes
Severity: Critical
Affected Versions: 21.2.0, only on pools with allow_ec_optimizations enabled
Bug Report: https://tracker.ceph.com/issues/81848
Description
With EC optimizations enabled, two unaligned writes that share a page can leave a gap that reads back as zeros. PGs stay clean and deep scrub does not detect it.
Recommendation
- Do not enable
allow_ec_optimizationson 21.2.0. - The Umbrella backport is PR #73835 (under review).
RGW Multisite Inconsistency During Rolling Upgrade
Severity: High Affected Versions: 21.2.0 multisite, upgrades from Squid or Tentacle Bug Report: https://tracker.ceph.com/issues/82574, https://tracker.ceph.com/issues/82573
Description
During a rolling multisite upgrade, stale version-list keys are never removed. On Umbrella, a stale null delete marker arriving through sync can delete the data of the current null version.
Recommendation
- Delay Umbrella upgrades of RGW multisite deployments until a fix is released.
Cephadm Marks Uppercase Hostnames as Unknown After Upgrade
Severity: High Affected Versions: Upgrades from 20.2.x to 21.2.0 Bug Report: https://tracker.ceph.com/issues/82666
Description
After the upgrade, hosts with uppercase characters in their names show up as "Unknown hosts" and the orchestrator becomes unusable.
Recommendation
- Check
ceph orch host lsfor uppercase hostnames before upgrading. If any exist, delay the upgrade.
LRC Pools Return EIO on Degraded Reads After Upgrade
Severity: High Affected Versions: 21.2.0, LRC pools created before Tentacle Bug Report: https://tracker.ceph.com/issues/81369
Description
Degraded reads from LRC pools created on Squid or earlier fail with EIO after upgrading.
Recommendation
- Clusters with LRC pools should not upgrade until a fix is released.
OSDs Hit the Start Timeout During Upgrade
Severity: Medium Affected Versions: Upgrades to 21.2.0 under client load Bug Report: https://tracker.ceph.com/issues/81933
Description
Under load, upgraded OSDs can exceed the systemd start timeout (200 s) and are marked failed.
Recommendation
- Upgrade during low-load windows and use a staggered
ceph orch upgrade.
Active MGR Memory Leak
Severity: Medium Affected Versions: 21.2.0 Bug Report: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/WT7NTYUUL6HMJ5CGHRP6BHRJ3W37JUHN/
Description
The active MGR grows by about 1 GB per hour.
Recommendation
- Workaround:
ceph config set mgr mgr_map_cache_enabled false. Slow growth remains, so fail over the MGR periodically.