Investigate 'profile rbd' for rbd client ceph auth caps
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceph Monitor Charm |
New
|
Undecided
|
Unassigned |
Bug Description
[Impact]
Ceph librbd clients make use of exclusive locks [0] to co-ordinate access to RBD. In the event that an exclusive lock is mismanaged (hung task, crash, network partition, etc.) a new client can negotiate a blacklist with the ceph mon. This requires that the client have the correct mon auth caps configured to allow the 'osd blacklist' command.
Without blacklist caps, a mismanaged exclusive lock will prevent nova-compute instance from starting.
The ceph-mon charm should configure nova/cinder and glance rbd clients with this capability.
[Workaround]
Pre-Luminous, the osd blacklist command must be explicitly allowed:
mon 'allow r, allow command "osd blacklist"'
Post-luminous, upstream recommends using 'profile rbd' when specifying auth caps for rbd clients:
> V12.0.0 Luminous: Specifying user authorization capabilities for RBD clients has been simplified. The general syntax for using RBD capability profiles is “mon ‘profile rbd’ osd ‘profile rbd[-read-only][ pool={pool-name}[, …]]’”. [1]
[Other Info]
There is an upstream guide for rbd OpenStack integration that details how to setup ceph client authentication for nova/cinder and glance [2].
An important note! Luminous and mimic do not support mgr profiles. This was introduced in nautilus 14.2.8 [3]:
> The MGR now accepts profile rbd and profile rbd-read-only user caps. These caps can be used to provide users access to MGR-based RBD functionality such as `rbd perf image iostat` and `rbd perf image iotop`.
[0] https:/
[1] https:/
[2] https:/
[3] https:/
tags: | added: sts |
Changed in charm-ceph-mon: | |
status: | Incomplete → New |
summary: |
- Failure to start ceph-backed instance after crash + Investigate 'profile rbd' for rbd client ceph auth caps |
Bug 1773449