Fix an OSD that has become associated with two block devices
Problem
A user replaced some OSDs in a cluster. After the replacement, one of the OSDs in the cluster had two block devices associated with it. The user wanted only one block device to be associated with the OSD.
Solution
-
Confirm that more than one block device is associated with the OSD under consideration by running the command
cephadm ceph-volume lvm list xxx, wherexxxis the number associated with the OSD:$ cephadm ceph-volume lvm list 001
Using recent ceph image registry.suse.com/ses/7.1/ceph/ceph@sha256:000aaaaaa00aa0aa00000aa00a0000000000000aa0a000aa0a00000a0a000a0a
ceph-00000aa0-000a-0a00-a0a0-aa0a0aaa0000/osd-block-a0a00a00-a000-0a0b-aa00-a00000a000a0
[block] /dev/ceph-AAAAAaa0-000a-0a00-a0a0-aa0a0aaa0000/osd-block-a0a00a00-a000-0a0a-aa00-a00000a000a0
block device /dev/ceph-AAAAAaa0-000a-0a00-a0a0-aa0a0aaa0000/osd-block-a0a00a00-a000-0a0a-aa00-a00000a000a0
block uuid xxxxxx-xxxx-xxx0-0xxx-xxx-xxxx-xxx0xx
cephx lockbox secret
cluster fsid xxxxxxxx-xxxx-xxxx-xx00-xxxxxxxxxxxx
cluster name ceph
crush device class
encrypted 0
osd fsid xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
osd id 209
osdspec affinity drive_group_ssd-backed
type block
vdo 0
devices /dev/sdq
[block] /dev/ceph-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/osd-block-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
block device /dev/ceph-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/osd-block-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
block uuid xxxxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxxxx
cephx lockbox secret
cluster fsid xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
cluster name ceph
crush device class
db device /dev/ceph-block-dbs-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/osd-db-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
db uuid xxxxxx-xxmx-xxxx-xxxx-xxxx-xxxx-xxxxxx
encrypted 0
osd fsid xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
osd id 209
osdspec affinity drive_group_ssd-backed
type block
vdo 0
devices /dev/sdag -
Use the command
dmsetup removeto remove the target logical volume:dmsetup remove ceph-AAAAAaa0-000a-0a00-a0a0-aa0a0aaa0000/osd-block-a0a00a00-a000-0a0b-aa00-a00000a000a0 -
Use the command
wipefs -ato remove/dev/sdq.wipefs -a /dev/sdqWarning: Using
wipefsdeletes the filesystem on the disk (in this example/dev/sdq). This command erases all filesystem, RAID, and partition-table signatures on the storage device (in this example/dev/sdq). -
/dev/sdqis now empty and can be removed from the system.