lvm should not scan the entire /dev tree by default

I have lost data in the past due to the permissive default scanning rules in /etc/lvm/lvm.conf before and have helped two others who have had issues recently.

After digging through the linux multipathing, udev, dm and LVM code for hours the only devices safe to scan for LVM and for multipathing on our KVM hosts are the /dev/disks/by-id/ata.*|scsi.* devices

There really is no reason for LVM to scan the entire device tree by default, as the mapper will build links under /dev/disk/by-id/

/dev/disk/by-uuid is particularly risky as it is last writer win (as is by-id/wwn) and a guest disk image or host snapshot can hijack the device name and mount or worse, as happened to me due to poor mulitpath blacklist command (from a tier 1 storage provider) cause a merge of a snapshot and data loss. This behavior is as documented as intentional in Bug #460906 although I can not find jusitifcation for it.

In this day of people running many guests on even their laptops scanning the entire /dev tree is also a risk due to possable leakage from guests, expecially if they choose to mount volumes based on UUID.

Placing a single filter in by default will also ensure that those who are using multipathing and LVM will actually use the multipath device as multipath-tools updates /dev/disk/by-id/scsi.* but not /dev/disk/by-id/wwn.* which is currently last writer wins.

The two changes I make to /etc/lvm/lvm.conf are:

    preferred_names = [ "^/dev/disk/by-id/scsi" ]

This will prefer the SCSI wwn devices if this next rule fails:

    filter = [ "a|/dev/disk/by-id/scsi-.*|", "r/.*/" ]

This rule adds the /dev/disk/by-id/scsi-* devices to be scanned and excludes all others.

Here I will document that with multipathing by-id/scsi-.* is the best choice.

root@usdckvm201:~# multipath -ll
3600144f0f26a8a0000004fa322d90008dm-59 SUN ,COMSTAR
[size=1.0T][features=0][hwhandler=0]
\_ round-robin 0 [prio=4][active]
 \_ 6:0:1:0 sdb 8:16 [active][ready]
 \_ 5:0:0:0 sdc 8:32 [active][ready]
 \_ 5:0:1:0 sdd 8:48 [active][ready]
 \_ 6:0:0:0 sde 8:64 [active][ready]
root@usdckvm201:~# pvs
  PV VG Fmt Attr PSize PFree
  /dev/disk/by-id/scsi-3600144f0f26a8a0000004fa322d90008-part1 lvm2 a- 1023.99g 1023.99g
  /dev/disk/by-id/scsi-360024e8052404f001608f3c305ead9bb-part5 usdckvm201 lvm2 a- 1.64t 227.99g

Note that I do not have errors about multiple block devices being found as you see with the default configuration.

Above you see that dm-59 is the device name for the multi-pathed disk, device-mapper and multipathd pre-pends "mpath" to the dm-uuid device so if you boot with only one path the device path will be invalid, the wwn-* devices SHOULD point at the multipath device but there is a bug and/or feature where the last writer wins, and it never seems to be the multipath device so although teh wwn-* device would be safe to use it will not be multipathed and the loss of it's current path will result in blocked IO.

I think this is due to the mapper only mapping DISKS to wwn- but the scsi-* rule would still find the volume.

lrwxrwxrwx 1 root root 11 2012-05-03 17:55 dm-uuid-mpath-3600144f0f26a8a0000004fa322d90008 -> ../../dm-59
lrwxrwxrwx 1 root root 11 2012-05-03 18:04 dm-uuid-part1-mpath-3600144f0f26a8a0000004fa322d90008 -> ../../dm-61
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 scsi-3600144f0f26a8a0000004fa322d90008 -> ../../dm-59
lrwxrwxrwx 1 root root 11 2012-05-03 18:04 scsi-3600144f0f26a8a0000004fa322d90008-part1 -> ../../dm-61
lrwxrwxrwx 1 root root 9 2012-05-03 17:55 scsi-360024e8052404f001608f3c305ead9bb -> ../../sda
lrwxrwxrwx 1 root root 10 2011-09-18 13:32 scsi-360024e8052404f001608f3c305ead9bb-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 scsi-360024e8052404f001608f3c305ead9bb-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2012-05-03 18:28 scsi-360024e8052404f001608f3c305ead9bb-part5 -> ../../sda5
lrwxrwxrwx 1 root root 9 2012-05-03 17:55 wwn-0x600144f0f26a8a0000004fa322d90008 -> ../../sde
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 wwn-0x600144f0f26a8a0000004fa322d90008-part1 -> ../../sde1
lrwxrwxrwx 1 root root 9 2012-05-03 17:55 wwn-0x60024e8052404f001608f3c305ead9bb -> ../../sda
lrwxrwxrwx 1 root root 10 2011-09-18 13:32 wwn-0x60024e8052404f001608f3c305ead9bb-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 wwn-0x60024e8052404f001608f3c305ead9bb-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2012-05-03 18:28 wwn-0x60024e8052404f001608f3c305ead9bb-part5 -> ../../sda5

/dev/disk/by-uuid would seem to be a logical place but it also is not updated by multipath and is dangerous to use because a lvm snapshot will hand the device to the snapshot.

root@usdckvm201:/dev/disk/by-uuid# ls -l
total 0
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 1A92F8DC92F8BD77 -> ../../dm-39
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 34CCBD64CCBD20D2 -> ../../dm-33
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 3E5424485424056D -> ../../dm-43
lrwxrwxrwx 1 root root 10 2011-09-18 13:32 56d777e4-3826-4b4e-aa92-38e9e98db42a -> ../../sda1
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 5e8a9900-546c-4b2e-bd5b-70db4be7d3e7 -> ../../dm-2
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 64f269cf-71d7-4d67-a82f-0e3482f0f115 -> ../../dm-1
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 66106E98106E6ECD -> ../../dm-55
lrwxrwxrwx 1 root root 10 2012-05-03 17:55 6fb0670c-f98b-44a3-93ce-a3045aa1c21d -> ../../dm-0
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 76E822ECE822A9F7 -> ../../dm-42
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 8A64F21F64F20E27 -> ../../dm-57
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 B82ECEBF2ECE75C2 -> ../../dm-31
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 BEA46CD3A46C8FA7 -> ../../dm-54
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 E4166BCE166B9FF2 -> ../../dm-28
lrwxrwxrwx 1 root root 11 2012-05-03 17:55 EE90EA5C90EA2AB3 -> ../../dm-58

ATA devices have /dev/by-id/scsi.* names created by the current ruleset so LVM volumes, as do USB devices, thus by default most devices people would use should be scanned by a single filter rule of "filter = [ "a|/dev/disk/by-id/scsi-.*|", "r/.*/" ]"

lrwxrwxrwx 1 root root 9 May 4 13:54 ata-HL-DT-ST_DVD+_-RW_GS30N_KZ8BBJ15907 -> ../../sr0
lrwxrwxrwx 1 root root 9 May 4 13:54 ata-KINGSTON_SVP200S3240G_50026B7222013006 -> ../../sda
lrwxrwxrwx 1 root root 10 May 4 13:54 ata-KINGSTON_SVP200S3240G_50026B7222013006-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 May 4 13:54 ata-KINGSTON_SVP200S3240G_50026B7222013006-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 May 4 13:54 dm-name-cryptswap1 -> ../../dm-0
lrwxrwxrwx 1 root root 10 May 4 13:54 dm-uuid-CRYPT-PLAIN-cryptswap1_unformatted -> ../../dm-0
lrwxrwxrwx 1 root root 9 May 4 13:54 scsi-SATA_KINGSTON_SVP20050026B7222013006 -> ../../sda
lrwxrwxrwx 1 root root 10 May 4 13:54 scsi-SATA_KINGSTON_SVP20050026B7222013006-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 May 4 13:54 scsi-SATA_KINGSTON_SVP20050026B7222013006-part2 -> ../../sda2
lrwxrwxrwx 1 root root 9 May 4 13:54 wwn-0x50026b7222013006 -> ../../sda
lrwxrwxrwx 1 root root 10 May 4 13:54 wwn-0x50026b7222013006-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 May 4 13:54 wwn-0x50026b7222013006-part2 -> ../../sda2

I am marking this as a security vulnerability because if a host uses by-id/ devices it is quite possible for a guest to take over a volume if it can duplicate the UUID of other guests or the hypervisor.