curtin has trouble removing partitions on redeployment

Hi Shawn,

Can you attach the curtin configuration from maas?

maas <maasuser> node get-curtin-config <system id>

Thanks!
Ryan

On Fri, Mar 25, 2016 at 8:58 AM, Shawn Iverson <email address hidden>
wrote:

> Public bug reported:
>
> Using MAAS Version 1.9.1+bzr4543-0ubuntu1 (trusty1)
>
> System partitioned using LVM as follows
>
> sda-part1 255.9 MB /boot ext2
> vg0-root 10.0 GB /ext4
> vg0-tmp 10.0 GB /tmp ext4
> vg0-var 10.0 GB /var ext4
>
> sda
> 299.4 GB
> Free: 235.2 GB
> Physical rotary
>
> vg0
> 64.0 GB
> Free: 34.0 GB
> Volume group
>
>
> First attempt to deploy after releasing results in this output:
>
> File descriptor 3 (socket:[13883]) leaked on lvremove invocation. Parent
> PID 11104: python
> File descriptor 4 (/tmp/install.log) leaked on lvremove invocation. Parent
> PID 11104: python
> File descriptor 5 (/tmp/install.log) leaked on lvremove invocation. Parent
> PID 11104: python
> Logical volume "root" successfully removed
> Logical volume "tmp" successfully removed
> Logical volume "var" successfully removed
> Volume group "root" not found
> Skipping volume group root
> File descriptor 3 (socket:[13883]) leaked on vgremove invocation. Parent
> PID 11104: python
> File descriptor 4 (/tmp/install.log) leaked on vgremove invocation. Parent
> PID 11104: python
> File descriptor 5 (/tmp/install.log) leaked on vgremove invocation. Parent
> PID 11104: python
> Volume group "vg0" successfully removed
> An error occured handling 'sda': OSError - [Errno 2] No such file or
> directory:
> '/sys/devices/pci0000:00/0000:00:03.0/0000:09:00.0/host4/target4:2:0/4:2:0:0/block/sda/sda2/holders/dm-1/holders'
> [Errno 2] No such file or directory:
> '/sys/devices/pci0000:00/0000:00:03.0/0000:09:00.0/host4/target4:2:0/4:2:0:0/block/sda/sda2/holders/dm-1/holders'
> Installation failed with exception: Unexpected error while running command.
> Command: ['curtin', 'block-meta', 'custom']
> Exit code: 3
> Reason: -
> Stdout: 'File descriptor 3 (socket:[13883]) leaked on lvremove invocation.
> Parent PID 11104: python\nFile descriptor 4 (/tmp/install.log) leaked on
> lvremove invocation. Parent PID 11104: python\nFile descriptor 5
> (/tmp/install.log) leaked on lvremove invocation. Parent PID 11104:
> python\n Logical volume "root" successfully removed\n Logical volume
> "tmp" successfully removed\n Logical volume "var" successfully removed\n
> Volume group "root" not found\n Skipping volume group root\nFile
> descriptor 3 (socket:[13883]) leaked on vgremove invocation. Parent PID
> 11104: python\nFile descriptor 4 (/tmp/install.log) leaked on vgremove
> invocation. Parent PID 11104: python\nFile descriptor 5 (/tmp/install.log)
> leaked on vgremove invocation. Parent PID 11104: python\n Volume group
> "vg0" successfully removed\nAn error occured handling \'sda\': OSError -
> [Errno 2] No such file or directory:
> \'/sys/devices/pci0000:00/0000:00:03.0/0000:09:00.0/host4/target4:2:0/4:2:0:0/block/sda/sda2/holders/dm-1/holders\'\n[Errno
> 2] No such file or directory:
> \'/sys/devices/pci0000:00/0000:00:03.0/0000:09:00.0/host4/target4:2:0/4:2:0:0/block/sda/sda2/holders/dm-1/holders\'\n'
> Stderr: ''
>
> Second attempt succeeds every t...

Attached.

Hi,

This is effectively the same issue as (LP: 1588875). What is happening is that there are several lvm partitions on a single physical volume, and when curtin attempts to shut down the first of these, it also shuts down the others at the same time. Curtin then attempts to shut down the next of the lvm partitions, but is unable to because it no longer exists. There is logic in place to allow installation to continue, but due to a bug it does not work on trusty.

I am currently working on a better design for shutting down old partitions which should be ready in the next week or so. Development on that is happening in lp:~wesley-wiedenmeier/curtin/fix-extended-clear-holders

However, since that is not available right away, there is a temporary fix in lp:~wesley-wiedenmeier/curtin/1588875

If you would like to test out the temporary fix before the full fix for shutting down old storage volumes is complete, there is a build in the ppa at: ppa:wesley-wiedenmeier/test

This bug is believed to be fixed in curtin in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.