[Impact]
* grub-pc currently installs new core to MBR and installs new modules to /boot in an unsafe manner, which may lead to incompatible combination of MBR and modules resulting in failure to boot.
[Test Case]
* Install using old point media, of an old release. I.e. 16.04.(p-1) for testing upgrades to 18.04 sru, in bios mode.
* backup the contents of /boot
* First we will test a case where target boot device exists, yet writes to it are denied, thus one can update modules, but cannot update the MBR.
* install /etc/apparmor.d/usr.sbin.grub-install profile
"/usr/sbin/grub-install" {
capability,
mount,
ptrace,
signal,
unix,
file,
deny /dev/* w,
}
and load it with
sudo apparmor_parser -r usr.sbin.grub-install
* Upgrade to the package from next series-proposed, non-interactively
* Observe the package installation has failed, the grub-pc package is in a broken state.
* Compare the backup of /boot with current /boot, it should have remained the same, and is different to modules in /usr/lib/grub/i386-pc
* Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
* Reboot, reboot should be successful. If possible observe the version number in the grub menu, it should still be old.
* Now we will test a case where a non-existing device ended up being configured in debconf. For example, due to old buggy cloud-init having been used during first boot, or because the VM got migrated from one hardware configuration to another (i.e. offline switch from SCSI sda, to VIRTIO vda).
* Configure invalid grub-pc/install_devices to a non existing device (e.g. /dev/sdk)
* Attempt non-interactive configuration of the grub-pc package
* Observe the package fails, and the grub-pc package remains in a broken state.
* Compare the backup of /boot with current /boot, it should have remained the same, and is different to modules in /usr/lib/grub/i386-pc
* Reboot, reboot should be successful. If possible observe the version number in the grub menu, it should still be old.
* Try to configure all the packages, interactively (i.e. using $ sudo dpkg --configure -a or by using $ sudo apt install -f) and ensure to select the right drive for grub installation offer
* Observe that now /boot matches /usr/lib/grub/i386-pc contents, and is different from the backup taken at the start.
* Reboot should be successful, and grub menu should have the new version number finally
[Regression Potential]
* Existing call to grub-install, is now split into two. And when any
devices fail to configure, non-interactively error is reported just
like it was already done with the interactive case.
It means, it will fail configuration of the package, where
previously it would report success. However, it is now safer and
keeps the system bootable, whilst having unconfigured
packages. This mostly affects non-interactive upgrades, as the
interactive ones have always shown critical errors trying to
correct grub-pc installation problems.
The first stage of grub-install only tries to update the MBR,
whilst utilizing tmpdirectory to create the core image. This is a
slight increase in disk space usage, as previously core was created
in-pace in /boot. Then whilst tmpdir is still populated, /boot
modules and core are upgraded.
These changes do not address multi-mbr systems, or cases where
updating modules fails. For example, it is possible that MBR update
is successful, yet writting updated modules fails (out of disk space),
in such scenario MBR is not rolled back to previous one. Or a case
where MBR updates have succeeded, but only on some devices.
A choice has been made to update modules in /boot, if at least one
device has a successful MBR update. No backup, or rollback of MBR is
performed if module updates fail. This is tricky to do, as it is
uncertain if current MBR matches the core.img & boot.img from /boot, or
if some other bootsectors code was in use before. Ideally in the
future, grub-install itself will be able to stage module updates, and
commit/rollback them upon successful MBR update.
[Other Info]
* Original bug report description
Currently on upgrade if the debconf variable for the drive to install grub-pc to point to a non-existent drive, the grub package will nevertheless happily carry on and the postinst will exit 0 - as a result leaving the /boot/grub contents and the MBR in an inconsistent state, which due to recent ABI changes will leave the system unbootable on reboot.
Three changes required in order to make grub upgrades more resilient:
- exit non-zero from the postinst when the drive targets are invalid, so that we signal to the user that there is a problem BEFORE they reboot and give them the opportunity to deal with it. This is addressed by https://code.launchpad.net/~xnox/grub/+git/grub/+merge/388383
- include a check for target drive validity in the grub preinst, not just in the postinst, so that we avoid unpacking boot assets onto disk that might be incorrectly used by another package (despite grub-pc being in an unconfigured state) and still render the system unbootable; this will in general break release upgrades for affected users, but a failing postinst would do the same anyway, and failing early should leave the package manager in a more consistent state overall. This is addressed by https://code.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/+merge/388423
- modify grub-install so that it handles the flaky part of the install - updating the BIOS disks - FIRST, and aborts if this fails; instead of the current behavior, which is that /boot/grub is updated on disk first, then it attempts to install to the BIOS disk, and if this part fails, no rollback of the contents of /boot/grub is possible.
Status changed to 'Confirmed' because the bug affects multiple users.