There's a long history of bugs related to attempts to limit access to files/dirs in /var/lib/nova. Considering that this is the most recent bug opened that is related to this topic, I will leave a history summary here in case we need it in the future.
= (1) April 2020: change permissions of /var/lib/nova to 640 and 750 ==
= (6) April 2022: Permission denied when trying to resize instance after CIS hardening =
And finally, the current bug we're looking at was opened due to instance resizing failing on a CIS hardened machine.
As part of hardening the umask is set to 0027. Therefore, new file permissions get 640 mode set instead of 644. This causes an access denial for the libvirt-qemu user as it no longer has read access to the disk. For example:
ubuntu@juju-da8cbf-zaza-41517351cce1-28:~$ sudo ls -al /var/lib/nova/instances/ed133a88-4f6e-4585-b208-2133c348ff35
total 20816
drwxrwx--- 2 nova nova 4096 Aug 31 16:58 .
drwxr-xr-x 6 nova nova 4096 Aug 31 16:58 ..
-rw------- 1 root root 0 Aug 31 16:58 console.log
-rw-r----- 1 nova nova 21299520 Aug 31 16:58 disk
-rw-r----- 1 nova nova 79 Aug 31 16:58 disk.info
There's a long history of bugs related to attempts to limit access to files/dirs in /var/lib/nova. Considering that this is the most recent bug opened that is related to this topic, I will leave a history summary here in case we need it in the future.
= (1) April 2020: change permissions of /var/lib/nova to 640 and 750 ==
In the focal development cycle we set file permissions under /var/lib/nova to 640 and directory permissions to 0750. That was done as part of an effort across all the openstack packages via LP: #1859422 /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=6acf39faa09 ff7cfae799513a0 5fefbefa568abf
For nova, that was handled in the following commit: https:/
= (2) April 2020: add libvirt-qemu to nova group =
As a result of the previous change, the following bug was opened because the libvirt-qemu user needed access to /var/lib/ nova/instances/ _base: LP: #1870415 /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=a6eb3638389 bb0818db6ebd338 6eb8fe500487c6
To fix that, we added the libvirt-qemu user to the nova group, since /var/lib/nova is owned by nova:nova.
That was handled in the following commit: https:/
= (3) June 2020: change permissions of /var/lib/nova to 644 and 755 =
The previous changes turned out to cause access issues and the following bug was opened: "unable to start and stop an instance post ugprade": LP: #1885269 /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=327e37428a2 5df3b96f5dfb9d0 8d3bc02caaff4f
As a result we reset file permissions under /var/lib/nova to 644 and directory permissions to 0755.
That was handled in the following commit: https:/
= (4) Sept 2020: drop libvirt-qemu from nova group =
The following bug was opened due to instance snapshots being broken: LP: #1896617 /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=864aa4e744e 9f7495caa353ba2 4efd2c4f7306cc /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=5e120a021d2 d4440fff2f6f25a c7bd6955a5e0d0
Removing the libvirt-qemu user from the nova group fixed this.
This was handled in the following commits:
https:/
https:/
= (5) March 2022: don't change permissions of /var/lib/ nova/.ssh/ id_rsa =
We then had a bug opened as the 0644 permissions for '/var/lib/ nova/.ssh/ id_rsa' were too open: LP: #1904580 /git.launchpad. net/~ubuntu- openstack- dev/ubuntu/ +source/ nova/commit/ ?id=655b04243c5 fd7e6dc32bb722b 0242c4efbb65e7
That was handled in the following commit: https:/
= (6) April 2022: Permission denied when trying to resize instance after CIS hardening =
And finally, the current bug we're looking at was opened due to instance resizing failing on a CIS hardened machine.
As part of hardening the umask is set to 0027. Therefore, new file permissions get 640 mode set instead of 644. This causes an access denial for the libvirt-qemu user as it no longer has read access to the disk. For example:
ubuntu@ juju-da8cbf- zaza-41517351cc e1-28:~ $ sudo ls -al /var/lib/ nova/instances/ ed133a88- 4f6e-4585- b208-2133c348ff 35
total 20816
drwxrwx--- 2 nova nova 4096 Aug 31 16:58 .
drwxr-xr-x 6 nova nova 4096 Aug 31 16:58 ..
-rw------- 1 root root 0 Aug 31 16:58 console.log
-rw-r----- 1 nova nova 21299520 Aug 31 16:58 disk
-rw-r----- 1 nova nova 79 Aug 31 16:58 disk.info