OpenStack Compute (nova)

"TypeError: catching classes that do not inherit from BaseException is not allowed" is raised if volume mount fails in python3

Series zed
Bug #1984736

Bug #1984736 reported by melanie witt on 2022-08-10

This bug affects 3 people

	Status	Importance	Assigned to
OpenStack Compute (nova)	Fix Released	Medium	melanie witt
Antelope	Fix Released	Undecided	Unassigned
Wallaby	In Progress	Undecided	Unassigned
Xena	In Progress	Undecided	Unassigned
Yoga	In Progress	Undecided	Unassigned
Zed	In Progress	Undecided	Unassigned

Bug Description

Saw this on a downstream CI run where a volume mount failed:

2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [req-67e1cef8-e30a-4a47-8010-9e966fd30fce 8882186b6a324a0e9fb6fd268d337cce 8b290d651e9b42fd89c95b5e2a9a25fb - default default] [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Failed to attach 5a6a5f37-0888-44b2-9456-cf087ae8c356 at /dev/vdb: TypeError: catching classes that do not inherit from BaseException is not allowed
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Traceback (most recent call last):
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/mount.py", line 305, in mount
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] nova.privsep.fs.mount(fstype, export, mountpoint, options)
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/oslo_privsep/priv_context.py", line 247, in _wrap
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] return self.channel.remote_call(name, args, kwargs)
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/oslo_privsep/daemon.py", line 224, in remote_call
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] raise exc_type(*result[2])
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Command: mount -t nfs 192.168.1.50:/vol_cinder /var/lib/nova/mnt/724dab229d80c6a1a1e49a71c8356eed
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Exit code: 32
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Stdout: ''
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Stderr: 'Failed to connect to bus: No data available\nmount.nfs: Operation not permitted\n'
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177]
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] During handling of the above exception, another exception occurred:
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177]
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] Traceback (most recent call last):
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 7023, in _attach_volume
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] bdm.attach(context, instance, self.volume_api, self.driver,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/block_device.py", line 46, in wrapped
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] ret_val = method(obj, context, *args, **kwargs)
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/block_device.py", line 672, in attach
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] self._do_attach(context, instance, volume, volume_api,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/block_device.py", line 657, in _do_attach
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] self._volume_attach(context, volume, connector, instance,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/block_device.py", line 606, in _volume_attach
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] volume_api.attachment_delete(context,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] self.force_reraise()
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] raise self.value
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/block_device.py", line 595, in _volume_attach
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] virt_driver.attach_volume(
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 2052, in attach_volume
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] self._connect_volume(context, connection_info, instance,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 1794, in _connect_volume
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] vol_driver.connect_volume(connection_info, instance)
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/fs.py", line 113, in connect_volume
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] mount.mount(self.fstype, export, vol_name, mountpoint, instance,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/mount.py", line 407, in mount
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] mount_state.mount(fstype, export, vol_name, mountpoint, instance,
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/mount.py", line 306, in mount
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] except processutils.ProcessExecutionError():
2022-07-29 11:56:57.606 2 ERROR nova.compute.manager [instance: 6a9a59d1-861d-4536-84ed-e54d817f0177] TypeError: catching classes that do not inherit from BaseException is not allowed

There's a piece of code in nova/virt/libvirt/volume/mount.py that is catching an instance of a class instead of a class [1]:

                try:
                    nova.privsep.fs.mount(fstype, export, mountpoint, options)
                except processutils.ProcessExecutionError():

This results in a TypeError in python3 but not in python2.

Python 2.7.18 (default, Jul 1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
... raise RuntimeError('oops!')
... except RuntimeError():
... print('hi')
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
RuntimeError: oops!

Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
... raise RuntimeError('oops!')
... except RuntimeError():
... print('hi')
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
RuntimeError: oops!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 3, in <module>
TypeError: catching classes that do not inherit from BaseException is not allowed

[1] https://github.com/openstack/nova/blob/1220d53c2fd6231bb569a54f004a0e858536e970/nova/virt/libvirt/volume/mount.py#L309

Tags:

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2022-08-10: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/852795

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-08-25: Fix merged to nova (master)

Reviewed: https://review.opendev.org/c/openstack/nova/+/852795
Committed: https://opendev.org/openstack/nova/commit/f996674c36fe1a109853d88113e27b44a7a5cf45
Submitter: "Zuul (22348)"
Branch: master

commit f996674c36fe1a109853d88113e27b44a7a5cf45
Author: melanie witt <email address hidden>
Date: Wed Aug 10 23:22:47 2022 +0000

Fix exception catch when volume mount fails

    When we call nova.privsep.fs.mount(), we are catching
    ProcessExecutionError() (an instance of a class) and that is not allowed
    in python3.

This changes the except statement to catch ProcessExecutionError
(class) instead.

Closes-Bug: #1984736

Change-Id: I24a269f9809063f864a4f0443d6724dfa8703d9d

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-09-21: Fix included in openstack/nova 28.0.0.0rc1

This issue was fixed in the openstack/nova 28.0.0.0rc1 release candidate.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-10-20: Fix proposed to nova (stable/2023.1)

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/898954

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-11-14: Fix proposed to nova (stable/zed)

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/nova/+/900835

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-11-14: Fix proposed to nova (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/900836

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-11-14: Fix proposed to nova (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/900837

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2023-11-14: Fix proposed to nova (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/900838

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-02-06: Change abandoned on nova (stable/yoga)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/900836
Reason: stable/yoga branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/yoga if you want to further work on this patch.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-03-05: Change abandoned on nova (stable/wallaby)

#10

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/900838
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-03-05: Change abandoned on nova (stable/xena)

#11

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/900837
Reason: stable/xena branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/xena if you want to further work on this patch.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-04-29: Change abandoned on nova (stable/zed)

#12

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/zed
Review: https://review.opendev.org/c/openstack/nova/+/900835
Reason: stable/zed branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/zed if you want to further work on this patch.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-08-01: Fix merged to nova (stable/2023.1)

#13

Reviewed: https://review.opendev.org/c/openstack/nova/+/898954
Committed: https://opendev.org/openstack/nova/commit/baa0b4ff87a044228438cc32c27d4424fa6bc0c4
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit baa0b4ff87a044228438cc32c27d4424fa6bc0c4
Author: melanie witt <email address hidden>
Date: Wed Aug 10 23:22:47 2022 +0000

Fix exception catch when volume mount fails

    When we call nova.privsep.fs.mount(), we are catching
    ProcessExecutionError() (an instance of a class) and that is not allowed
    in python3.

This changes the except statement to catch ProcessExecutionError
(class) instead.

Closes-Bug: #1984736

Change-Id: I24a269f9809063f864a4f0443d6724dfa8703d9d
(cherry picked from commit f996674c36fe1a109853d88113e27b44a7a5cf45)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-08-01: Fix proposed to nova (unmaintained/zed)

#14

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/nova/+/925483

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2024-08-09: Fix included in openstack/nova 27.5.0

#15

This issue was fixed in the openstack/nova 27.5.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

Duplicates of this bug

Bug #2032776

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.