Unable to snapshot an instance with ephemeral RBD

Bug #1261590 reported by Dmitry Borodaenko
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Dmitry Borodaenko

Bug Description

When creating a snapshot from a VM instance, following error appears in Nova compute log:

  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1403, in snapshot
    snapshot_backend.snapshot_extract(out_path, image_format)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 658, in snapshot_extract
    images.convert_image(snap, target, out_format)
  File "/usr/lib/python2.6/site-packages/nova/virt/images.py", line 179, in convert_image
    utils.execute(*cmd, run_as_root=run_as_root)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: qemu-img convert -O raw rbd:compute/5fac0ad2-b39e-4006-90a8-b331ec38a57e_disk /var/lib/nova/instances/snapshots/tmpz0Nq0E/ab5eae70d5114c4e965de7163452934c
Exit code: 1
Stdout: ''
Stderr: "qemu-img: error connecting\nqemu-img: Could not open 'rbd:compute/5fac0ad2-b39e-4006-90a8-b331ec38a57e_disk': No such file or directory\nqemu-img: Could not open 'rbd:compute/5fac0ad2-b39e-4006-90a8-b331ec38a57e_disk'\n"

Apparently Nova doesn't have (or doesn't give qemu-img) enough permissions to access the VM image in RBD. When Rbd.snapshot_extract is patched to pass run_as_root to convert_image (which in turn runs qemu-img), snapshot is successfully created.

Tags: ceph
Revision history for this message
Dmitry Borodaenko (angdraug) wrote :
Changed in fuel:
status: Confirmed → In Progress
assignee: Ryan Moe (rmoe) → Dmitry Borodaenko (dborodaenko)
Revision history for this message
Dmitry Borodaenko (angdraug) wrote :

Ticket OSCI-974 created to propagate this patch to Nova packages in Mirantis OpenStack 4.0.

Revision history for this message
Dmitry Borodaenko (angdraug) wrote :

Verified with ISO #165. Creating snapshots from VMs with Ceph RBD backend works as expected.

Upstream (improved) version of the same patch: https://review.openstack.org/#/c/62592/

Changed in fuel:
status: In Progress → Fix Committed
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.