Image uploads to the filesystem driver are not fully atomic

Bug #1297451 reported by Nicolas Simonds
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
glance_store
In Progress
Wishlist
Srinivas Sakhamuri

Bug Description

When uploading to the filesystem store, the image is uploaded, checksummed, and added to the database, with appropriate rollback operations in place in case of failures. This is all good, but there are various corner-cases where partially-uploaded files can be left lying around in the filesystem_store_datadir.

Mostly this kind of thing interferes with operational processes we have that are external to Glance and not Glance itself. e.g., we have a monitoring process to reconcile the list/checksums of glance images against what's on-disk, and this occasionally goes funny on us. Additionally, we have a side-band backup/replication process that ends up occasionally transferring partial files for no reason.

Using a temporary upload space and moving the file into place on the filesystem store would cause a lot of our co-processes to work more cleanly, as well as reducing the risk of the _delete_partial job deleting the wrong thing should it go sideways.

Tags: lite-spec
Revision history for this message
Nicolas Simonds (nicolas.simonds) wrote :

Because the Launchpad sync job is apparently asleep:

https://review.openstack.org/82899

Changed in glance:
assignee: nobody → Nicolas Simonds (nicolas.simonds)
Changed in glance:
status: New → In Progress
Revision history for this message
haruka tanizawa (h-tanizawa) wrote :

How do you think about Zhi Yan Liu's comment in the above patch?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on glance (master)

Change abandoned by Nicolas Simonds (<email address hidden>) on branch: master
Review: https://review.openstack.org/82899
Reason: No longer applicable -- this change belongs in the glance_store project

affects: glance → glance-store
Changed in glance-store:
importance: Undecided → Wishlist
tags: added: lite-spec
Revision history for this message
Srinivas Sakhamuri (srinivas-sakhamuri) wrote :
Changed in glance-store:
assignee: Nicolas Simonds (nicolas.simonds) → Srinivas Sakhamuri (srinivas-sakhamuri)
Revision history for this message
Ian Cordasco (icordasc) wrote :

https://review.openstack.org/#/c/242084/ still needs review by cores. After next Friday, I will add this to our weekly meeting agenda.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.