Glance

Bug #1990157
Comment #11

Comment 11 for bug 1990157

Revision history for this message

Brian Rosmaita (brian-rosmaita) wrote on 2022-09-22: Re: Malicious image data modification can happen when using COW

#11

The glance team discussed this extensively at a PTG a few years ago, and rejected background computation of the hash largely because the time/load are exactly what operators are trying to avoid by using a common ceph to back nova and glance (Dan's point 4).

Plus, if the hash isn't going to be checked by nova when the image is consumed, recomputing it for each location seems kind of pointless (all we would know is that the data at location L had hash H at the time glance checked and allowed L to be added to the image, but if someone changes the data at L later (not the location uri, changes the actual data ... doesn't have to be malicious, could just be some kind of failure in the backend), there's no way to know without downloading and hashing the image. So I don't think that a verify_additional_locations option would actually increase security.

What could increase security would be to allow images that have the img_signature* properties on them to *always* go through the validation path. However, last time I looked, image signature verification is only available for the libvirt compute driver (not sure that's a big deal) and when NOT using the rbd image backend (which is the backend we're using here). But I think forcing the "normal" data path for signed images would not increase load too much (it's a PITA for users to set up an image that can be verified), and you would have the check done at the point of consumption, which is really where you want it. (This is the case for cinder, too; when using the cinder glance_store, the image-volume is cloned directly in the backend.)

Or, now that we have glance multi-store, an operator could use a second (non-rbd, non-cinder) store and tell end users to put all signed images into that store. (I think this would still require code changes in nova ... last I looked, if you turn verify_glance_signatures on for nova, if the img_signature* properties are incomplete or missing, nova puts the instance into an error state. I think we should look into changing this, though, so that there is some way to do image verification when nova and glance are using the same rbd backend. Cinder doesn't have the all-or-nothing signature verification setting, so I think this dedicated store for signed images plan could work for cinder with no code change.)

In any case, I think we should offer some way to guarantee (for the users who want it) that a consumed image is verified at the point of use. But that's not directly related to this bug.

For this bug right now, I think the situation is:
- If you don't expose locations on any end-user-facing glance-api, end users cannot modify locations via the Images API.
- If you do expose locations to end users, end users can modify them, but only on their own images. So if you set the policies for publicize_image, communitize_image, and add_member to be admin-only, an end user cannot spread a malicious image outside of their own project.

Plus, if the hash isn't going to be checked by nova when the image is consumed, recomputing it for each location seems kind of pointless (all we would know is that the data at location L had hash H at the time glance checked and allowed L to be added to the image, but if someone changes the data at L later (not the location uri, changes the actual data ... doesn't have to be malicious, could just be some kind of failure in the backend), there's no way to know without downloading and hashing the image.  So I don't think that a verify_additional_locations option would actually increase security.

What could increase security would be to allow images that have the img_signature* properties on them to *always* go through the validation path.  However, last time I looked, image signature verification is only available for the libvirt compute driver (not sure that's a big deal) and when NOT using the rbd image backend (which is the backend we're using here).  But I think forcing the "normal" data path for signed images would not increase load too much (it's a PITA for users to set up an image that can be verified), and you would have the check done at the point of consumption, which is really where you want it.  (This is the case for cinder, too; when using the cinder glance_store, the image-volume is cloned directly in the backend.)

Or, now that we have glance multi-store, an operator could use a second (non-rbd, non-cinder) store and tell end users to put all signed images into that store.  (I think this would still require code changes in nova ... last I looked, if you turn verify_glance_signatures on for nova, if the img_signature* properties are incomplete or missing, nova puts the instance into an error state.  I think we should look into changing this, though, so that there is some way to do image verification when nova and glance are using the same rbd backend.  Cinder doesn't have the all-or-nothing signature verification setting, so I think this dedicated store for signed images plan could work for cinder with no code change.)

In any case, I think we should offer some way to guarantee (for the users who want it) that a consumed image is verified at the point of use.  But that's not directly related to this bug.

For this bug right now, I think the situation is:
- If you don't expose locations on any end-user-facing glance-api, end users cannot modify locations via the Images API.
- If you do expose locations to end users, end users can modify them, but only on their own images.  So if you set the policies for publicize_image, communitize_image, and add_member to be admin-only, an end user cannot spread a malicious image outside of their own project.