Glance cannot determine the size of a "fake" image when the image is being uploaded to the Swift

sla1 for 9.0-updates

sla2 for 9.0-updates

Hello,
I assume you were using glance v2 when faced with this message.
Glance uses chunked transfer encoding when uploading images into back-end. Glance v2 also does not trust provided image-size and it calculates the size based on input chunk(s) size.
While I agree we can do some enhancements here and respect provided image size I think that this is the feature rather than bug because we don't break anything here. We just need to change the approach how to store image when size is provided in v2. It doesn't bring any business value so I guess we can move it to wishlist.

Uploading of a normal image produces the same message;
root@node-3:~# glance image-create --disk-format qcow2 --container-format bare --file Fedora-Cloud-Base-23-20151030.x86_64.qcow2 --name fedora --progress
[=============================>] 100%
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | 38d62e2e1909c89f72ba4d5f5c0005d5 |
| container_format | bare |
| created_at | 2016-08-30T05:20:09Z |
| disk_format | qcow2 |
| id | 30ceae23-1e6f-4b28-a2e0-9323c4425cae |
| min_disk | 0 |
| min_ram | 0 |
| name | fedora |
| owner | 0fec4b413ec24e2aab4b5bad4028e114 |
| protected | False |
| size | 234363392 |
| status | active |
| tags | [] |
| updated_at | 2016-08-30T05:20:40Z |
| virtual_size | None |
| visibility | private |
+------------------+--------------------------------------+
2016-08-30 05:20:09.373 10887 DEBUG glance_store._drivers.swift.store [req-92c4d130-1609-46b3-9f3e-b125232b3217 63c26adccf4146a7833324829dcfa8de 0fec4b413ec24e2aab4b5bad4028e114 - - -] Adding image object '30cea
e23-1e6f-4b28-a2e0-9323c4425cae' to Swift add /usr/lib/python2.7/dist-packages/glance_store/_drivers/swift/store.py:538
2016-08-30 05:20:09.374 10887 DEBUG glance_store._drivers.swift.store [req-92c4d130-1609-46b3-9f3e-b125232b3217 63c26adccf4146a7833324829dcfa8de 0fec4b413ec24e2aab4b5bad4028e114 - - -] Cannot determine image size. Adding as a segmented object to Swift. add /usr/lib/python2.7/dist-packages/glance_store/_drivers/swift/store.py:565

Uploading to RBD produces these WARNING messages:
2016-08-30 04:56:02.814 246987 DEBUG glance_store._drivers.rbd [req-adc5abea-6fd7-4203-854e-c3ff6f12ac0a 81306ea21d84459b808860a519731620 91e2e15dce2e438cad8171191fa31bcc - - -] creating image 9b4b6671-019c-4d69-8a2c-e6b1a18fb78d with order 23 and size 0 add /usr/lib/python2.7/dist-packages/glance_store/_drivers/rbd.py:382
2016-08-30 04:56:02.815 246987 WARNING glance_store._drivers.rbd [req-adc5abea-6fd7-4203-854e-c3ff6f12ac0a 81306ea21d84459b808860a519731620 91e2e15dce2e438cad8171191fa31bcc - - -] since image size is zero we will be doing resize-before-write for each chunk which will be considerably slower than normal
2016-08-30 04:56:03.178 246987 DEBUG glance_store._drivers.rbd [req-adc5abea-6fd7-4203-854e-c3ff6f12ac0a 81306ea21d84459b808860a519731620 91e2e15dce2e438cad8171191fa31bcc - - -] resizing image to 8192 KiB add /usr/lib/python2.7/dist-packages/glance_store/_drivers/rbd.py:4112

This is in connection with 16149

The size was used in v1 IIRC. In glance v2 we don't pass the size to glance_store anymore. So we don't trust client info about size and we do images in chunks when uploading to swift despite of image size. IIRC Nova doesn't know the size when creating an image so we don't have a size from nova in most cases. Additionally, it helps us not to assume maximal file size allowed in swift and just store any file we want. HTH.

Hello! The fix https://review.openstack.org/#/c/383745/ in the upstream verson has been merged. Could you please backport this fix? Shall I open a new request?

@Anatolii, I've created review for mitaka in upstream https://review.openstack.org/#/c/388626/ After the patch will be merged in upstream we consume the patch via regular code sync.

@Sergii, since this fix is not important at all (not a security-related fix), I presume it will not be backported to Mitaka release. All in all, I suggest backporting this fix to MOS 9 version only, since this version now has full support from Mirantis.

CR for MOS 9: https://review.fuel-infra.org/#/c/27749/

Reviewed: https://review.fuel-infra.org/27749
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 7cd77597cd23011ae996bd21fe280a54a6b8eca0
Author: Darja Shakhray <email address hidden>
Date: Mon Oct 24 10:54:07 2016

Fix dbg msg when swift can't determine image size

Some enhancement for debug message that do not
confuse people who debugging glance.

Closes-bug: #1617562

Change-Id: I425cb28cdc95c6a62bc7e65dc91105c5d2418952

Verified on 9.2 snapshot #464.

Fix proposed to branch: mcp/newton
Change author: Darja Shakhray <email address hidden>
Review: https://review.fuel-infra.org/33458

Change abandoned by Ihor Kalnytskyi <email address hidden> on branch: mcp/newton
Review: https://review.fuel-infra.org/33458
Reason: I don't think that changing *debug log phrasing* is actually worth to maintain the patch in downstream. It's been fixed in upstream to provide more useful description so let's keep in sync with upstream.