image.stat() call sometimes fails

Bug #1625489 reported by Dr. Jens Harbott
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
ceph (Ubuntu)
Fix Released
High
Unassigned
Xenial
Triaged
High
Unassigned
Yakkety
Triaged
High
Unassigned
Zesty
Fix Released
High
Unassigned

Bug Description

The error looks like

 'utf8' codec can't decode byte 0x82 in position 24: invalid start byte.

where the position always is 24, which seems to be the end of the "block_name_prefix" string, and the byte itself it varying.
About 50% of the time the test script gives an error.

Revision history for this message
Dr. Jens Harbott (j-harbott) wrote :

Right, after doing a bit more debugging, this looks like a bug in python-rbd only. Running the following python code, I am seeing the same errors about 50% of the time:

import rados
import rbd
cluster = rados.Rados(conffile='/etc/ceph/ceph.conf')
cluster.connect()
ioctx = cluster.open_ioctx('images')
image = rbd.Image(ioctx, '41a6aadf-c655-49c8-b77d-eedd356f5349')
stat = image.stat()
print stat['size'] == image.size()

$ python test.py
True
$ python test.py
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    stat = image.stat()
  File "rbd.pyx", line 1124, in rbd.Image.stat (/build/ceph-XmVvyr/ceph-10.2.2/src/build/rbd.c:10803)
  File "rbd.pyx", line 433, in rbd.decode_cstr (/build/ceph-XmVvyr/ceph-10.2.2/src/build/rbd.c:2440)
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf4 in position 24: unexpected end of data

The "position" in the error is always 24, with varying byte values.

description: updated
tags: removed: rbd
summary: - Glance fails to retrieve image from rbd backend
+ image.stat() call sometimes fails
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ceph (Ubuntu):
status: New → Confirmed
Changed in glance (Ubuntu):
status: New → Confirmed
Revision history for this message
Dr. Jens Harbott (j-harbott) wrote :
no longer affects: glance (Ubuntu)
Revision history for this message
Dr. Jens Harbott (j-harbott) wrote :

There was a patch merged to glance_store that can serve as a workaround: https://review.openstack.org/373155

Revision history for this message
James Page (james-page) wrote :

Fix is in 10.2.4 which is up next.

Changed in ceph (Ubuntu):
status: Confirmed → Triaged
importance: Undecided → High
Changed in ceph (Ubuntu Yakkety):
status: New → Triaged
Changed in ceph (Ubuntu Xenial):
status: New → Triaged
Changed in ceph (Ubuntu Yakkety):
importance: Undecided → High
Changed in ceph (Ubuntu Xenial):
importance: Undecided → High
Revision history for this message
James Page (james-page) wrote :

10.2.5 is currently blocked in zesty proposed - marking 'Fix Committed' until I get that resolved.

Changed in ceph (Ubuntu Zesty):
status: Triaged → Fix Committed
James Page (james-page)
Changed in ceph (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Grant Morley (adogrant) wrote :

Hi all,

We are seeing the same issues in Ubuntu Trusty running ceph 10.2.6-1trusty

Is there going to to a backport fix for trusty as well? As currently we are unable to take any snapshots or create images using glance with ceph rbd.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.