UnicodeEncodeError in st_delete() when object has non-ascii char

Bug #1431866 reported by Jordan Pittier
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-swiftclient
Fix Released
Undecided
Unassigned

Bug Description

Hi,

How to reproduce :
swift post éà
touch /tmp/ç
swift upload --segment-threads 10 --segment-size $((1024*1024)) éà /tmp/ç

jordan@jordan-XPS13:~$ swift delete éà
Traceback (most recent call last):
  File "/usr/local/bin/swift", line 11, in <module>
    sys.exit(main())
  File "/opt/stack/python-swiftclient/swiftclient/shell.py", line 1316, in main
    globals()['st_%s' % args[0]](parser, argv[1:], output)
  File "/opt/stack/python-swiftclient/swiftclient/shell.py", line 146, in st_delete
    output_manager.print_msg('{0}{1}'.format(p, a))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 4: ordinal not in range(128)

Possible cause :
o = r.get('object', '') returns a Unicode object which can not be passed to str.format()

Ugly fix:
                if o is not None:
                    o = o.encode('utf-8')

Changed in python-swiftclient:
status: New → Incomplete
status: Incomplete → New
Revision history for this message
Christian Schwede (cschwede) wrote :

I don't see this error on current master, but that might be related to different terminal settings.

Could you share your terminal settings?

Revision history for this message
Jordan Pittier (jordan-pittier) wrote :

Hi
What setting are you interested in ?

I have:
LANG=en_US.UTF-8
GDM_LANG=en_US
LANGUAGE=en_US
LC_TIME=fr_FR.UTF-8
LC_NAME=fr_FR.UTF-8

I can reproduce with python-swiftclient-2.4.0, I have checkouted swift-master on the 16th of April. The 4 commands I provided are enough to repro the issue. It's likely it's a terminal setting issue.

Revision history for this message
Alistair Coles (alistair-coles) wrote :

Reproduced on my SAIO:

````
swift@saio-1:~/swiftclient$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.4 LTS
Release: 12.04
Codename: precise
swift@saio-1:~/swiftclient$ git log --pretty=oneline -n 1
ea6d2bf74d8cf7f5faeff3d3052265db717db99e Merge "Add socket-level read timeout parameter"
swift@saio-1:~/swiftclient$ swift upload --segment-threads 10 --segment-size $((1024*1024)) éà /tmp/ç
/tmp/ç
swift@saio-1:~/swiftclient$ swift delete éà
Traceback (most recent call last):
  File "/usr/local/bin/swift", line 10, in <module>
    execfile(__file__)
  File "/home/swift/swiftclient/bin/swift", line 24, in <module>
    sys.exit(main())
  File "/home/swift/swiftclient/swiftclient/shell.py", line 1309, in main
    globals()['st_%s' % args[0]](parser, argv[1:], output)
  File "/home/swift/swiftclient/swiftclient/shell.py", line 139, in st_delete
    output_manager.print_msg('{0}{1}'.format(p, a))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 4: ordinal not in range(128)
````

Changed in python-swiftclient:
status: New → Confirmed
Revision history for this message
Alistair Coles (alistair-coles) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-swiftclient (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/184336

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-swiftclient (master)

Reviewed: https://review.openstack.org/176945
Committed: https://git.openstack.org/cgit/openstack/python-swiftclient/commit/?id=bb252130ac2f332110172b8e2094dc629c8a896b
Submitter: Jenkins
Branch: master

commit bb252130ac2f332110172b8e2094dc629c8a896b
Author: Tim Burke <email address hidden>
Date: Mon May 18 10:14:09 2015 -0700

    Always decode command-line arguments as UTF-8

    There was always an implicit assumption that they were UTF-8 before, and
    by converting them to unicode we close another hole allowing raw bytes
    to appear in user-facing messages.

    Closes-Bug: #1431866
    Change-Id: If2e41d9a592c3ad02818e9c6f0959fd4b73cd0e0

Changed in python-swiftclient:
status: Confirmed → Fix Committed
Changed in python-swiftclient:
milestone: none → 2.5.0
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-swiftclient (master)

Reviewed: https://review.openstack.org/184336
Committed: https://git.openstack.org/cgit/openstack/python-swiftclient/commit/?id=ce569f46517e10f2ce0d27e9ee0a922ad1d84e2f
Submitter: Jenkins
Branch: master

commit ce569f46517e10f2ce0d27e9ee0a922ad1d84e2f
Author: Tim Burke <email address hidden>
Date: Mon May 18 08:05:02 2015 -0700

    Centralize header parsing

    All response headers are now exposed as unicode objects. Any
    url-encoding is interpretted as UTF-8; if that causes decoding to fail,
    the url-encoded form is returned.

    As a result, deleting DLOs with unicode characters will no longer raise
    UnicodeEncodeErrors under Python 2.

    Related-Bug: #1431866
    Change-Id: Idb111c5bf3ac1f5ccfa724b3f4ede8f37d5bfac4

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.