list operations fail using Python binding with object of type 'NoneType' has no len()

Bug #1431449 reported by Gavin B
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
python-neutronclient
Fix Released
Critical
Anand Shanmugam

Bug Description

A recent change in python-neutronclient has led to simple operations such as a neutron net-list failing when using the python binding (the CLI operation works OK).

(test_venv2)cetest@cer405n0046:~/gavin$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from os import environ
>>> from neutronclient.v2_0.client import Client as NeutronClient
>>> nc = NeutronClient(username=environ['OS_USERNAME'], password=environ['OS_PASSWORD'], tenant_name=environ['OS_TENANT_NAME'], auth_url=environ['OS_AUTH_URL'], ca_cert=environ.get('OS_CACERT', None))
>>> nc.list_networks()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params
    ret = self.function(instance, *args, **kwargs)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 568, in list_networks
    **_params)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 315, in list
    for r in self._pagination(collection, path, **params):
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 328, in _pagination
    res = self.get(path, params=params)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 301, in get
    headers=headers, params=params)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 278, in retry_request
    headers=headers, params=params)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 201, in do_request
    self._check_uri_length(action)
  File "/home/cetest/gavin/test_venv2/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 188, in _check_uri_length
    uri_len = len(self.httpclient.endpoint_url) + len(action)
TypeError: object of type 'NoneType' has no len()
>>>
(test_venv2)cetest@cer405n0046:~/gavin$ pip freeze | grep neutron
python-neutronclient==2.3.11.post21

We have seen the problem with python-neutronclient==2.3.9.46.gde7bb1e

But not with python-neutronclient==2.3.9.31.g64b2d8a

Other similar list operations fail the same way.

Revision history for this message
Gavin B (gavin-brebner-orange) wrote :
Download full text (4.7 KiB)

Virtual environment where the only thing I change is the python-neutronclient :

(toto)cetest@cer405n0046:~/gavin$ python repro_neutron.py
Traceback (most recent call last):
  File "repro_neutron.py", line 13, in <module>
    nc.list_networks()
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params
    ret = self.function(instance, *args, **kwargs)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 568, in list_networks
    **_params)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 315, in list
    for r in self._pagination(collection, path, **params):
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 328, in _pagination
    res = self.get(path, params=params)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 301, in get
    headers=headers, params=params)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 278, in retry_request
    headers=headers, params=params)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 201, in do_request
    self._check_uri_length(action)
  File "/home/cetest/gavin/toto/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 188, in _check_uri_length
    uri_len = len(self.httpclient.endpoint_url) + len(action)
TypeError: object of type 'NoneType' has no len()
(toto)cetest@cer405n0046:~/gavin$ pip freeze | grep neutron
python-neutronclient==2.3.11.post24
(toto)cetest@cer405n0046:~/gavin$ pip uninstall python-neutronclient
Uninstalling python-neutronclient:
  /home/cetest/gavin/toto/bin/neutron
  /home/cetest/gavin/toto/lib/python2.7/site-packages/neutronclient
  /home/cetest/gavin/toto/lib/python2.7/site-packages/python_neutronclient-2.3.11.post24-py2.7.egg-info
Proceed (y/n)? y
  Successfully uninstalled python-neutronclient
(toto)cetest@cer405n0046:~/gavin$ pip install python-neutronclient
Downloading/unpacking python-neutronclient
  http://10.22.157.17/pypi/latest/python-neutronclient/ uses an insecure transport scheme (http). Consider using https if 10.22.157.17 has it available
  http://pypi.python.org/simple/python-neutronclient/ uses an insecure transport scheme (http). Consider using https if pypi.python.org has it available
  Downloading python_neutronclient-2.3.11-py2.py3-none-any.whl (199kB): 199kB downloaded
Requirement already satisfied (use --upgrade to upgrade): iso8601>=0.1.9 in ./toto/lib/python2.7/site-packages (from python-neutronclient)
Requirement already satisfied (use --upgrade to upgrade): requests>=2.2.0,!=2.4.0 in ./toto/lib/python2.7/site-packages (from python-neutronclient)
Requirement already satisfied (use --upgrade to upgrade): cliff>=1.7.0 in ./toto/lib/python2.7/site-packages (from python-neutronclient)
Requirement already satisfied (use --upgrade to upgrade): oslo.serialization>=1.2.0 in ./toto/lib/python2.7/site-packages (from python-neutronclient)
Requirement already satisfied (use --upgrade t...

Read more...

description: updated
Changed in python-neutronclient:
assignee: nobody → Anand Shanmugam (anand1712)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-neutronclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/164456

Changed in python-neutronclient:
status: New → In Progress
Akihiro Motoki (amotoki)
Changed in python-neutronclient:
importance: Undecided → Critical
milestone: none → 2.3.12
Revision history for this message
Akihiro Motoki (amotoki) wrote :

This is a critical regression in commit 4b6ed760d4303744907feefd81e60f38ae3750ef.
It prevents all library usage of neutronclient without passing endpoint_url and token.
(Note that Horizon is not affected as Horizon passes endpoint_url and token when initializing neutronclient object)
IMHO we should revert the change first and then explore a better way to check max_uri_length again.

Revision history for this message
Akihiro Motoki (amotoki) wrote :

After more investigation, the situation is no so simple.

- Before SessionClient support, check_max_uri() worked as expected.
- After SessionClient support was introduced, it seems that check_max_uri() did not work (bug 1422736)
- Then the fix for bug 1422736 ( https://review.openstack.org/#/c/160923/ ) introduced this bug.

As you can see, just reverting some patch does not solve the problem.

Apart from the solution, our test framework for the Client class is not sufficient. This failure comes from the fact most neutron client tests mock out "Client" class and we don't have good unit tests for Client class. Most tests are done as a part of CLI tests unfortunately.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-neutronclient (master)

Reviewed: https://review.openstack.org/164456
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=5f0f2802a83646dc4cad29078e2c8e82c69dc041
Submitter: Jenkins
Branch: master

commit 5f0f2802a83646dc4cad29078e2c8e82c69dc041
Author: Anand Shanmugam <email address hidden>
Date: Sat Mar 14 23:47:13 2015 +0530

    Fix failures when calling list operations using Python binding

    This bug is caused because of the regression caused by
    I1b719bed406b83c5f2deac06e127798a91f51ad7 . The original bug was
    raised because check_max_uri was not working after the introduction
    of sessionclient. The change created a regression in which the
    python bindings of the neutron client was not usable and causing
    traces.

    The fix is to revert the change id and add a new check_max_uri to
    the sessionclient. Now uri length will be checked in sessionclient
    and httpclient as well. please see bug for further info.

    Closes-Bug: #1431449

    Change-Id: Ief2352a90bb75a76e8c671d51beb0fb7a53a22f9

Changed in python-neutronclient:
status: In Progress → Fix Committed
Kyle Mestery (mestery)
Changed in python-neutronclient:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.