Glance Client

Test failures with urllib3-2.2.1

Bug #2069684 reported by Michał Górny on 2024-06-18

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Glance Client	In Progress	Undecided	Unassigned

Bug Description

When urllib3 is upgraded beyond version 2, 6 tests fail. This is a significant problem for Gentoo since python-glanceclient is literally the last package requiring urllib3 < 2, and that version of urllib3 is pretty broken for us already (including hang-level bugs). I was able to reproduce these failures by running, on git master (59331d56eb9f7d4bb19f05c251e17161a3e8c98d), the following commands:

$ tox -e py311 # passes
$ . .tox/py311/bin/activate
$ pip install -U urllib3
$ stestr run # fails

==============================
Failed 6 tests - output below:
==============================

glanceclient.tests.unit.test_http.TestClient.test_http_chunked_response(session)
--------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):

      File "/tmp/python-glanceclient/glanceclient/tests/unit/test_http.py", line 351, in test_http_chunked_response
    self.assertEqual([data], list(body))
                             ^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/common/http.py", line 332, in _close_after_stream
for chunk in response.iter_content(chunk_size=chunk_size):

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 1060, in stream
    data = self.read(amt=amt, decode_content=decode_content)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 980, in read
    data = self._decoded_buffer.get(amt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 276, in get
ret.write(chunk)

TypeError: a bytes-like object is required, not 'str'

glanceclient.tests.unit.test_http.TestClient.test_log_request_id_once(httpclient)
---------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):

      File "/tmp/python-glanceclient/glanceclient/tests/unit/test_http.py", line 468, in test_log_request_id_once
    self.assertEqual([data], list(body))
                             ^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/common/http.py", line 332, in _close_after_stream
for chunk in response.iter_content(chunk_size=chunk_size):

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 980, in read
    data = self._decoded_buffer.get(amt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 276, in get
ret.write(chunk)

TypeError: a bytes-like object is required, not 'str'

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
curl -g -i -X GET -H 'b'Content-Type': b'application/octet-stream'' -H 'b'X-Auth-Token': {SHA1}6367c48dd193d56ea7b0baad25b19455e529f5ee' -H 'User-Agent: python-glanceclient' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' http://example.com:9292/v1/images/
GET http://example.com:9292/v1/images/ 200
GET call to image for http://example.com:9292/v1/images/ used request id 1234

HTTP/0.0 200 None
Content-Type: application/octet-stream
x-openstack-request-id: 1234

glanceclient.tests.unit.test_http.TestClient.test_log_request_id_once(session)
------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):

      File "/tmp/python-glanceclient/glanceclient/tests/unit/test_http.py", line 468, in test_log_request_id_once
    self.assertEqual([data], list(body))
                             ^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/common/http.py", line 332, in _close_after_stream
for chunk in response.iter_content(chunk_size=chunk_size):

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 980, in read
    data = self._decoded_buffer.get(amt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 276, in get
ret.write(chunk)

TypeError: a bytes-like object is required, not 'str'

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
REQ: curl -g -i -X GET http://example.com:9292/v1/images/ -H "Content-Type: application/octet-stream" -H "User-Agent: python-glanceclient" -H "X-Auth-Token: {SHA256}6ca13d52ca70c883e0f0bb101e425a89e8624de51db2d2392593af6a84118090"
GET http://example.com:9292/v1/images/ 200
RESP: [200] Content-Type: application/octet-stream x-openstack-request-id: 1234
RESP BODY: Omitted, Content-Type is set to application/octet-stream. Only application/json, text/plain responses have their bodies logged.
GET call to image for http://example.com:9292/v1/images/ used request id 1234

glanceclient.tests.unit.test_http.TestClient.test_http_chunked_response(httpclient)
-----------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):

      File "/tmp/python-glanceclient/glanceclient/tests/unit/test_http.py", line 351, in test_http_chunked_response
    self.assertEqual([data], list(body))
                             ^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/common/http.py", line 332, in _close_after_stream
for chunk in response.iter_content(chunk_size=chunk_size):

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 980, in read
    data = self._decoded_buffer.get(amt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/response.py", line 276, in get
ret.write(chunk)

TypeError: a bytes-like object is required, not 'str'

glanceclient.tests.unit.test_ssl.TestHTTPSVerifyCert.test_v2_requests_valid_cert_verification
---------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
self._validate_conn(conn)

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
conn.connect()

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connection.py", line 652, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connection.py", line 805, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
               ^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 465, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 509, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/lib/python3.11/ssl.py", line 1104, in _create
self.do_handshake()

File "/usr/lib/python3.11/ssl.py", line 1382, in do_handshake
self._sslobj.do_handshake()

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '0.0.0.0'. (_ssl.c:1010)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
raise new_e

urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '0.0.0.0'. (_ssl.c:1010)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='0.0.0.0', port=34009): Max retries exceeded with url: /v2/images/image123 (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '0.0.0.0'. (_ssl.c:1010)")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

      File "/tmp/python-glanceclient/glanceclient/common/http.py", line 268, in _request
    resp = self.session.request(method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/.tox/py311/lib/python3.11/site-packages/requests/adapters.py", line 517, in send
raise SSLError(e, request=request)

requests.exceptions.SSLError: HTTPSConnectionPool(host='0.0.0.0', port=34009): Max retries exceeded with url: /v2/images/image123 (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '0.0.0.0'. (_ssl.c:1010)")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/tmp/python-glanceclient/glanceclient/tests/unit/test_ssl.py", line 155, in test_v2_requests_valid_cert_verification
gc.images.get('image123')

      File "/tmp/python-glanceclient/glanceclient/v2/images.py", line 197, in get
    return self._get(image_id)
           ^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/glanceclient/common/utils.py", line 652, in inner
    return RequestIdProxy(wrapped(*args, **kwargs))
                          ^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/glanceclient/v2/images.py", line 190, in _get
    resp, body = self.http_client.get(url, headers=header)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

      File "/tmp/python-glanceclient/glanceclient/common/http.py", line 314, in get
    return self._request('GET', url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/common/http.py", line 281, in _request
raise exc.CommunicationError(message=message)

glanceclient.exc.CommunicationError: Error finding address for https://0.0.0.0:34009/v2/images/image123: HTTPSConnectionPool(host='0.0.0.0', port=34009): Max retries exceeded with url: /v2/images/image123 (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '0.0.0.0'. (_ssl.c:1010)")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

      File "/usr/lib/python3.11/unittest/mock.py", line 1378, in patched
    return func(*newargs, **newkeywargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/tmp/python-glanceclient/glanceclient/tests/unit/test_ssl.py", line 158, in test_v2_requests_valid_cert_verification
self.fail('Certificate failure message is received')

File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)

AssertionError: Certificate failure message is received

glanceclient.tests.unit.test_ssl.TestHTTPSVerifyCert.test_v2_requests_valid_cert_verification_no_compression
------------------------------------------------------------------------------------------------------------