Comment 19 for bug 1908452

A reproducer would be nice you say? Well, you can point your MAAS server to this URL for MAAS images:
  https://images.maas.io/ephemeral-v3/daily/

Then you can simulate whatever-the-heck is going on in our lab environment by adding an iptables rule that will drop packets containing the server cert from the image server:

-------------
$ sudo iptables -A INPUT -p tcp -s 91.189.88.136 -m string --string maas.io --algo bm -j DROP
-------------

Then initiate an image sync. You'll find that roughly 25% of API connections hang indefinitely:

-------------
ubuntu@maas:~$ wget --tries=1 http://localhost:5240/MAAS/rpc -O /dev/null
--2021-01-29 20:59:42-- http://localhost:5240/MAAS/rpc
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:5240... connected.
HTTP request sent, awaiting response...
-------------

What's that? Would be nice if it were a standalone reproducer? Well, you can do the same thing with the same iptables rule and a simple script that makes the same simplestreams API call as regiond does:

-------------------------
$ cat repro.py
#!/usr/bin/env python3

from simplestreams.contentsource import RequestsUrlReader

url = "https://images.maas.io/ephemeral-v3/stable/streams/v1/index.sjson"
r = RequestsUrlReader(url)
-------------------------

This will hang seemingly forever. strace can be used to demonstrate it is also hung in a read call, as I showed regiond was in Comment #11:

-------------------------
$ sudo strace -p 562333
strace: Process 562333 attached
read(20, ^Cstrace: Process 562333 detached
 <detached ...>
-------------------------

And if you ^c it, it will show same backtrace I extracted from regiond in comment #17:

-------------------------
$ ./repro.py
^CTraceback (most recent call last):
  File "./repro.py", line 6, in <module>
    r = RequestsUrlReader(url)
  File "/usr/lib/python3/dist-packages/simplestreams/contentsource.py", line 381, in __init__
    self.req = requests.get(url, stream=True, auth=auth, headers=headers)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 996, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 366, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1914, in do_handshake
    result = _lib.SSL_do_handshake(self._ssl)
KeyboardInterrupt
-------------------------