Ubuntu
simplestreams package

simplestreams do not raise errors for failures behind a proxy

Bug #1790161 reported by Andres Rodriguez
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
simplestreams
New
Undecided
Unassigned
simplestreams (Ubuntu)
New
Undecided
Unassigned

Bug Description

Simplestreams do not catch errors created for being behind a proxy.

In MAAS, we were expecting / validating for json to be returned from simplestreams, but we kept seeing this failure:

[...]
            line 325, in load_content
            return json.loads(content)
          File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
            return _default_decoder.decode(s)
          File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
            obj, end = self.raw_decode(s, idx=_w(s, 0).end())
          File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
            raise JSONDecodeError("Expecting value", s, err.value) from None
        json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

To debug, I had to do this:

--- simplestreams/util.py 2016-11-28 14:54:35 +0000
+++ simplestreams/util.py 2018-08-31 14:32:54 +0000
@@ -320,6 +320,7 @@

 def load_content(content):
+ print(content)
     if isinstance(content, bytes):
         content = content.decode('utf-8')
     return json.loads(content)

And this resulted in:

2018-08-31 14:13:36 stdout: [info] body
2018-08-31 14:13:36 stdout: [info] :lang(fa) { direction: rtl; font-size: 100%; font-family: Tahoma, Roya, sans-serif; float: right; }
2018-08-31 14:13:36 stdout: [info] :lang(he) { direction: rtl; }
2018-08-31 14:13:36 stdout: [info] --></style>
2018-08-31 14:13:36 stdout: [info] </head><body id=ERR_CONNECT_FAIL>
2018-08-31 14:13:36 stdout: [info] <div id="titles">
2018-08-31 14:13:36 stdout: [info] <h1>ERROR</h1>
2018-08-31 14:13:36 stdout: [info] <h2>The requested URL could not be retrieved</h2>
2018-08-31 14:13:36 stdout: [info] </div>
2018-08-31 14:13:36 stdout: [info] <hr>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <div id="content">
2018-08-31 14:13:36 stdout: [info] <p>The following error was encountered while trying to retrieve the URL: <a href="http://[2001:67c:1562:800a::6]:5240/MAAS/images-stream/streams/v1/index.json">http://[2001:67c:1562:800a::6]:5240/MAAS/images-stream/streams
/v1/index.json</a></p>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <blockquote id="error">
2018-08-31 14:13:36 stdout: [info] <p><b>Connection to 2001:67c:1562:800a::6 failed.</b></p>
2018-08-31 14:13:36 stdout: [info] </blockquote>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <p id="sysmsg">The system returned: <i>(110) Connection timed out</i></p>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <p>The remote host or network may be down. Please try the request again.</p>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <p>Your cache administrator is <a href="mailto:webmaster?subject=CacheErrorInfo%20-%20ERR_CONNECT_FAIL&amp;body=CacheHost%3A%20localhost%0D%0AErrPage%3A%20ERR_CONNECT_FAIL%0D%0AErr%3A%20(110)%20Connection%20timed%20out%0D%
0ATimeStamp%3A%20Fri,%2031%20Aug%202018%2014%3A13%3A36%20GMT%0D%0A%0D%0AClientIP%3A%2010.246.88.6%0D%0AServerIP%3A%202001%3A67c%3A1562%3A800a%3A%3A6%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2FMAAS%2Fimages-stream%2Fstreams%2Fv1%2Findex.json%20HTTP%2F1.1%0AU
ser-Agent%3A%20python-simplestreams%2F0.1%0D%0AAccept-Encoding%3A%20gzip,%20deflate%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20keep-alive%0D%0AHost%3A%20%5B2001%3A67c%3A1562%3A800a%3A%3A6%5D%3A5240%0D%0A%0D%0A%0D%0A">webmaster</a>.</p>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <br>
2018-08-31 14:13:36 stdout: [info] </div>
2018-08-31 14:13:36 stdout: [info]
2018-08-31 14:13:36 stdout: [info] <hr>
2018-08-31 14:13:36 stdout: [info] <div id="footer">
2018-08-31 14:13:36 stdout: [info] <p>Generated Fri, 31 Aug 2018 14:13:36 GMT by localhost (squid/3.1.19)</p>
2018-08-31 14:13:36 stdout: [info] <!-- ERR_CONNECT_FAIL -->
2018-08-31 14:13:36 stdout: [info] </div>
2018-08-31 14:13:36 stdout: [info] </body></html>
2018-08-31 14:13:36 stdout: [info]

Which shows that simplestreams was failing to access the mirror because of a proxy issue, but it is not catching such issues.

Andres Rodriguez (andreserl)
Changed in simplestreams (Ubuntu):
importance: Undecided → High
importance: High → Undecided
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.