keystone api v3/ec2tokens return 500

Bug #1895670 reported by Ethan Lynn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
New
Undecided
Unassigned

Bug Description

Some heat users still use ec2tokens api for scaling authentication, but when heat tried to authenticate through ec2tokens API, it got 500 error code.

Here is the keystone api logs
172.18.3.0 - - [15/Sep/2020:11:24:10 +0000] "GET / HTTP/1.1" 300 266 "-" "kube-probe/1.14"
2020-09-15 11:24:14.355421 2020-09-15 11:24:14.354 8 DEBUG keystone.server.flask.request_processing.req_logging [req-6cf43a75-4c37-4942-ab87-a2b12e10434c - - - - -] REQUEST_METHOD: `POST` log_request_info /usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/req_logging.py:27\x1b[00m
2020-09-15 11:24:14.356149 2020-09-15 11:24:14.355 8 DEBUG keystone.server.flask.request_processing.req_logging [req-6cf43a75-4c37-4942-ab87-a2b12e10434c - - - - -] SCRIPT_NAME: `` log_request_info /usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/req_logging.py:28\x1b[00m
2020-09-15 11:24:14.356789 2020-09-15 11:24:14.356 8 DEBUG keystone.server.flask.request_processing.req_logging [req-6cf43a75-4c37-4942-ab87-a2b12e10434c - - - - -] PATH_INFO: `/v3/ec2tokens` log_request_info /usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/req_logging.py:29\x1b[00m
2020-09-15 11:24:14.406988 2020-09-15 11:24:14.406 8 WARNING keystone.common.fernet_utils [req-6cf43a75-4c37-4942-ab87-a2b12e10434c - - - - -] key_repository is world readable: /etc/keystone/credential-keys/\x1b[00m
2020-09-15 11:24:14.425123 mod_wsgi (pid=8): Exception occurred processing WSGI script '/var/www/cgi-bin/keystone/keystone-wsgi-public'.
2020-09-15 11:24:14.425209 Traceback (most recent call last):
2020-09-15 11:24:14.425250 File "/usr/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
2020-09-15 11:24:14.426541 return self.wsgi_app(environ, start_response)
2020-09-15 11:24:14.426598 File "/usr/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__
2020-09-15 11:24:14.426940 return self.app(environ, start_response)
2020-09-15 11:24:14.426998 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__
2020-09-15 11:24:14.427415 resp = self.call_func(req, *args, **kw)
2020-09-15 11:24:14.427486 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func
2020-09-15 11:24:14.427571 return self.func(req, *args, **kwargs)
2020-09-15 11:24:14.427607 File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__
2020-09-15 11:24:14.427874 response = req.get_response(self.application)
2020-09-15 11:24:14.427960 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send
2020-09-15 11:24:14.429143 application, catch_exc_info=False)
2020-09-15 11:24:14.429203 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application
2020-09-15 11:24:14.429295 app_iter = application(self.environ, start_response)
2020-09-15 11:24:14.429348 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 143, in __call__
2020-09-15 11:24:14.429430 return resp(environ, start_response)
2020-09-15 11:24:14.429467 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__
2020-09-15 11:24:14.429511 resp = self.call_func(req, *args, **kw)
2020-09-15 11:24:14.429537 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func
2020-09-15 11:24:14.429624 return self.func(req, *args, **kwargs)
2020-09-15 11:24:14.429668 File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__
2020-09-15 11:24:14.429742 response = req.get_response(self.application)
2020-09-15 11:24:14.429786 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send
2020-09-15 11:24:14.429861 application, catch_exc_info=False)
2020-09-15 11:24:14.429928 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application
2020-09-15 11:24:14.430006 app_iter = application(self.environ, start_response)
2020-09-15 11:24:14.430040 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__
2020-09-15 11:24:14.434266 resp = self.call_func(req, *args, **kw)
2020-09-15 11:24:14.434351 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func
2020-09-15 11:24:14.434460 return self.func(req, *args, **kwargs)
2020-09-15 11:24:14.434513 File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 112, in __call__
2020-09-15 11:24:14.434842 return request.get_response(self.application)
2020-09-15 11:24:14.434898 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send
2020-09-15 11:24:14.435009 application, catch_exc_info=False)
2020-09-15 11:24:14.435061 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application
2020-09-15 11:24:14.435144 app_iter = application(self.environ, start_response)
2020-09-15 11:24:14.435183 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__
2020-09-15 11:24:14.435261 resp = self.call_func(req, *args, **kw)
2020-09-15 11:24:14.435298 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func
2020-09-15 11:24:14.435378 return self.func(req, *args, **kwargs)
2020-09-15 11:24:14.435429 File "/usr/lib/python2.7/site-packages/oslo_middleware/request_id.py", line 58, in __call__
2020-09-15 11:24:14.435697 response = req.get_response(self.application)
2020-09-15 11:24:14.435748 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send
2020-09-15 11:24:14.435822 application, catch_exc_info=False)
2020-09-15 11:24:14.435877 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application
2020-09-15 11:24:14.435956 app_iter = application(self.environ, start_response)
2020-09-15 11:24:14.436008 File "/usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__
2020-09-15 11:24:14.436228 return self.app(environ, start_response)
2020-09-15 11:24:14.436279 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__
2020-09-15 11:24:14.436355 resp = self.call_func(req, *args, **kw)
2020-09-15 11:24:14.436400 File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func
2020-09-15 11:24:14.436477 return self.func(req, *args, **kwargs)
2020-09-15 11:24:14.436513 File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 332, in __call__
2020-09-15 11:24:14.437564 response = req.get_response(self._app)
2020-09-15 11:24:14.437622 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send
2020-09-15 11:24:14.437706 application, catch_exc_info=False)
2020-09-15 11:24:14.437755 File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application
2020-09-15 11:24:14.437832 app_iter = application(self.environ, start_response)
2020-09-15 11:24:14.437882 File "/usr/lib/python2.7/site-packages/werkzeug/wsgi.py", line 826, in __call__
2020-09-15 11:24:14.439175 return app(environ, start_response)
2020-09-15 11:24:14.439233 File "/usr/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
2020-09-15 11:24:14.439320 response = self.handle_exception(e)
2020-09-15 11:24:14.439367 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440091 return original_handler(e)
2020-09-15 11:24:14.440143 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440228 return original_handler(e)
2020-09-15 11:24:14.440275 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440349 return original_handler(e)
2020-09-15 11:24:14.440396 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440467 return original_handler(e)
2020-09-15 11:24:14.440513 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440587 return original_handler(e)
2020-09-15 11:24:14.440634 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440707 return original_handler(e)
2020-09-15 11:24:14.440754 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440818 return original_handler(e)
2020-09-15 11:24:14.440863 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.440933 return original_handler(e)
2020-09-15 11:24:14.440980 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441051 return original_handler(e)
2020-09-15 11:24:14.441097 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441169 return original_handler(e)
2020-09-15 11:24:14.441214 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441286 return original_handler(e)
2020-09-15 11:24:14.441322 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441391 return original_handler(e)
2020-09-15 11:24:14.441438 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441514 return original_handler(e)
2020-09-15 11:24:14.441549 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441618 return original_handler(e)
2020-09-15 11:24:14.441663 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441733 return original_handler(e)
2020-09-15 11:24:14.441772 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441859 return original_handler(e)
2020-09-15 11:24:14.441903 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.441999 return original_handler(e)
2020-09-15 11:24:14.442039 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442111 return original_handler(e)
2020-09-15 11:24:14.442237 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442335 return original_handler(e)
2020-09-15 11:24:14.442390 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442464 return original_handler(e)
2020-09-15 11:24:14.442508 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442581 return original_handler(e)
2020-09-15 11:24:14.442622 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442692 return original_handler(e)
2020-09-15 11:24:14.442729 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442798 return original_handler(e)
2020-09-15 11:24:14.442844 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.442919 return original_handler(e)
2020-09-15 11:24:14.442956 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443028 return original_handler(e)
2020-09-15 11:24:14.443074 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443147 return original_handler(e)
2020-09-15 11:24:14.443194 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443265 return original_handler(e)
2020-09-15 11:24:14.443313 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443384 return original_handler(e)
2020-09-15 11:24:14.443431 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443505 return original_handler(e)
2020-09-15 11:24:14.443542 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.443612 return original_handler(e)
2020-09-15 11:24:14.443659 File "/usr/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
2020-09-15 11:24:14.443734 reraise(exc_type, exc_value, tb)
2020-09-15 11:24:14.443783 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 266, in error_router
2020-09-15 11:24:14.443854 return self.handle_error(e)
2020-09-15 11:24:14.443889 File "/usr/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
2020-09-15 11:24:14.443960 response = self.full_dispatch_request()
2020-09-15 11:24:14.443996 File "/usr/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
2020-09-15 11:24:14.444064 rv = self.handle_user_exception(e)
2020-09-15 11:24:14.444100 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444171 return original_handler(e)
2020-09-15 11:24:14.444218 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444286 return original_handler(e)
2020-09-15 11:24:14.444332 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444429 return original_handler(e)
2020-09-15 11:24:14.444475 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444549 return original_handler(e)
2020-09-15 11:24:14.444593 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444664 return original_handler(e)
2020-09-15 11:24:14.444699 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444766 return original_handler(e)
2020-09-15 11:24:14.444808 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.444876 return original_handler(e)
2020-09-15 11:24:14.444913 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445027 return original_handler(e)
2020-09-15 11:24:14.445070 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445140 return original_handler(e)
2020-09-15 11:24:14.445182 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445253 return original_handler(e)
2020-09-15 11:24:14.445296 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445361 return original_handler(e)
2020-09-15 11:24:14.445401 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445469 return original_handler(e)
2020-09-15 11:24:14.445511 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445580 return original_handler(e)
2020-09-15 11:24:14.445622 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445692 return original_handler(e)
2020-09-15 11:24:14.445725 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445805 return original_handler(e)
2020-09-15 11:24:14.445850 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.445923 return original_handler(e)
2020-09-15 11:24:14.445957 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.446025 return original_handler(e)
2020-09-15 11:24:14.446066 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.450825 return original_handler(e)
2020-09-15 11:24:14.450900 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451019 return original_handler(e)
2020-09-15 11:24:14.451071 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451155 return original_handler(e)
2020-09-15 11:24:14.451201 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451314 return original_handler(e)
2020-09-15 11:24:14.451361 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451448 return original_handler(e)
2020-09-15 11:24:14.451488 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451570 return original_handler(e)
2020-09-15 11:24:14.451607 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.451682 return original_handler(e)
2020-09-15 11:24:14.451976 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452073 return original_handler(e)
2020-09-15 11:24:14.452110 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452180 return original_handler(e)
2020-09-15 11:24:14.452217 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452289 return original_handler(e)
2020-09-15 11:24:14.452332 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452430 return original_handler(e)
2020-09-15 11:24:14.452486 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452556 return original_handler(e)
2020-09-15 11:24:14.452607 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router
2020-09-15 11:24:14.452679 return original_handler(e)
2020-09-15 11:24:14.452714 File "/usr/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
2020-09-15 11:24:14.452793 reraise(exc_type, exc_value, tb)
2020-09-15 11:24:14.452839 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 266, in error_router
2020-09-15 11:24:14.452910 return self.handle_error(e)
2020-09-15 11:24:14.452947 File "/usr/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
2020-09-15 11:24:14.453049 rv = self.dispatch_request()
2020-09-15 11:24:14.453099 File "/usr/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
2020-09-15 11:24:14.453175 return self.view_functions[rule.endpoint](**req.view_args)
2020-09-15 11:24:14.453216 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 458, in wrapper
2020-09-15 11:24:14.453290 resp = resource(*args, **kwargs)
2020-09-15 11:24:14.453339 File "/usr/lib/python2.7/site-packages/flask/views.py", line 88, in view
2020-09-15 11:24:14.453682 return self.dispatch_request(*args, **kwargs)
2020-09-15 11:24:14.453736 File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
2020-09-15 11:24:14.453816 resp = meth(*args, **kwargs)
2020-09-15 11:24:14.453851 File "/usr/lib/python2.7/site-packages/keystone/server/flask/common.py", line 1058, in wrapper
2020-09-15 11:24:14.454874 return f(*args, **kwargs)
2020-09-15 11:24:14.454932 File "/usr/lib/python2.7/site-packages/keystone/api/ec2tokens.py", line 67, in post
2020-09-15 11:24:14.455263 token = self.handle_authenticate()
2020-09-15 11:24:14.455318 File "/usr/lib/python2.7/site-packages/keystone/api/_shared/EC2_S3_Resource.py", line 93, in handle_authenticate
2020-09-15 11:24:14.455625 self._check_signature(cred_data, credentials)
2020-09-15 11:24:14.455681 File "/usr/lib/python2.7/site-packages/keystone/api/ec2tokens.py", line 45, in _check_signature
2020-09-15 11:24:14.455768 hostname, _port = credentials.split(':')
2020-09-15 11:24:14.455846 AttributeError: 'dict' object has no attribute 'split'
172.18.2.128 - - [15/Sep/2020:11:24:14 +0000] "POST /v3/ec2tokens HTTP/1.1" 500 531 "-" "python-requests/2.21.0"
2020-09-15 11:24:15.227147 2020-09-15 11:24:15.226 11 DEBUG keystone.server.flask.request_processing.req_logging [req-6ee8cf32-cc2a-4cb3-8bd8-c7ed848b90f8 - - - - -] REQUEST_METHOD: `GET` log_request_info /usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/req_logging.py:27\x1b[00m

I think this is the api code issue, at this line https://github.com/openstack/keystone/blame/master/keystone/api/ec2tokens.py#L45

it should be
hostname, _port = credentials['host'].split(':')

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.