Juniper Openstack

Upgrade to Contrail 3.2.6 through fab adds params related to Keystone v3 & causes neutron/nova CLI failures

Bug #1722401 reported by Savithru Lokanath
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.2
Fix Committed
Critical
Ignatious Johnson Christopher
Juniper Openstack r3.2.6.0
R4.0
Fix Committed
Critical
Ignatious Johnson Christopher
Juniper Openstack r4.0.2.0
R4.1
Fix Committed
Critical
Ignatious Johnson Christopher
Juniper Openstack r4.1.0.0-fcs "r4.1"
Trunk
Fix Committed
Critical
Ignatious Johnson Christopher
Juniper Openstack r5.0.0

Bug Description

Upgraded Contrail from 3.2.5.0-51 to 3.2.6.0-56 via fab upgrade_contrail task. Seeing this error messages when I enter neutron/nova commands.

root@a4s2:~# source /etc/contrail/openstackrc
root@a4s2:~# nova net-list
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'neutronclient.common.exceptions.InternalServerError'> (HTTP 500) (Request-ID: req-a226fcf5-d9f0-40e8-a7b8-946ff834328a)

Also see this traceback:

2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors Traceback (most recent call last):
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/oslo_middleware/catch_errors.py", line 38, in __call__
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors response = req.get_response(self.application)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1317, in send
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors application, catch_exc_info=False)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1281, in call_application
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors app_iter = application(self.environ, start_response)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors resp = self.call_func(req, *args, **self.kwargs)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors return self.func(req, *args, **kwargs)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 464, in __call__
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors response = self.process_request(req)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 732, in process_request
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors resp = super(AuthProtocol, self).process_request(request)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 492, in process_request
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors data, user_auth_ref = self._do_fetch_token(request.user_token)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 531, in _do_fetch_token
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors data = self.fetch_token(token)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 847, in fetch_token
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors data = self._identity_server.verify_token(token)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/_identity.py", line 214, in verify_token
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors auth_ref = self._request_strategy.verify_token(user_token)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/_identity.py", line 166, in _request_strategy
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors strategy_class = self._get_strategy_class()
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/_identity.py", line 188, in _get_strategy_class
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors if self._adapter.get_endpoint(version=klass.AUTH_VERSION):
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 132, in get_endpoint
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors return self.session.get_endpoint(auth or self.auth, **kwargs)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 661, in get_endpoint
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors return auth.get_endpoint(self, **kwargs)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 210, in get_endpoint
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors service_catalog = self.get_access(session).service_catalog
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 136, in get_access
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors self.auth_ref = self.get_auth_ref(session)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 179, in get_auth_ref
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors self._plugin = self._do_create_plugin(session)
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 168, in _do_create_plugin
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors 'Cannot use v2 authentication with domain scope')
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors DiscoveryFailure: Cannot use v2 authentication with domain scope
2017-10-08 23:41:40.310 19209 ERROR oslo_middleware.catch_errors
2017-10-08 23:41:40.314 19209 INFO neutron.wsgi [-] 10.84.18.2 - - [08/Oct/2017 23:41:40] "GET /v2.0/networks.json HTTP/1.1" 500 317 0.020453

In /etc/neutron/neutron.conf, we need to remove the commented params which fab provisions on upgrade

auth_uri = http://10.84.18.2:35357/v2.0/
identity_uri = http://10.84.18.2:5000
admin_tenant_name = service
admin_user = neutron
admin_password = c0ntrail123
auth_host = 10.84.18.2
admin_token = c996f6d88dfd50efc883
auth_protocol = http

#auth_type = password
#username = neutron
#project_name = service
#auth_url = http://10.84.18.2:35357/v2.0/a
#user_domain_name = Default
#password = c0ntrail123
#project_domain_name = Default

Tags: blocker csg
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/36369
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.0

Review in progress for https://review.opencontrail.org/36370
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/36370
Committed: http://github.com/Juniper/contrail-provisioning/commit/b19d7d88c2f850cb9b5e21f9401cc7accbdbcd2e
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit b19d7d88c2f850cb9b5e21f9401cc7accbdbcd2e
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Oct 9 14:05:20 2017 -0700

Provision keystone v3 params in contrail keystone

auth conf and neutron conf file during upgrade only
if the base setup is provisionied with keystone v3.

Change-Id: I09cc3ae8b177177bda8c0e0da9710c924cd2d4cb
Closes-Bug: 1722401

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/36369
Committed: http://github.com/Juniper/contrail-provisioning/commit/13c4dd8aa165cb3fc81d82353d0b0b781f0932f8
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 13c4dd8aa165cb3fc81d82353d0b0b781f0932f8
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Oct 9 14:05:20 2017 -0700

Provision keystone v3 params in contrail keystone

auth conf and neutron conf file during upgrade only
if the base setup is provisionied with keystone v3.

Change-Id: I09cc3ae8b177177bda8c0e0da9710c924cd2d4cb
Closes-Bug: 1722401

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R4.1

Review in progress for https://review.opencontrail.org/36748
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/36749
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/36748
Committed: http://github.com/Juniper/contrail-provisioning/commit/e56483072de6025b90567897700b72c1a6406ee2
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit e56483072de6025b90567897700b72c1a6406ee2
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Oct 9 14:05:20 2017 -0700

Provision keystone v3 params in contrail keystone

auth conf and neutron conf file during upgrade only
if the base setup is provisionied with keystone v3.

Change-Id: I09cc3ae8b177177bda8c0e0da9710c924cd2d4cb
Closes-Bug: 1722401
(cherry picked from commit b19d7d88c2f850cb9b5e21f9401cc7accbdbcd2e)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/36749
Committed: http://github.com/Juniper/contrail-provisioning/commit/7f3034201914335ea4e25643a418ac1a76e9935e
Submitter: Zuul (<email address hidden>)
Branch: master

commit 7f3034201914335ea4e25643a418ac1a76e9935e
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Oct 9 14:05:20 2017 -0700

Provision keystone v3 params in contrail keystone

auth conf and neutron conf file during upgrade only
if the base setup is provisionied with keystone v3.

Change-Id: I09cc3ae8b177177bda8c0e0da9710c924cd2d4cb
Closes-Bug: 1722401
(cherry picked from commit b19d7d88c2f850cb9b5e21f9401cc7accbdbcd2e)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.