build69: SM provision status doesn't complain about errors seen during provision

Bug #1715858 reported by Sudheendra Rao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Won't Fix
Low
Sudheendra Rao
R4.0
Incomplete
Low
Sudheendra Rao
R4.1
Incomplete
Low
Sudheendra Rao
Trunk
Won't Fix
Low
Sudheendra Rao

Bug Description

SM provision status doesn't complain about error seen during openstack provisioning.
During R4.0.1 build69 mitaka provision with SMLite, there were package dependency errors seen while openstack provision (openstack HA setup) but SM continued provisioning whole cluster and status was shown provision completed.
Haven't seen this before, not sure it can be reproduced also, hence filing a low priority bug for tracking.

Syslog output on one of the openstack node:

Sep 7 12:41:34 nodem7 puppet-agent[24696]: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install python-mysqldb' returned 100: Reading package lists...
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Building dependency tree...
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Reading state information...
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Suggested packages:
Sep 7 12:41:34 nodem7 puppet-agent[24696]: python-mysqldb-dbg
Sep 7 12:41:34 nodem7 puppet-agent[24696]: The following NEW packages will be installed:
Sep 7 12:41:34 nodem7 puppet-agent[24696]: python-mysqldb
Sep 7 12:41:34 nodem7 puppet-agent[24696]: 0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Need to get 55.4 kB of archives.
Sep 7 12:41:34 nodem7 puppet-agent[24696]: After this operation, 196 kB of additional disk space will be used.
Sep 7 12:41:34 nodem7 puppet-agent[24696]: WARNING: The following packages cannot be authenticated!
Sep 7 12:41:34 nodem7 puppet-agent[24696]: python-mysqldb
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Authentication warning overridden.
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Err http://puppet/contrail/repo/ubuntu14mitaka69/ contrail/main python-mysqldb amd64 1.2.3-2ubuntu1
Sep 7 12:41:34 nodem7 puppet-agent[24696]: Could not connect to puppet:80 (10.204.216.103). - connect (111: Connection refused)
Sep 7 12:41:34 nodem7 puppet-agent[24696]: E: Failed to fetch http://puppet/contrail/repo/ubuntu14mitaka69/pool/main/p/python-mysqldb/python-mysqldb_1.2.3-2ubuntu1_amd64.deb Could not connect to puppet:80 (10.204.216.103). - connect (111: Connection refused)
Sep 7 12:41:34 nodem7 puppet-agent[24696]:
Sep 7 12:41:34 nodem7 puppet-agent[24696]: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Mysql::Bindings::Python/Package[python-mysqldb]/ensure) change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install python-mysqldb' returned 100: Reading package lists...

Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/connection]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/connection]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/idle_timeout]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/idle_timeout]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/min_pool_size]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/min_pool_size]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_retries]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_retries]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/retry_interval]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/retry_interval]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_pool_size]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_pool_size]) Skipping because of failed dependencies
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_overflow]) Dependency Package[python-mysqldb] has failures: true
Sep 7 12:41:34 nodem7 puppet-agent[24696]: (/Stage[openstack]/Keystone::Db/Keystone_config[database/max_overflow]) Skipping because of failed dependencies

root@nodem14:~# /opt/contrail/contrail_server_manager/provision_status.sh

+---------+---------------------+----------------+-------------------+
| id | status | ip_address | mac_address |
+---------+---------------------+----------------+-------------------+
| nodem14 | provision_completed | 10.204.216.103 | 0C:C4:7A:DC:42:C8 |
| nodem6 | provision_completed | 10.204.216.95 | 0C:C4:7A:DC:44:1E |
| nodem7 | provision_completed | 10.204.216.96 | 0C:C4:7A:DC:44:3A |
| nodem8 | provision_completed | 10.204.216.97 | 0C:C4:7A:DC:42:A2 |
| nodem9 | provision_completed | 10.204.216.98 | 0C:C4:7A:DC:44:22 |
| nodec35 | provision_completed | 10.204.216.160 | 00:25:90:C4:7A:70 |
| nodem10 | provision_completed | 10.204.216.99 | 0C:C4:7A:DC:44:3C |
+---------+---------------------+----------------+-------------------+

+---------+------------------+------------------------+-------------------------+---------------------------------------------------------------------------------------------------------------+
| id | cluster_id | role_being_provisioned | roles_pending_provision | provisioned_roles |
+---------+------------------+------------------------+-------------------------+---------------------------------------------------------------------------------------------------------------+
| nodem14 | new_regr_cluster | None | [] | [u'haproxy', u'pre_exec_vnc_galera', u'post_provision', u'post_exec_vnc_galera', u'keepalived', u'openstack'] |
| nodem6 | new_regr_cluster | None | [] | [u'haproxy', u'pre_exec_vnc_galera', u'post_provision', u'post_exec_vnc_galera', u'keepalived', u'openstack'] |
| nodem7 | new_regr_cluster | None | [] | [u'haproxy', u'pre_exec_vnc_galera', u'post_provision', u'post_exec_vnc_galera', u'keepalived', u'openstack'] |
+---------+------------------+------------------------+-------------------------+---------------------------------------------------------------------------------------------------------------+
root@nodem14:~#

Due to this all sanity TCs (during nova boot) failed complaining:

File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 179, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 94, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
 (HTTP 500) (Request-ID: req-cdef2c03-23a2-4fab-beb7-663ae44db456)

Changed in juniperopenstack:
milestone: none → r4.1.0.0-fcs
Revision history for this message
Abhay Joshi (abhayj) wrote :

>>>> Haven't seen this before, not sure it can be reproduced also, hence filing a low priority bug for tracking.

Moving the bug milestone to 4.0.2.

Revision history for this message
Abhay Joshi (abhayj) wrote :

Can you please replicate the problem and provide access to setup?

Changed in juniperopenstack:
assignee: Abhay Joshi (abhayj) → Sudheendra Rao (sudheendra-k)
status: New → Incomplete
Jeba Paulaiyan (jebap)
Changed in juniperopenstack:
milestone: r5.0.0 → none
status: Incomplete → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.