Third controller doesn't see the rest. The first two don't see the third

Bug #1296604 reported by Anastasia Palkina
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
Undecided
Fuel Library (Deprecated)

Bug Description

"build_id": "2014-03-21_01-01-16",
"mirantis": "yes",
"build_number": "36",
"nailgun_sha": "102bfc56d2e7c9d7295066cc30c21053fca44d79",
"ostf_sha": "608f109b6f07b695305468627941b8505abe3c7f",
"fuelmain_sha": "b853f8ad06516d83ad4956e945be923b5aa3ec9d",
"astute_sha": "f52db1262401cfb2beaf41e7d6eeaeb456d5fe95",
"release": "5.0",
"fuellib_sha": "54072cce48fc539e4d99f4862f9a5568b3ef3384"

1. Create new environment (CentOS, HA mode)
2. Choose nova-network, VLAN manager
3. Add 3 controllers and 2 computes
4. Start deployment. It was successful
5. But third controller doesn't see the rest. The first two don't see the third

[root@node-30 ~]# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert node-30.domain.tld internal enabled :-) 2014-03-24 08:44:36
nova-consoleauth node-30.domain.tld internal enabled :-) 2014-03-24 08:44:36
nova-scheduler node-30.domain.tld internal enabled :-) 2014-03-24 08:44:36
nova-conductor node-30.domain.tld internal enabled :-) 2014-03-24 08:44:36
nova-cert node-31.domain.tld internal enabled :-) 2014-03-24 08:44:37
nova-consoleauth node-31.domain.tld internal enabled :-) 2014-03-24 08:44:37
nova-scheduler node-31.domain.tld internal enabled :-) 2014-03-24 08:44:37
nova-conductor node-31.domain.tld internal enabled :-) 2014-03-24 08:44:38
nova-cert node-32.domain.tld internal enabled XXX 2014-03-24 07:52:25
nova-consoleauth node-32.domain.tld internal enabled XXX 2014-03-24 07:52:25
nova-scheduler node-32.domain.tld internal enabled XXX 2014-03-24 07:52:25
nova-conductor node-32.domain.tld internal enabled XXX 2014-03-24 07:52:30
nova-compute node-34.domain.tld nova enabled XXX 2014-03-24 07:52:33
nova-compute node-33.domain.tld nova enabled :-) 2014-03-24 08:44:27
nova-network node-33.domain.tld internal enabled :-) 2014-03-24 08:44:39
nova-network node-34.domain.tld internal enabled :-) 2014-03-24 08:44:35

[root@node-31 ~]# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert node-30.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-consoleauth node-30.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-scheduler node-30.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-conductor node-30.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-cert node-31.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-consoleauth node-31.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-scheduler node-31.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-conductor node-31.domain.tld internal enabled :-) 2014-03-24 08:49:08
nova-cert node-32.domain.tld internal enabled XXX 2014-03-24 07:57:06
nova-consoleauth node-32.domain.tld internal enabled XXX 2014-03-24 07:57:06
nova-scheduler node-32.domain.tld internal enabled XXX 2014-03-24 07:57:05
nova-conductor node-32.domain.tld internal enabled XXX 2014-03-24 07:57:01
nova-compute node-34.domain.tld nova enabled :-) 2014-03-24 08:49:09
nova-compute node-33.domain.tld nova enabled XXX 2014-03-24 07:57:00
nova-network node-33.domain.tld internal enabled :-) 2014-03-24 08:49:09
nova-network node-34.domain.tld internal enabled :-) 2014-03-24 08:49:06

[root@node-32 ~]# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert node-30.domain.tld internal enabled XXX 2014-03-24 08:48:28
nova-consoleauth node-30.domain.tld internal enabled XXX 2014-03-24 08:48:28
nova-scheduler node-30.domain.tld internal enabled XXX 2014-03-24 08:48:28
nova-conductor node-30.domain.tld internal enabled XXX 2014-03-24 08:48:28
nova-cert node-31.domain.tld internal enabled XXX 2014-03-24 08:48:38
nova-consoleauth node-31.domain.tld internal enabled XXX 2014-03-24 08:48:38
nova-scheduler node-31.domain.tld internal enabled XXX 2014-03-24 08:48:38
nova-conductor node-31.domain.tld internal enabled XXX 2014-03-24 08:48:38
nova-cert node-32.domain.tld internal enabled :-) 2014-03-24 07:56:26
nova-consoleauth node-32.domain.tld internal enabled :-) 2014-03-24 07:56:26
nova-scheduler node-32.domain.tld internal enabled :-) 2014-03-24 07:56:26
nova-conductor node-32.domain.tld internal enabled :-) 2014-03-24 07:56:21
nova-compute node-34.domain.tld nova enabled :-) 2014-03-24 07:56:30
nova-compute node-33.domain.tld nova enabled XXX 2014-03-24 08:48:37
nova-network node-33.domain.tld internal enabled XXX 2014-03-24 08:48:39
nova-network node-34.domain.tld internal enabled XXX 2014-03-24 08:48:36

[root@node-31 ~]# pcs status
Cluster name:
Last updated: Mon Mar 24 08:49:26 2014
Last change: Fri Mar 21 16:06:30 2014 via cibadmin on node-32.domain.tld
Stack: classic openais (with plugin)
Current DC: node-30.domain.tld - partition with quorum
Version: 1.1.10-3.el6.2-368c726
3 Nodes configured, 3 expected votes
9 Resources configured

Online: [ node-30.domain.tld node-31.domain.tld node-32.domain.tld ]

Full list of resources:

 vip__management_old (ocf::heartbeat:IPaddr2): Started node-30.domain.tld
 vip__public_old (ocf::heartbeat:IPaddr2): Started node-30.domain.tld
 Clone Set: clone_p_mysql [p_mysql]
     Started: [ node-30.domain.tld node-31.domain.tld node-32.domain.tld ]
 Clone Set: clone_p_haproxy [p_haproxy]
     Started: [ node-30.domain.tld node-31.domain.tld node-32.domain.tld ]
 openstack-heat-engine (ocf::mirantis:openstack-heat-engine): Started node-31.domain.tld

[root@node-31 ~]# mysql -e 'show status'
...
 wsrep_incoming_addresses | 192.168.0.4:3307,192.168.0.3:3307,192.168.0.5:3307

Logs here https://drive.google.com/file/d/0B6SjzarTGFxaUUdBSlJNUzBtSjg/edit?usp=sharing

Revision history for this message
Anastasia Palkina (apalkina) wrote :

The reason was in the wrong date on third controller.

Changed in fuel:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.