So, as we see at the log, fuel-devops couldn't sync time on some node, and this node is not controller. The difference between syncing time on controllers and other nodes is an NTP host. For controller it is some remote host and for other nodes it is the NTP server which runs on virtual namespace managed by Pacemaker at vip__vrouter resource. I tried to sync time on compute: root@node-5:~# ntpdate -d 10.109.1.8 1 Jun 10:31:29 ntpdate[32556]: ntpdate 4.2.6p5@1.2349-o Thu Feb 11 18:30:41 UTC 2016 (1) Looking for host 10.109.1.8 and service ntp host found : 10.109.1.8 transmit(10.109.1.8) transmit(10.109.1.8) transmit(10.109.1.8) transmit(10.109.1.8) transmit(10.109.1.8) 10.109.1.8: Server dropped: no data server 10.109.1.8, port 123 stratum 0, precision 0, leap 00, trust 000 refid [10.109.1.8], delay 0.00000, dispersion 64.00000 transmitted 4, in filter 4 reference time: 00000000.00000000 Mon, Jan 1 1900 0:00:00.000 originate timestamp: 00000000.00000000 Mon, Jan 1 1900 0:00:00.000 transmit timestamp: daf93807.6a5ce4c3 Wed, Jun 1 2016 10:31:35.415 filter delay: 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 filter offset: 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 delay 0.00000, dispersion 64.00000 offset 0.000000 ​ 1 Jun 10:31:37 ntpdate[32556]: no server suitable for synchronization found Then I tried to get know what's wrong with NTP in namespace, on controller: root@node-6:~# ip netns exec vrouter netstat -nap|grep :123 So, there is no alive NTP instance Pacemaker status: root@node-6:~# pcs status Cluster name: WARNING: corosync and pacemaker node names do not match (IPs used in setup?) Last updated: Wed Jun 1 11:46:43 2016 Last change: Tue May 31 12:27:14 2016 by root via cibadmin on node-2.test.domain.local Stack: corosync Current DC: node-6.test.domain.local (version 1.1.14-70404b0) - partition with quorum 3 nodes and 46 resources configured ​ Online: [ node-1.test.domain.local node-2.test.domain.local node-6.test.domain.local ] ​ Full list of resources: ​ Clone Set: clone_p_vrouter [p_vrouter] Started: [ node-1.test.domain.local node-2.test.domain.local node-6.test.domain.local ] vip__management (ocf::fuel:ns_IPaddr2): Started node-1.test.domain.local vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-6.test.domain.local vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-6.test.domain.local vip__public (ocf::fuel:ns_IPaddr2): Stopped Clone Set: clone_p_haproxy [p_haproxy] Started: [ node-1.test.domain.local node-2.test.domain.local node-6.test.domain.local ] Clone Set: clone_p_mysqld [p_mysqld] Stopped: [ node-1.test.domain.local node-2.test.domain.local node-6.test.domain.local ] sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local sysinfo_node-6.test.domain.local (ocf::pacemaker:SysInfo): Started node-6.test.domain.local Master/Slave Set: master_p_conntrackd [p_conntrackd] Masters: [ node-2.test.domain.local ] Slaves: [ node-6.test.domain.local ] Stopped: [ node-1.test.domain.local ] Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Slaves: [ node-1.test.domain.local node-2.test.domain.local node-6.test.domain.local ] Clone Set: clone_p_dns [p_dns] Started: [ node-2.test.domain.local node-6.test.domain.local ] Stopped: [ node-1.test.domain.local ] sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Clone Set: clone_neutron-openvswitch-agent [neutron-openvswitch-agent] Started: [ node-2.test.domain.local node-6.test.domain.local ] Clone Set: clone_neutron-l3-agent [neutron-l3-agent] Started: [ node-6.test.domain.local ] Stopped: [ node-2.test.domain.local ] Clone Set: clone_p_heat-engine [p_heat-engine] Stopped: [ node-6.test.domain.local ] Clone Set: clone_neutron-metadata-agent [neutron-metadata-agent] Clone Set: clone_neutron-dhcp-agent [neutron-dhcp-agent] Clone Set: clone_p_ntp [p_ntp] Clone Set: clone_ping_vip__public [ping_vip__public] ​ Failed Actions: * p_mysqld_start_0 on node-1.test.domain.local 'unknown error' (1): call=168, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 13:02:47 2016', queued=0ms, exec=300002ms * neutron-openvswitch-agent_monitor_20000 on node-6.test.domain.local 'unknown error' (1): call=93, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 12:53:36 2016', queued=0ms, exec=0ms * p_mysqld_monitor_60000 on node-6.test.domain.local 'unknown error' (1): call=41, status=complete, exitreason='none', last-rc-change='Tue May 31 12:54:24 2016', queued=0ms, exec=0ms * neutron-l3-agent_monitor_20000 on node-6.test.domain.local 'unknown error' (1): call=100, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 12:53:30 2016', queued=0ms, exec=0ms * sysinfo_node-6.test.domain.local_monitor_15000 on node-6.test.domain.local 'unknown error' (1): call=47, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 12:53:28 2016', queued=0ms, exec=0ms * neutron-openvswitch-agent_monitor_20000 on node-2.test.domain.local 'unknown error' (1): call=96, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 12:54:35 2016', queued=0ms, exec=0ms * p_mysqld_start_0 on node-2.test.domain.local 'unknown error' (1): call=326, status=Timed Out, exitreason='none', last-rc-change='Wed Jun 1 11:37:52 2016', queued=0ms, exec=300004ms * sysinfo_node-2.test.domain.local_monitor_15000 on node-2.test.domain.local 'unknown error' (1): call=47, status=Timed Out, exitreason='none', last-rc-change='Tue May 31 12:54:18 2016', queued=0ms, exec=0ms ​ ​ PCSD Status: node-1.test.domain.local member (10.109.1.4): Offline node-2.test.domain.local member (10.109.1.3): Offline node-6.test.domain.local member (10.109.1.2): Offline There are many failed actions. We need to investigate what exactly prevent us to normally run NTP demon. This is not QA bug, I've analyze all our code related to the problem and we couldn't fix it on our side.