Comment 6 for bug 1253594

Revision history for this message
Aleksandr Didenko (adidenko) wrote :

The problem is intermittent. Sometimes "crmd" daemon dies on 2-nd or 3-rd controller node during deployment, which breaks mysql galera sync and leads to the puppet errors provided in the initial post.

Logs on the failed controller node:
<28>Nov 29 13:38:34 node-4 crmd[11704]: warning: do_exit: Inhibiting respawn by Heartbeat

Logs on the first controller node:
<29>Nov 29 13:38:27 node-2 crmd[4366]: notice: peer_update_callback: do_shutdown of node-4.domain.tld (op 34) is complete
<29>Nov 29 13:38:27 node-2 crmd[4366]: notice: crm_update_peer_state: crm_update_ais_node: Node node-4.domain.tld[100706496] - state is now lost
<29>Nov 29 13:38:28 node-2 crmd[4366]: notice: crm_update_peer_state: crm_update_ais_node: Node node-4.domain.tld[100706496] - state is now member
<29>Nov 29 13:38:35 node-2 crmd[4366]: notice: peer_update_callback: Stonith/shutdown of node-4.domain.tld not matched

This happens during "corosync_setup" puppet agent run stage.