[puppet4] database task fails to detect mysqld backend start

Bug #1656249 reported by Nikita Karpin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Nikita Karpin

Bug Description

During deployment of environment using Puppet 4, primary-database task fails with the next error:

(/Stage[main]/Osnailyfacter::Database::Database_backend_wait/Osnailyfacter::Wait_for_backend[mysql]/Haproxy_backend_status[mysql]/ensure) change from down to up failed: Timeout waiting for HAProxy backend: 'mysqld' status to become: 'up' after 600 seconds!

On primary controller mysqld is launched and responsive, moreover galera status is synced:

mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 12472
Server version: 5.6.34-0~u16.04+mos1 (Ubuntu), wsrep_25.18

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW STATUS LIKE 'wsrep_local_state';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| wsrep_local_state | 4 |
+-------------------+-------+
1 row in set (0.01 sec)

But when running galeracheck script, we get:

/usr/bin/galeracheck
HTTP/1.1 503 Service Unavailable
Content-Type: text/plain
Connection: close
Content-Length: 44

Percona XtraDB Cluster Node is not synced.

When checking /etc/wsrepclustercheckrc I found that mysql credentials are empty:

MYSQL_USERNAME=""
MYSQL_PASSWORD=""
MYSQL_HOST="192.168.0.3"
MYSQL_PORT="3307"
AVAILABLE_WHEN_DONOR=${3:-1}
ERR_FILE="${4:-/dev/null}"
AVAILABLE_WHEN_READONLY=${5:-1}
DEFAULTS_EXTRA_FILE=${6:-/etc/my.cnf}

Nikita Karpin (mkarpin)
Changed in fuel:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Nikita Karpin (mkarpin)
milestone: none → 11.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/419870

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/419870
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=9961a4b1028503e0d28342d31437c9e8b3f6bd28
Submitter: Jenkins
Branch: master

commit 9961a4b1028503e0d28342d31437c9e8b3f6bd28
Author: Mykyta Karpin <email address hidden>
Date: Fri Jan 13 12:09:38 2017 +0200

    Pass mysql credentials to galera_status

    In puppet 4 templates require variables to be
    explicitly set in parent class scope.

    This patch adds status_password and status_user parameters
    to cluster::galera_status class so they can be available
    for openstack/galera_clustercheck.erb template.

    Closes-Bug: #1656249

    Change-Id: Ie1a5d371fbe18a41f6b4b0f15e4076fd3dadfaca

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/430949

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/430950

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/newton)

Reviewed: https://review.openstack.org/430949
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=0e9b4cafec35f2d9c29f3880963e9a1d0074856d
Submitter: Jenkins
Branch: stable/newton

commit 0e9b4cafec35f2d9c29f3880963e9a1d0074856d
Author: Mykyta Karpin <email address hidden>
Date: Fri Jan 13 12:09:38 2017 +0200

    Pass mysql credentials to galera_status

    In puppet 4 templates require variables to be
    explicitly set in parent class scope.

    This patch adds status_password and status_user parameters
    to cluster::galera_status class so they can be available
    for openstack/galera_clustercheck.erb template.

    Closes-Bug: #1656249

    Change-Id: Ie1a5d371fbe18a41f6b4b0f15e4076fd3dadfaca
    (cherry picked from commit 9961a4b1028503e0d28342d31437c9e8b3f6bd28)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-library 11.0.0.0rc1

This issue was fixed in the openstack/fuel-library 11.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (stable/mitaka)

Change abandoned by Oleksiy Molchanov (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/430950

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.