umm.sh doesn't work inside pacemaker checks and produse a lot of ussles information in log file

Fix proposed to branch: master
Review: https://review.openstack.org/294490

Reviewed: https://review.openstack.org/294490
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=9cecd81351b2fc4783ebf64d44d3c53a30313440
Submitter: Jenkins
Branch: master

commit 9cecd81351b2fc4783ebf64d44d3c53a30313440
Author: Peter Zhurba <email address hidden>
Date: Fri Mar 18 12:11:46 2016 +0200

    Fix for backward compatibility in /etc/profile.d/umm.sh

    Pacemaker use /bin/sh even #!/bin/bash is set.

    Change-Id: Ifea408f5825b4b0b2564c7f8e8b9216e9fa2b25d
    Closes-Bug: #1558963

[root@nailgun remote]# grep -ir "zrunlevel N 2" .
===
cat /etc/fuel_build_number:
 490
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6349.noarch
 fuel-misc-9.0.0-1.mos8460.noarch
 python-packetary-9.0.0-1.mos140.noarch
 fuel-bootstrap-cli-9.0.0-1.mos285.noarch
 fuel-migrate-9.0.0-1.mos8460.noarch
 rubygem-astute-9.0.0-1.mos750.noarch
 fuel-mirror-9.0.0-1.mos140.noarch
 shotgun-9.0.0-1.mos90.noarch
 fuel-openstack-metadata-9.0.0-1.mos8743.noarch
 fuel-notify-9.0.0-1.mos8460.noarch
 nailgun-mcagents-9.0.0-1.mos750.noarch
 python-fuelclient-9.0.0-1.mos325.noarch
 fuel-9.0.0-1.mos6349.noarch
 fuel-utils-9.0.0-1.mos8460.noarch
 fuel-setup-9.0.0-1.mos6349.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8743.noarch
 fuel-library9.0-9.0.0-1.mos8460.noarch
 network-checker-9.0.0-1.mos74.x86_64
 fuel-agent-9.0.0-1.mos285.noarch
 fuel-ui-9.0.0-1.mos2717.noarch
 fuel-ostf-9.0.0-1.mos936.noarch
 fuelmenu-9.0.0-1.mos274.noarch
 fuel-nailgun-9.0.0-1.mos8743.noarch

MOS 8.0 + MU3:

--- daemon.log on a controller node ---
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ /usr/bin/umm: line 40: runlevel: command not found ]
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ -su: 1: [: zrunlevel: unexpected operator ]
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ /usr/bin/umm: line 40: runlevel: command not found ]
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ -su: 1: [: zrunlevel: unexpected operator ]
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ /usr/bin/umm: line 40: runlevel: command not found ]
<29>Feb 17 10:55:55 lxf824s002 pacemaker_remoted[28851]: notice: operation_finished: p_rabbitmq-server_monitor_30000:4052:stderr [ -su: 1: [: zrunlevel: unexpected operator ]

a) for the first problem (/usr/bin/umm: line 40: runlevel: command not found):
- we have (in the line 40): "echo runlevel $(runlevel)"
- it seems, we could update the line with: "echo runlevel $(/sbin/runlevel)"

b) "zrunlevel" error seems to be exactly as in the bug "1558963"

Please provide us with the appropriate patch.

Issue "A" in comment https://bugs.launchpad.net/fuel/+bug/1558963/comments/4 is about https://bugs.launchpad.net/fuel/+bug/1528686

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/451356

Related fix proposed to branch: stable/8.0
Review: https://review.openstack.org/451417

Reviewed: https://review.openstack.org/451417
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=c206327d51a799eca7d41f46b1cb235c34e18b30
Submitter: Jenkins
Branch: stable/8.0

commit c206327d51a799eca7d41f46b1cb235c34e18b30
Author: Artem Panchenko <email address hidden>
Date: Tue Dec 22 22:52:15 2015 +0200

    Use absolute path to runlevel binary in umm

    There is no PATH profile load in that script. So when it invoked by
    pacemaker we have to use abosolute path.

    Change-Id: I7eca95667c2711f9611c87ff58c02487dbdc019c
    Picked-From: Idcde2e87eb14089e4a152f5936d0ea197faa0793
    Closes-Bug: #1528686
    Related-Bug: #1558963

Reviewed: https://review.openstack.org/451356
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=38875ecac91fbaf8d7b560cff0c5a936d5b99bb8
Submitter: Jenkins
Branch: stable/8.0

commit 38875ecac91fbaf8d7b560cff0c5a936d5b99bb8
Author: Anton Chevychalov <email address hidden>
Date: Wed Mar 29 14:49:20 2017 +0300

    Fix for backward compatibility in /etc/profile.d/umm.sh

    Pacemaker uses /bin/sh. So we have to be compatible with
    original "test" util

    Change-Id: Ifea408f5825b4b0b2564c7f8e8b9216e9fa2b25d
    Closes-Bug: #1558963

Verified on 8.0 + MU4 updates.

Actual results:
There is no messages with "zrunlevel N 2" in pacemaker_remoted.log:
[root@nailgun ~]# cd /var/log/docker-logs/remote/
[root@nailgun remote]# grep -c "zrunlevel N 2" node-1.test.domain.local/pacemaker_remoted.log
0
[root@nailgun remote]# grep -c "zrunlevel N 2" node-2.test.domain.local/pacemaker_remoted.log
0
[root@nailgun remote]# grep -c "zrunlevel N 2" node-3.test.domain.local/pacemaker_remoted.log
0