[fuel-library] Pacemaker resource 'ceilometer-alarm-evaluator' fails to start

@Andrey, can you please attach diagnostic snapshot?

I have no environments for this time, but the problem is definitely in undefined variable OCF_RESKEY_amqp_server_port.

It is used by this piece of code:
https://github.com/openstack/fuel-library/blob/master/files/fuel-ha-utils/ocf/ceilometer-alarm-evaluator#L145

Workarounds

1) When I tried to define default value for this variable like this:
OCF_RESKEY_amqp_server_port_default="5673"
...
: ${OCF_RESKEY_amqp_server_port=${OCF_RESKEY_amqp_server_port_default}}

@@ -37,6 +37,7 @@
 OCF_RESKEY_config_default="/etc/ceilometer/ceilometer.conf"
 OCF_RESKEY_user_default="ceilometer"
 OCF_RESKEY_pid_default="${HA_RSCTMP}/${__SCRIPT_NAME}/${__SCRIPT_NAME}.pid"
+OCF_RESKEY_amqp_server_port_default="5673"

 : ${HA_LOGTAG="ocf-ceilometer-alarm-evaluator"}
 : ${HA_LOGFACILITY="daemon"}
@@ -44,6 +45,7 @@
 : ${OCF_RESKEY_config=${OCF_RESKEY_config_default}}
 : ${OCF_RESKEY_user=${OCF_RESKEY_user_default}}
 : ${OCF_RESKEY_pid=${OCF_RESKEY_pid_default}}
+: ${OCF_RESKEY_amqp_server_port=${OCF_RESKEY_amqp_server_port_default}}

Then

/usr/sbin/pcs resource clear p_ceilometer-alarm-evaluator node-1.test.domain.local'
/usr/sbin/pcs resource enable p_ceilometer-alarm-evaluator
/usr/sbin/pcs resource debug-start p_ceilometer-alarm-evaluator
(OR /usr/sbin/crm resource restart p_ceilometer-alarm-evaluator)

And resource becomes up and running:
p_ceilometer-alarm-evaluator (ocf::fuel:ceilometer-alarm-evaluator): Started node-1.test.domain.local

2) Another workaround - just to comment out block with validate_port() function which uses this undefined variable
OCF_RESKEY_amqp_server_port

@@ -142,9 +142,9 @@
     check_binary $OCF_RESKEY_binary
     check_binary netstat

- if ! validate_port $OCF_RESKEY_amqp_server_port; then
- return ${OCF_ERR_CONFIGURED}
- fi
+ #if ! validate_port $OCF_RESKEY_amqp_server_port; then
+ # return ${OCF_ERR_CONFIGURED}
+ #fi

     # A config file on shared storage that is not available
     # during probes is OK.

Fix proposed to branch: master
Review: https://review.openstack.org/284471

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.openstack.org/284471
Reason: should be fixed with the switch to aodh, https://review.openstack.org/#/c/279127

Reviewed: https://review.openstack.org/284471
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=0a4d7e6fe745a5a7eefdd0e18f62b05ad1a5750c
Submitter: Jenkins
Branch: master

commit 0a4d7e6fe745a5a7eefdd0e18f62b05ad1a5750c
Author: Alex Schultz <email address hidden>
Date: Wed Feb 24 16:46:57 2016 -0700

    Remove port check from alarm-evaluator

    This change removes the port check from the ocf script for
    ceilometer-alarm-evaulator. This port check for amqp was improperly
    added as part of Ic4bcecdbc05f6306be63dea211df96a104cb2d36.

    Change-Id: I9999401787f31ce6cd9f30021011fd198f1f24ad
    Closes-Bug: #1549062

Verified on MOS 9.0, ISO build 257.
The issue is fixed.

ISO details:
cat /etc/fuel_build_id:
 257
cat /etc/fuel_build_number:
 257
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6342.noarch
 fuel-misc-9.0.0-1.mos8329.noarch
 fuel-mirror-9.0.0-1.mos133.noarch
 shotgun-9.0.0-1.mos88.noarch
 fuel-openstack-metadata-9.0.0-1.mos8671.noarch
 fuel-notify-9.0.0-1.mos8329.noarch
 fuel-ostf-9.0.0-1.mos928.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8671.noarch
 python-fuelclient-9.0.0-1.mos313.noarch
 fuel-9.0.0-1.mos6342.noarch
 fuel-utils-9.0.0-1.mos8329.noarch
 fuel-nailgun-9.0.0-1.mos8671.noarch
 rubygem-astute-9.0.0-1.mos741.noarch
 fuel-library9.0-9.0.0-1.mos8329.noarch
 network-checker-9.0.0-1.mos72.x86_64
 fuel-agent-9.0.0-1.mos273.noarch
 fuel-ui-9.0.0-1.mos2676.noarch
 fuel-setup-9.0.0-1.mos6342.noarch
 nailgun-mcagents-9.0.0-1.mos741.noarch
 python-packetary-9.0.0-1.mos133.noarch
 fuelmenu-9.0.0-1.mos269.noarch
 fuel-bootstrap-cli-9.0.0-1.mos273.noarch
 fuel-migrate-9.0.0-1.mos8329.noarch