Deploy fails because aodh cannot start

Bug #1634044 reported by Nikita Karpin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Nikita Karpin

Bug Description

Detailed bug description:
 When deploying environment with Ceilometer and Aodh using Fuel 10 deploy is failed with the next errors:

AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[aodh/1], Task[aodh/3], Task[aodh/5] Stopping the deployment process!'

when checking systemctl status we can see:

● aodh-api.service - OpenStack Aodh API
  Loaded: loaded (/lib/systemd/system/aodh-api.service; enabled; vendor preset: enabled)
  Active: inactive (dead) (Result: exit-code) since Wed 2016-10-12 08:37:58 UTC; 56s ago
 Process: 25864 ExecStart=/etc/init.d/aodh-api systemd-start (code=exited, status=2)
 Process: 25861 ExecStartPre=/bin/chown aodh:adm /var/log/aodh (code=exited, status=0/SUCCESS)
 Process: 25855 ExecStartPre=/bin/chown aodh:aodh /var/lock/aodh /var/lib/aodh (code=exited, status=0/SUCCESS)
 Process: 25852 ExecStartPre=/bin/mkdir -p /var/lock/aodh /var/log/aodh /var/lib/aodh (code=exited, status=0/SUCCESS)
Main PID: 25864 (code=exited, status=2)

Oct 12 08:37:58 node-4.domain.tld systemd[1]: aodh-api.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 08:37:58 node-4.domain.tld systemd[1]: aodh-api.service: Unit entered failed state.
Oct 12 08:37:58 node-4.domain.tld systemd[1]: aodh-api.service: Failed with result 'exit-code'.
Oct 12 08:37:58 node-4.domain.tld systemd[1]: aodh-api.service: Service hold-off time over, scheduling restart.
Oct 12 08:37:58 node-4.domain.tld systemd[1]: Stopped OpenStack Aodh API.
Oct 12 08:37:58 node-4.domain.tld systemd[1]: aodh-api.service: Start request repeated too quickly.
Oct 12 08:37:58 node-4.domain.tld systemd[1]: Failed to start OpenStack Aodh API.

Steps to reproduce:
 1) Configure in Fuel 10 (fuel-10.0-community-776-2016-10-11_11-14-00.iso) environment with 3 controllers and 1 compute node, enabled Ceilometer and Aodh
 2) apply fix for https://bugs.launchpad.net/fuel/+bug/1632385 - https://review.openstack.org/#/c/385357/
 3) Deploy the environment

Expected results:
 Deployment pass
Actual result:
 Deployment fails with:

AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[aodh/1], Task[aodh/3], Task[aodh/5] Stopping the deployment process!

Reproducibility:
 Always

Revision history for this message
Nikita Karpin (mkarpin) wrote :

snapshot of the environment

Changed in fuel:
status: New → Confirmed
assignee: nobody → Nikita Karpin (mkarpin)
Revision history for this message
Nikita Karpin (mkarpin) wrote :

looks like we need to start aodh under WSGI, I cannot find other way here http://docs.openstack.org/developer/aodh/index.html

Changed in fuel:
importance: Undecided → High
milestone: none → next
milestone: next → 10.0
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Nikita Karpin (mkarpin) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/389731

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/387247
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=6bd961cc07f67d0925643a7952bed04cf9a10b85
Submitter: Jenkins
Branch: master

commit 6bd961cc07f67d0925643a7952bed04cf9a10b85
Author: Mykyta Karpin <email address hidden>
Date: Wed Oct 12 15:01:06 2016 +0300

    Move aodh-api to wsgi under apache

    Current aodh-api implementation can be run under wsgi or uwsgi,
    by analogy with ceilometer setup in fuel-library, this commit moves
    aodh-api to wsgi under apache

    Change-Id: I96cd0ed44adef8c691bb5c1d98f788fe87858cad
    Closes-Bug: #1634044

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/newton)

Reviewed: https://review.openstack.org/389731
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=35ee57be70848856da525172b0d6e5c930a8028c
Submitter: Jenkins
Branch: stable/newton

commit 35ee57be70848856da525172b0d6e5c930a8028c
Author: Mykyta Karpin <email address hidden>
Date: Wed Oct 12 15:01:06 2016 +0300

    Move aodh-api to wsgi under apache

    Current aodh-api implementation can be run under wsgi or uwsgi,
    by analogy with ceilometer setup in fuel-library, this commit moves
    aodh-api to wsgi under apache

    Change-Id: I96cd0ed44adef8c691bb5c1d98f788fe87858cad
    Closes-Bug: #1634044
    (cherry picked from commit 6bd961cc07f67d0925643a7952bed04cf9a10b85)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-library 11.0.0.0rc1

This issue was fixed in the openstack/fuel-library 11.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.