Introspection failing in train branch with message: mistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}

Bug #1909582 reported by Sandeep Yadav
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

Description:-

Introspection failing in train branch with message: mistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}

Issue started on 26th dec:-

https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train

Log:-

https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/d84a06d/job-output.txt

~~~
2020-12-29 05:51:55.413053 | primary | TASK [overcloud-prep-images : Introspect overcloud images] *********************
2020-12-29 05:51:55.413062 | primary | Tuesday 29 December 2020 05:51:55 +0000 (0:00:01.767) 0:43:35.621 ******
2020-12-29 05:52:01.633539 | primary | fatal: [undercloud]: FAILED! => {
2020-12-29 05:52:01.633745 | primary | "changed": true,
2020-12-29 05:52:01.633767 | primary | "cmd": "set -o pipefail && /home/zuul/overcloud-introspect.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%d %H:%M:%S |\"), $0; fflush(); }' > /home/zuul/overcloud_introspect.log\n",
2020-12-29 05:52:01.633784 | primary | "delta": "0:00:05.848500",
2020-12-29 05:52:01.633794 | primary | "end": "2020-12-29 05:52:01.605028",
2020-12-29 05:52:01.633801 | primary | "rc": 1,
2020-12-29 05:52:01.633807 | primary | "start": "2020-12-29 05:51:55.756528"
~~~

https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/d84a06d/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz

~~~
2020-12-29 05:52:01 | Exception introspecting nodes: {'result': 'Failed to run task [error=Failed to find action [action_name=baremetal_introspection.get_status], wf=tripleo.baremetal.v1.introspect, task=get_introspection_status]:\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task\n task.run()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run\n self._run_new()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new\n self._schedule_actions()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 795, in _schedule_actions\n action = self._build_action()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action\n self.wf_spec.get_name()\n File "/usr/lib/python3.6/site-packages/mistral/engine/actions.py", line 680, in resolve_action_definition\n "Failed to find action [action_name=%s]" % action_spec_name\nmistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}
2020-12-29 05:52:01 | Waiting for introspection to finish...
2020-12-29 05:52:01 | {'result': 'Failed to run task [error=Failed to find action [action_name=baremetal_introspection.get_status], wf=tripleo.baremetal.v1.introspect, task=get_introspection_status]:\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task\n task.run()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run\n self._run_new()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new\n self._schedule_actions()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 795, in _schedule_actions\n action = self._build_action()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action\n self.wf_spec.get_name()\n File "/usr/lib/python3.6/site-packages/mistral/engine/actions.py", line 680, in resolve_action_definition\n "Failed to find action [action_name=%s]" % action_spec_name\nmistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}
~~~

Another example:-

https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-train/b8bb257/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :
Download full text (3.4 KiB)

https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/d84a06d/logs/undercloud/var/log/containers/mistral/engine.log.txt.gz

~~~
2020-12-29 05:51:59.778 7 ERROR mistral.engine.task_handler [req-bd549338-e398-4f91-8b80-30eb79cfbb50 993ebdf86580482f875b0e124da3b308 3ade29af34244a50a17eeece163b1815 - default default] Failed to run task [error=Failed to find action [action_name=baremetal_introspection.introspect], wf=tripleo.baremetal.v1._introspect, task=start_introspection]:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task
    task.run()
  File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run
    self._run_new()
  File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new
    self._schedule_actions()
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 610, in _schedule_actions
    action = self._build_action()
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action
    self.wf_spec.get_name()
  File "/usr/lib/python3.6/site-packages/mistral/engine/actions.py", line 680, in resolve_action_definition
    "Failed to find action [action_name=%s]" % action_spec_name
mistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.introspect]
: mistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.introspect]
2020-12-29 05:51:59.782 7 INFO workflow_trace [req-bd549338-e398-4f91-8b80-30eb79cfbb50 993ebdf86580482f875b0e124da3b308 3ade29af34244a50a17eeece163b1815 - default default] Task 'start_introspection' (9e5bd8fc-5b86-4790-b841-f56c5b03ab9d) [RUNNING -> ERROR, msg=Failed to run task [error=Failed to find action [action_name=baremetal_introspection.introspect], wf=tripleo.baremetal.v1._introspect, task=start_introspection]:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task
    task.run()
  File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run
    self._run_new()
  File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new
    self._schedule_actions()
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 610, in _schedule_actions
    action = self._build_action()
  File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action
    self.wf_spec.get_name()
  File "/usr/lib/python3.6/site-packages/mistral/engine...

Read more...

Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :
Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :
Changed in tripleo:
status: Triaged → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 11.5.0

This issue was fixed in the openstack/puppet-tripleo 11.5.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.