Description:-
Introspection failing in train branch with message: mistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}
Issue started on 26th dec:-
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train
Log:-
https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/d84a06d/job-output.txt
~~~
2020-12-29 05:51:55.413053 | primary | TASK [overcloud-prep-images : Introspect overcloud images] *********************
2020-12-29 05:51:55.413062 | primary | Tuesday 29 December 2020 05:51:55 +0000 (0:00:01.767) 0:43:35.621 ******
2020-12-29 05:52:01.633539 | primary | fatal: [undercloud]: FAILED! => {
2020-12-29 05:52:01.633745 | primary | "changed": true,
2020-12-29 05:52:01.633767 | primary | "cmd": "set -o pipefail && /home/zuul/overcloud-introspect.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%d %H:%M:%S |\"), $0; fflush(); }' > /home/zuul/overcloud_introspect.log\n",
2020-12-29 05:52:01.633784 | primary | "delta": "0:00:05.848500",
2020-12-29 05:52:01.633794 | primary | "end": "2020-12-29 05:52:01.605028",
2020-12-29 05:52:01.633801 | primary | "rc": 1,
2020-12-29 05:52:01.633807 | primary | "start": "2020-12-29 05:51:55.756528"
~~~
https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/d84a06d/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz
~~~
2020-12-29 05:52:01 | Exception introspecting nodes: {'result': 'Failed to run task [error=Failed to find action [action_name=baremetal_introspection.get_status], wf=tripleo.baremetal.v1.introspect, task=get_introspection_status]:\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task\n task.run()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run\n self._run_new()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new\n self._schedule_actions()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 795, in _schedule_actions\n action = self._build_action()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action\n self.wf_spec.get_name()\n File "/usr/lib/python3.6/site-packages/mistral/engine/actions.py", line 680, in resolve_action_definition\n "Failed to find action [action_name=%s]" % action_spec_name\nmistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}
2020-12-29 05:52:01 | Waiting for introspection to finish...
2020-12-29 05:52:01 | {'result': 'Failed to run task [error=Failed to find action [action_name=baremetal_introspection.get_status], wf=tripleo.baremetal.v1.introspect, task=get_introspection_status]:\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/mistral/engine/task_handler.py", line 63, in run_task\n task.run()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 502, in run\n self._run_new()\n File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper\n result = f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 534, in _run_new\n self._schedule_actions()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 795, in _schedule_actions\n action = self._build_action()\n File "/usr/lib/python3.6/site-packages/mistral/engine/tasks.py", line 693, in _build_action\n self.wf_spec.get_name()\n File "/usr/lib/python3.6/site-packages/mistral/engine/actions.py", line 680, in resolve_action_definition\n "Failed to find action [action_name=%s]" % action_spec_name\nmistral.exceptions.InvalidActionException: Failed to find action [action_name=baremetal_introspection.get_status]\n', 'introspection_attempt': 0}
~~~
Another example:-
https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-train/b8bb257/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz
https:/ /logserver. rdoproject. org/openstack- periodic- integration- stable3/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 8-ovb-3ctlr_ 1comp-featurese t001-train/ d84a06d/ logs/undercloud /var/log/ containers/ mistral/ engine. log.txt. gz
~~~ engine. task_handler [req-bd549338- e398-4f91- 8b80-30eb79cfbb 50 993ebdf86580482 f875b0e124da3b3 08 3ade29af34244a5 0a17eeece163b18 15 - default default] Failed to run task [error=Failed to find action [action_ name=baremetal_ introspection. introspect] , wf=tripleo. baremetal. v1._introspect, task=start_ introspection] : python3. 6/site- packages/ mistral/ engine/ task_handler. py", line 63, in run_task python3. 6/site- packages/ osprofiler/ profiler. py", line 160, in wrapper python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 502, in run python3. 6/site- packages/ osprofiler/ profiler. py", line 160, in wrapper python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 534, in _run_new _schedule_ actions( ) python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 610, in _schedule_actions action( ) python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 693, in _build_action wf_spec. get_name( ) python3. 6/site- packages/ mistral/ engine/ actions. py", line 680, in resolve_ action_ definition exceptions. InvalidActionEx ception: Failed to find action [action_ name=baremetal_ introspection. introspect] exceptions. InvalidActionEx ception: Failed to find action [action_ name=baremetal_ introspection. introspect] e398-4f91- 8b80-30eb79cfbb 50 993ebdf86580482 f875b0e124da3b3 08 3ade29af34244a5 0a17eeece163b18 15 - default default] Task 'start_ introspection' (9e5bd8fc- 5b86-4790- b841-f56c5b03ab 9d) [RUNNING -> ERROR, msg=Failed to run task [error=Failed to find action [action_ name=baremetal_ introspection. introspect] , wf=tripleo. baremetal. v1._introspect, task=start_ introspection] : python3. 6/site- packages/ mistral/ engine/ task_handler. py", line 63, in run_task python3. 6/site- packages/ osprofiler/ profiler. py", line 160, in wrapper python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 502, in run python3. 6/site- packages/ osprofiler/ profiler. py", line 160, in wrapper python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 534, in _run_new _schedule_ actions( ) python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 610, in _schedule_actions action( ) python3. 6/site- packages/ mistral/ engine/ tasks.py" , line 693, in _build_action wf_spec. get_name( ) python3. 6/site- packages/ mistral/ engine. ..
2020-12-29 05:51:59.778 7 ERROR mistral.
Traceback (most recent call last):
File "/usr/lib/
task.run()
File "/usr/lib/
result = f(*args, **kwargs)
File "/usr/lib/
self._run_new()
File "/usr/lib/
result = f(*args, **kwargs)
File "/usr/lib/
self.
File "/usr/lib/
action = self._build_
File "/usr/lib/
self.
File "/usr/lib/
"Failed to find action [action_name=%s]" % action_spec_name
mistral.
: mistral.
2020-12-29 05:51:59.782 7 INFO workflow_trace [req-bd549338-
Traceback (most recent call last):
File "/usr/lib/
task.run()
File "/usr/lib/
result = f(*args, **kwargs)
File "/usr/lib/
self._run_new()
File "/usr/lib/
result = f(*args, **kwargs)
File "/usr/lib/
self.
File "/usr/lib/
action = self._build_
File "/usr/lib/
self.
File "/usr/lib/