Premature evaluation of "publish" if a task has "retry" with "continue-on"

Bug #1878167 reported by Renat Akhmerov
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
New
High
Unassigned

Bug Description

Consider the following example:

wf:
  tasks:
    task1:
      retry:
        continue-on: <% global(global_var) = null %>
        count: 10
        delay: 1
      on-success:
        publish:
          branch:
            global_var_doubled: <% $.global_var + $.global_var %>

    task2:
      wait-before: 3
      on-success:
        publish:
          global:
            global_var: foobar

The problem is that the "publish" clause of "task1" gets evaluated after the first iteration of retry when "global_var" is not yet available in the global context, so it can't find the variable and fails with the error:

 File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/engine/task_handler.py", line 120, in _on_action_complete
    task.on_action_complete(action_ex)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/.tox/py37/lib/python3.7/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/engine/tasks.py", line 499, in on_action_complete
    self.complete(state, state_info)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/.tox/py37/lib/python3.7/site-packages/osprofiler/profiler.py", line 160, in wrapper
    result = f(*args, **kwargs)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/engine/tasks.py", line 283, in complete
    data_flow.publish_variables(self.task_ex, self.task_spec)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/workflow/data_flow.py", line 220, in publish_variables
    task_ex.published = expr.evaluate_recursively(branch_vars, expr_ctx)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/expressions/__init__.py", line 100, in evaluate_recursively
    data[key] = _evaluate_item(data[key], context)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/expressions/__init__.py", line 79, in _evaluate_item
    return evaluate(item, context)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/expressions/__init__.py", line 71, in evaluate
    return evaluator.evaluate(expression, context)
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/expressions/yaql_expression.py", line 228, in evaluate
    data_context
  File "/Users/renat/workspace/openstack/mistral_group/mistral/mistral/expressions/yaql_expression.py", line 182, in evaluate
    ", data=%s]" % (expression, str(e), data_context)
mistral.exceptions.YaqlEvaluationException: Can not evaluate YAQL expression [expression=$.global_var + $.global_var, error='global_var', data={'__task_execution': {'id': '668067b7-e015-409d-ba8b-acb38244e482', 'name': 'task1'}, , '__env': {}, '__execution': {'id': '610d252f-574e-44e7-a14f-9c5a109d5ed7'}, }]
] (execution_id=610d252f-574e-44e7-a14f-9c5a109d5ed7)

If we move "wait-before" from "task2" to "task1" the example will be working.

The issue is in the design of task policies. In case if a task action (std.noop in our case since "action" is not explicitly specified) succeeds it will run the logic of task completion before the "retry" policy has a chance to change the flow. See how the method "complete" works at https://github.com/openstack/mistral/blob/master/mistral/engine/tasks.py#L283. It publishes variables before the the call "self._after_task_complete()" that triggers policies.

Changed in mistral:
importance: Undecided → High
milestone: none → victoria-1
Changed in mistral:
milestone: victoria-1 → wallaby-1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.