tasks listed after pause do not run when workflow is resumed

Bug #1714054 reported by Matt B
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
Fix Released
High
Renat Akhmerov

Bug Description

When "pause" is in an on-success list, followed by other tasks, it's expected that the following tasks will run after the workflow is resumed. They do not appear to be triggered.

Mistral Workflow Service, version 4.0.1

Example workflow:
--------------------------------------
version: 2.0
workflow:
  tasks:
    task1:
      action: std.echo output="task1"
      on-success:
        - task2
        - pause
        - task3
    task2:
      action: std.echo output="task2"
    task3:
      action: std.echo output="task3"
--------------------------------------

Reproduction steps:

➜ ~ mistral workflow-create -f yaml test_pause.yaml
- Created at: '2017-08-30 17:06:14'
  ID: 9d4954c3-6cca-497c-8357-c60234df1a29
  Input: ''
  Name: workflow
  Project ID: <default-project>
  Tags: <none>
  Updated at: null
➜ ~ mistral execution-create workflow
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| ID | 33f43510-a188-4c42-ab11-f02f0089dfa6 |
| Workflow ID | 9d4954c3-6cca-497c-8357-c60234df1a29 |
| Workflow name | workflow |
| Description | |
| Task Execution ID | <none> |
| State | RUNNING |
| State info | None |
| Created at | 2017-08-30 17:06:25 |
| Updated at | 2017-08-30 17:06:25 |
+-------------------+--------------------------------------+
➜ ~ mistral task-list 33f43510-a188-4c42-ab11-f02f0089dfa6
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
| ID | Name | Workflow name | Execution ID | State | State info | Created at | Updated at |
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
| 39008327-2a9d-42ad- | task2 | workflow | 33f43510-a188-4c42-ab1 | SUCCESS | None | 2017-08-30 17:06:25 | 2017-08-30 17:06:25 |
| 8dbc-8c1dbd07ce5d | | | 1-f02f0089dfa6 | | | | |
| adb9f776-135f-451a- | task1 | workflow | 33f43510-a188-4c42-ab1 | SUCCESS | None | 2017-08-30 17:06:25 | 2017-08-30 17:06:25 |
| b05e-554a40134249 | | | 1-f02f0089dfa6 | | | | |
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
➜ ~ mistral execution-get 33f43510-a188-4c42-ab11-f02f0089dfa6
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| ID | 33f43510-a188-4c42-ab11-f02f0089dfa6 |
| Workflow ID | 9d4954c3-6cca-497c-8357-c60234df1a29 |
| Workflow name | workflow |
| Description | |
| Task Execution ID | <none> |
| State | PAUSED |
| State info | None |
| Created at | 2017-08-30 17:06:25 |
| Updated at | 2017-08-30 17:06:25 |
+-------------------+--------------------------------------+

➜ ~ mistral execution-update -s RUNNING 33f43510-a188-4c42-ab11-f02f0089dfa6
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| ID | 33f43510-a188-4c42-ab11-f02f0089dfa6 |
| Workflow ID | 9d4954c3-6cca-497c-8357-c60234df1a29 |
| Workflow name | workflow |
| Description | |
| Task Execution ID | <none> |
| State | SUCCESS |
| State info | None |
| Created at | 2017-08-30 17:06:25 |
| Updated at | 2017-08-30 17:08:06 |
+-------------------+--------------------------------------+
➜ ~ mistral task-list 33f43510-a188-4c42-ab11-f02f0089dfa6
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
| ID | Name | Workflow name | Execution ID | State | State info | Created at | Updated at |
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
| 39008327-2a9d-42ad- | task2 | workflow | 33f43510-a188-4c42-ab1 | SUCCESS | None | 2017-08-30 17:06:25 | 2017-08-30 17:08:06 |
| 8dbc-8c1dbd07ce5d | | | 1-f02f0089dfa6 | | | | |
| adb9f776-135f-451a- | task1 | workflow | 33f43510-a188-4c42-ab1 | SUCCESS | None | 2017-08-30 17:06:25 | 2017-08-30 17:06:25 |
| b05e-554a40134249 | | | 1-f02f0089dfa6 | | | | |
+------------------------+-------+---------------+------------------------+---------+------------+---------------------+---------------------+
➜ ~ mistral execution-get 33f43510-a188-4c42-ab11-f02f0089dfa6
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| ID | 33f43510-a188-4c42-ab11-f02f0089dfa6 |
| Workflow ID | 9d4954c3-6cca-497c-8357-c60234df1a29 |
| Workflow name | workflow |
| Description | |
| Task Execution ID | <none> |
| State | SUCCESS |
| State info | None |
| Created at | 2017-08-30 17:06:25 |
| Updated at | 2017-08-30 17:08:06 |
+-------------------+--------------------------------------+

Revision history for this message
Matt B (matty-brennan) wrote :
Revision history for this message
Renat Akhmerov (rakhmerov) wrote :

You wrote that you used Mistral 4.0.1. Did you try the latest version?

Changed in mistral:
milestone: none → queens-1
importance: Undecided → High
Changed in mistral:
assignee: nobody → noa (noa-koffman)
Changed in mistral:
milestone: queens-1 → queens-2
Changed in mistral:
milestone: queens-2 → queens-3
Changed in mistral:
assignee: noa (noa-koffman) → Hardik Jasani (hjasani)
Revision history for this message
Matt B (matty-brennan) wrote :

I've confirmed this is still an issue on mistral 5.2, running the provided docker image.

Changed in mistral:
milestone: queens-3 → rocky-1
Revision history for this message
Renat Akhmerov (rakhmerov) wrote :

Yes, I also just checked. It really doesn't work as expected.

Revision history for this message
Renat Akhmerov (rakhmerov) wrote :

I check on the latest master.

Changed in mistral:
assignee: Hardik Jasani (hjasani) → Renat Akhmerov (rakhmerov)
Revision history for this message
Renat Akhmerov (rakhmerov) wrote :

Seems like it was completely broken some time ago when doing engine refactoring. I'm surprised that it's not covered by tests.

Changed in mistral:
milestone: rocky-1 → rocky-rc2
status: New → Confirmed
tags: added: queens-backport
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to mistral (master)

Fix proposed to branch: master
Review: https://review.openstack.org/552455

Changed in mistral:
status: Confirmed → In Progress
Changed in mistral:
milestone: rocky-rc2 → rocky-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to mistral (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/554794

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to mistral (master)

Reviewed: https://review.openstack.org/552455
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=9726189c4309d8c5f07a63ac64eeaa2c15af2f76
Submitter: Zuul
Branch: master

commit 9726189c4309d8c5f07a63ac64eeaa2c15af2f76
Author: Renat Akhmerov <email address hidden>
Date: Mon Feb 19 17:35:11 2018 +0200

    Fix 'pause' engine command

    * Commands going after 'pause' in 'on-XXX' clauses
      were never processed after workflow resume. The
      solution is to introduce a notion of a workflow
      execution backlog where we can save these commands
      in a serialized form so that the engine dispatcher
      could see and process them after resume.
    * Other minor changes

    Change-Id: I963b5660daf528d1caf6a785311de4fb272cafd0
    Closes-Bug: #1714054

Changed in mistral:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to mistral (master)

Reviewed: https://review.openstack.org/554794
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=8b30743d0a5e44c7673f90b4509989f8f35a7cb3
Submitter: Zuul
Branch: master

commit 8b30743d0a5e44c7673f90b4509989f8f35a7cb3
Author: Renat Akhmerov <email address hidden>
Date: Wed Mar 21 11:44:05 2018 +0700

    Add a release note for the 'pause' command fix

    Change-Id: Ia609cb319a01b3685d91515c8d9d81e3006a001e
    Related-Bug: #1714054

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/mistral 7.0.0.0b1

This issue was fixed in the openstack/mistral 7.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.