[RFE] Get rid of tags and replace them with vars to make playbooks atomic

Bug #1657415 reported by Raoul Scarazzini
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Incomplete
High
wes hayutin

Bug Description

Background/Motivation:
----------------------

There's a lot of confusion today around how the atomic approach on the tasks inside playbooks is done. With atomic approach we mean the ability to execute just one specific task instead of the entire playbook.
Some roles uses tags, some others variables or nothing.
We need to find out a common way to make the tasks of the playbooks optional.

Proposed change:
----------------

The proposal is to get rid of all the tags declaration and replace them with specific "step_" variables declared inside each role defaults/main.yml file, in a specific (and hopefully commented) section (the beginning will be logically best).
To make an example, today the first task of baremetal-undercloud.yml playbook contains this:

- name: Baremetal undercloud install
  hosts: localhost
  roles:
    - baremetal-undercloud
  tags:
    - baremetal-undercloud
...
...

And inside the main task of the role we have:

- include: machine-provisioning.yml
  tags:
    - machine-provision

- include: machine-setup.yml
  tags:
    - machine-setup

After the proposed change, this will be inside roles/baremetal-undercloud/defaults/main.yml:

step_baremetal_undercloud: true
step_machine_provision: true
step_machine_setup: true

This inside the playbook:

- name: Baremetal undercloud install
  hosts: localhost
  roles:
    - baremetal-undercloud
  when: step_baremetal_undercloud|bool

And this inside of the main task:

- include: machine-provisioning.yml
  when: step_machine_provision|bool

- include: machine-setup.yml
  when: step_machine_setup|bool

So, no more tags.

Desired outcome:
----------------

After this the user should be able to invoke quickstart by passing extra vars to omit certain steps, like this:

/quickstart.sh \
  --ansible-debug \
  --bootstrap \
  --no-clone \
  --playbook baremetal-undercloud.yml \
  --extra-vars step_machine_provision=false \
  --release $RELEASE \
  <HOSTNAME>

In this case everything will be executed, except the machine provision step.

Open questions/Considerations:
------------------------------

- This modification will be somehow huge, since there are a lot of tags used today on the playbooks;

- The reason why we should use "step_" name convention is that we already have a huge amount of variables with this name, and so keeping this will save some work;

- Other ideas are welcome but we need to consider that even if we choose to use just tasks then we will still use also variables, since they are already used to control infra-task steps;

Tags: quickstart
Revision history for this message
John Trowbridge (trown) wrote :

Thanks for the write-up Raoul, this has been one of my major annoyances about quickstart for a while.

Changed in tripleo-quickstart:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Raoul Scarazzini (rasca) wrote :

So I've tried different approaches to accomplish the tags removal:

############
Approach #0:
############

---
- name: Baremetal undercloud install
  hosts: localhost
  roles:
    - { role: baremetal-undercloud, when: task_baremetal_undercloud|bool }
...
...

There are two tested reviews on this wip:
    https://review.openstack.org/#/c/427243/
    https://review.openstack.org/#/c/427244/

############
Approach #1:
############

main.yml:

- block:
    # Do machine provisioning
    - include: machine-provisioning.yml

    # Prepare machine to be used with TripleO
    - include: machine-setup.yml
  when: task_baremetal_undercloud|bool

###########
Approach #2
###########

tasks.yml (this will be the new name of main.yml):

# Do machine provisioning
- include: machine-provisioning.yml

# Prepare machine to be used with TripleO
- include: machine-setup.yml

main.yml:

- include: tasks.yml
  when: task_baremetal_undercloud|bool

###########
Approach #3
###########

main.yml:

---
- name: Baremetal undercloud install
  hosts: localhost
  tasks:
    - include_role:
        name: baremetal-undercloud
      when:
       - task_baremetal_undercloud|bool

###########

What we need to consider here is that even if a task is under a "when" condition and so you expect it to be skipped if the condition is not met, there are some cases (like the one described here [1], which is not a bug, but how ansible behaves by design) in which something will be processed in any case.
In conclusion each one of the above approaches as pros and cons, but they all do what we need, if we consider we'll have to deal with issues like the one I described then we can move on about this, otherwise we should not abandon tags, and instead maybe fixing rules around how and when to use them.

[1] https://github.com/ansible/ansible/issues/20919

wes hayutin (weshayutin)
no longer affects: tripleo-quickstart
no longer affects: tripleo-quickstart/trunk
Changed in tripleo:
status: New → Triaged
importance: Undecided → High
assignee: nobody → wes hayutin (weshayutin)
milestone: none → pike-2
tags: added: quickstart
Changed in tripleo:
milestone: pike-2 → pike-3
Revision history for this message
Raoul Scarazzini (rasca) wrote :

After some time and some discussion we came out with the idea of keeping tags inside the main playbooks (like quickstart-extras.yml) and managing all the rest with internal variables.
With this approach we will give users the ability to use an atomic approach on the different phases of the deployment.
A review [1] was created to initiate the discussion, today it just adds the tags to the quickstart-extras.yml playbook, but the aim here is to have documentation of the usage so a user can understand how to use all the stuff.

[1] https://review.openstack.org/#/c/473491/

Changed in tripleo:
status: Triaged → In Progress
Changed in tripleo:
milestone: pike-3 → pike-rc1
Changed in tripleo:
milestone: pike-rc1 → pike-rc2
Changed in tripleo:
milestone: pike-rc2 → queens-1
Changed in tripleo:
milestone: queens-1 → queens-2
Changed in tripleo:
milestone: queens-2 → queens-3
Changed in tripleo:
milestone: queens-3 → queens-rc1
Revision history for this message
Raoul Scarazzini (rasca) wrote :

I think we might close this one after this [1] review gets merged. With this in place, we will cover tags on all the primary playbooks, and so we'll give users the ability to make atomic deployments.

[1] https://review.openstack.org/#/c/537669

Changed in tripleo:
milestone: queens-rc1 → rocky-1
Changed in tripleo:
milestone: rocky-1 → rocky-2
Changed in tripleo:
milestone: rocky-2 → rocky-3
Changed in tripleo:
milestone: rocky-3 → rocky-rc1
Changed in tripleo:
milestone: rocky-rc1 → stein-1
Changed in tripleo:
milestone: stein-1 → stein-2
Changed in tripleo:
milestone: stein-2 → stein-3
Revision history for this message
Juan Antonio Osorio Robles (juan-osorio-robles) wrote :

Should we keep this open? is this something we want to pursue still?

Revision history for this message
Raoul Scarazzini (rasca) wrote :

I don't think so, and in addition to that I feel like this was partially achieved with the wider coverage that we used with the tags. See my comment#4.

Changed in tripleo:
milestone: stein-3 → stein-rc1
Changed in tripleo:
milestone: stein-rc1 → train-1
Changed in tripleo:
milestone: train-1 → train-2
Changed in tripleo:
milestone: train-2 → train-3
Changed in tripleo:
milestone: train-3 → ussuri-1
Changed in tripleo:
milestone: ussuri-1 → ussuri-2
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-2 → ussuri-3
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-3 → ussuri-rc3
wes hayutin (weshayutin)
Changed in tripleo:
milestone: ussuri-rc3 → victoria-1
Changed in tripleo:
milestone: victoria-1 → victoria-3
Changed in tripleo:
milestone: victoria-3 → wallaby-1
Changed in tripleo:
milestone: wallaby-1 → wallaby-2
Changed in tripleo:
milestone: wallaby-2 → wallaby-3
Revision history for this message
Marios Andreou (marios-b) wrote :

This is an automated action. Bug status has been set to 'Incomplete' and target milestone has been removed due to inactivity. If you disagree please re-set these values and reach out to us on freenode #tripleo

Changed in tripleo:
milestone: wallaby-3 → none
status: In Progress → Incomplete
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.