Paunch fails to find the containers it should find during config-download step deployment

Bug #1881229 reported by Brendan Shephard
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
paunch
Invalid
Undecided
Unassigned
tripleo
Invalid
Medium
Unassigned

Bug Description

Summary:

During step deployment, the playbook reaches the maximum number of attempts at this task:
https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/deploy-steps-tasks.yaml#L94-L110

When trying to run the paunch command manually like so:
paunch apply --debug --file /var/lib/tripleo-config/container-startup-config/step_3 --config-id=tripleo_step3 --managed-by=tripleo-Compute

We get:
$ podman ps -a --filter label=managed_by=tripleo-Undercloud --filter label=config_id=tripleo_step3 --format {{.Names}} {{.Labels.container_name}}
b''
b''
$ podman ps -a --filter label=managed_by=paunch --filter label=config_id=tripleo_step3 --format {{.Names}} {{.Labels.container_name}}
b''
b''
$ podman ps -a --filter label=managed_by=tripleo_Undercloud --format {{.Names}} {{.Labels.container_name}}
b''
b''
$ podman ps -a --filter label=managed_by=paunch --format {{.Names}} {{.Labels.container_name}}

Noting that this is an invalid podman command:
# podman ps -a --filter label=managed_by=tripleo-Undercloud --filter label=config_id=tripleo_step3 --format {{.Names}} {{.Labels.container_name}}
Error: `podman ps` takes no arguments

This is because it is missing a --format before {{.Labels.container_name}}. It should be:
# podman ps -a --filter label=managed_by=tripleo-Undercloud --filter label=config_id=tripleo_step3 --format {{.Names}} --format {{.Labels.container_name}}
nova_db_sync
nova_api_ensure_default_cell
ironic_inspector_db_sync
placement_api_db_sync
nova_api_map_cell0
ironic_db_sync
zaqar_db_sync
swift_rsync_fix
swift_copy_rings
placement_api_db_extract_data_from_nova_api
nova_api_db_sync
neutron_ovs_bridge
neutron_db_sync
mistral_db_sync
keystone_db_sync
heat_engine_db_sync
glance_api_db_sync
keystone
iscsid
ironic_inspector_get_ipa
ironic_inspector_init_dnsmasq_dhcp_hostsdir
swift_setup_srv
nova_statedir_owner
ironic_inspector_init_log

We can see that it's missing the --format label before the %s here:
https://opendev.org/openstack/paunch/src/tag/5.3.2/paunch/runner.py#L269
and
https://opendev.org/openstack/paunch/src/tag/5.3.2/paunch/runner.py#L253

Expected results:
paunch should find all of the containers that it needs to find and not fail during the config-download deployment

Changed in tripleo:
milestone: none → victoria-1
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to paunch (master)

Fix proposed to branch: master
Review: https://review.opendev.org/733883

Changed in tripleo:
assignee: nobody → Bogdan Dobrelya (bogdando)
status: New → In Progress
tags: added: queens-backport-potential train-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on paunch (master)

Change abandoned by Bogdan Dobrelya (bogdando) (<email address hidden>) on branch: master
Review: https://review.opendev.org/733883

Changed in tripleo:
status: In Progress → Triaged
assignee: Bogdan Dobrelya (bogdando) → nobody
Revision history for this message
Brendan Shephard (bshephar) wrote :

The quote issue is just how it's printed in debug. There is no bug here that I can find. My initial issue was the response from keystone was larger than the configured MTU on the interfaces. Calls from nova were going to keystone, but the response wasn't making it back.

I initially didn't see this error, and tried to run the paunch command manually to see where it was failing and noticed the "" issue. I've since been informed that this is just how it's printed with debug and not a real problem. So we can close this one off, not a bug.

Changed in paunch:
status: New → Invalid
Changed in tripleo:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.