tripleo

AllNodesValidationConfig ping test fails when OC containes multiple hosts groups

Bug #1896200 reported by Radoslaw Smigielski on 2020-09-18

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	In Progress	Undecided	Radoslaw Smigielski

Bug Description

AllNodesValidationConfig ping test fails when installation contains several host groups.
We have installation with following host types: Controller, DpdkPerformanceCompute, OvsCompute, Storage,...

For all these host groups we have ping_test_ips which looks like this:

> ping_test_ips:
> AvrsCompute: ' '
> Controller: 172.31.0.203 172.17.3.192 172.17.4.105 172.17.1.156 172.31.0.203 172.17.2.185
> 10.75.224.154
> DpdkPerformanceCompute: ' '
> Monitoring: ' '
> OvsCompute: 172.31.1.101 172.17.3.134 172.17.4.44 172.17.1.161 172.17.2.244
> SriovPerformanceCompute: ' '
> Storage: ' '

ping_test_ips is being converted to json in deploy_steps_playbook.yaml
> - name: AllNodesValidationConfig
> script: all_nodes_validation_script.sh
> environment:
> validate_controllers_icmp: "{{ validate_controllers_icmp }}"
> validate_gateways_icmp: "{{ validate_gateways_icmp }}"
> validate_fqdn: "{{ validate_fqdn }}"
> ping_test_ips: "{{ ping_test_ips | to_json }}"
> tripleo_role_name: "{{ tripleo_role_name }}"

Converted to json ping_test_ips is nor correctly broken down into IP addresses:
echo '{
        "AvrsCompute": " ",
        "Controller": "172.31.0.203 172.17.3.192 172.17.4.105 172.17.1.156 172.31.0.203 172.17.2.185 10.75.224.154",
        "DpdkPerformanceCompute": " ",
        "Monitoring": " ",
        "OvsCompute": "172.31.1.101 172.17.3.134 172.17.4.44 172.17.1.161 172.17.2.244",
        "SriovPerformanceCompute": " ",
        "Storage": " "
    }' | sed -e "s| |\n|g" | sort -u

"
",
{
}
10.75.224.154",
172.17.1.156
172.17.1.161
172.17.2.185
172.17.2.244",
172.17.3.134
172.17.3.192
172.17.4.105
172.17.4.44
"172.31.0.203
172.31.0.203
"172.31.1.101
"AvrsCompute":
"Controller":
"DpdkPerformanceCompute":
"Monitoring":
"OvsCompute":
"SriovPerformanceCompute":
"Storage":

Radoslaw Smigielski (radoslaw-smigielski) on 2020-09-18

Changed in tripleo:
assignee:	nobody → Radoslaw Smigielski (radoslaw-smigielski)
status:	New → In Progress

Revision history for this message

Radoslaw Smigielski (radoslaw-smigielski) wrote on 2020-09-18:

Download full text (4.9 KiB)

This is the Ansible error:

2020-09-17 20:01:10,880 p=11897 u=mistral n=ansible | TASK [AllNodesValidationConfig] ************************************************
2020-09-17 20:01:10,881 p=11897 u=mistral n=ansible | Thursday 17 September 2020 20:01:10 +0000 (0:00:00.786) 0:03:39.281 ****
2020-09-17 20:01:11,578 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-0]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.0.203 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.0.203 closed."], "stdout": "Traceback (most recent call last):\r\n File \"<string>\", line 1, in <module>\r\n File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", " File \"<string>\", line 1, in <module>", " File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", " address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,686 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-1]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.1.190 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.1.190 closed."], "stdout": "Traceback (most recent call last):\r\n File \"<string>\", line 1, in <module>\r\n File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", " File \"<string>\", line 1, in <module>", " File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", " address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,793 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-2]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.0.145 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.0.145 closed."], "stdout": "Traceback (most recent call last):\r\n File \"<string>\", line 1, in <module>\r\n File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", " File \"<string>\", line 1, in <module>", " File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", " address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,798 p=11897 u=mistral n=ansible | fatal: [overcloud-ovscompute-bagwell-cloud-1]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.1.123 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.1.123 closed."], "stdout": "Traceback (most recent call last):\r\n File \"<string>\", line 1, in <module>\r\n File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n ...

This is the Ansible error:

2020-09-17 20:01:10,880 p=11897 u=mistral n=ansible | TASK [AllNodesValidationConfig] ************************************************
2020-09-17 20:01:10,881 p=11897 u=mistral n=ansible | Thursday 17 September 2020  20:01:10 +0000 (0:00:00.786)       0:03:39.281 **** 
2020-09-17 20:01:11,578 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-0]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.0.203 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.0.203 closed."], "stdout": "Traceback (most recent call last):\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n    address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", "  File \"<string>\", line 1, in <module>", "  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", "    address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,686 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-1]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.1.190 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.1.190 closed."], "stdout": "Traceback (most recent call last):\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n    address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", "  File \"<string>\", line 1, in <module>", "  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", "    address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,793 p=11897 u=mistral n=ansible | fatal: [overcloud-controller-bagwell-cloud-2]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.0.145 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.0.145 closed."], "stdout": "Traceback (most recent call last):\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n    address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", "  File \"<string>\", line 1, in <module>", "  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", "    address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,798 p=11897 u=mistral n=ansible | fatal: [overcloud-ovscompute-bagwell-cloud-1]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.1.123 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.1.123 closed."], "stdout": "Traceback (most recent call last):\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n    address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", "  File \"<string>\", line 1, in <module>", "  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", "    address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}
2020-09-17 20:01:11,903 p=11897 u=mistral n=ansible | fatal: [overcloud-ovscompute-bagwell-cloud-0]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to 172.31.1.101 closed.\r\n", "stderr_lines": ["Shared connection to 172.31.1.101 closed."], "stdout": "Traceback (most recent call last):\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress\r\n    address)\r\nValueError: '\"' does not appear to be an IPv4 or IPv6 address\r\n", "stdout_lines": ["Traceback (most recent call last):", "  File \"<string>\", line 1, in <module>", "  File \"/usr/lib/python2.7/site-packages/ipaddr.py\", line 77, in IPAddress", "    address)", "ValueError: '\"' does not appear to be an IPv4 or IPv6 address"]}

And finally the main reason why this is failing:

$ python -c import ipaddr; net=ipaddr.IPNetwork('172.17.1.0/24'); addr=ipaddr.IPAddress('"172.17.1.156",'); print(addr in net)
-bash: syntax error near unexpected token `('
[heat-admin@overcloud-ovscompute-bagwell-cloud-1 ~]$ python -c "import ipaddr; net=ipaddr.IPNetwork('172.17.1.0/24'); addr=ipaddr.IPAddress('"172.17.1.156",'); print(addr in net)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/ipaddr.py", line 77, in IPAddress
    address)
ValueError: '172.17.1.156,' does not appear to be an IPv4 or IPv6 address

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-09-18: Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/752646

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-09-13: Change abandoned on tripleo-heat-templates (master)

Change abandoned by "James Slagle <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/752646
Reason: Abandoning this patch per the TripleO Patch Abandonment guidelines
(https://specs.openstack.org/openstack/tripleo-specs/specs/policy/patch-abandonment.html).
If you wish to have this restored and cannot do so yourself, please reach out
via #tripleo on OFTC or the OpenStack Dev mailing list.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.