Tempest failing on TestNetworkBasicOps

Bug #1422936 reported by David Wilde
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-ansible
Fix Released
Medium
Nolan Brubaker
Icehouse
Fix Released
Medium
Nolan Brubaker
Juno
Fix Released
Medium
Nolan Brubaker
Trunk
Fix Released
Medium
Nolan Brubaker

Bug Description

Tempest is failing running the TestNetworkBasicOps scenario on an AIO:

root@aio1_utility_container-419a9703:/opt/tempest_3# cat test_list2
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,gate,network,smoke]
root@aio1_utility_container-419a9703:/opt/tempest_3# ./run_tempest.sh --no-virtual-env -s -- --load-list=test_list2
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps
    test_network_basic_ops[compute,gate,network,smoke] FAIL

Slowest 1 tests took 1.21 secs:
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps
    test_network_basic_ops[compute,gate,network,smoke] 1.21

======================================================================
FAIL: tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,gate,network,smoke]
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{
2015-02-17 22:18:33,395 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 200 POST http://172.29.236.100:5000/v2.0/tokens
2015-02-17 22:18:33,535 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-groups 0.139s
2015-02-17 22:18:34,014 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.477s
2015-02-17 22:18:34,072 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.057s
2015-02-17 22:18:34,139 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.065s
2015-02-17 22:18:34,194 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.054s
2015-02-17 22:18:34,244 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/networks 0.049s
2015-02-17 22:18:34,264 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 404 GET http://172.29.236.100:9696/v2.0/routers/412c8b09-14d0-4c5a-840f-60d8529d80fb 0.020s
2015-02-17 22:18:34,338 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/networks/60ffa252-62e5-413d-9e33-46d9305d27ef 0.072s
2015-02-17 22:18:34,376 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/b29687dc-a43c-42c5-93dc-e9c1ce89dd22 0.037s
2015-02-17 22:18:34,416 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/c7167fdd-2e5e-4a9b-92bc-f4f27cf325e8 0.038s
2015-02-17 22:18:34,454 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/533b43dc-7b3c-46e5-850b-98c2faceffdc 0.037s
2015-02-17 22:18:34,493 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/2bf20591-c35d-4860-bbfd-3d8caec672d2 0.038s
2015-02-17 22:18:34,544 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-groups/9245500d-2e6b-4aa2-a42a-50591e800270 0.049s
}}}

Traceback (most recent call last):
  File "/opt/tempest_3/tempest/test.py", line 112, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 365, in test_network_basic_ops
    self._setup_network_and_servers()
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 108, in _setup_network_and_servers
    self.network, self.subnet, self.router = self.create_networks()
  File "/opt/tempest_3/tempest/scenario/manager.py", line 1012, in create_networks
    router = self._get_router(client=client, tenant_id=tenant_id)
  File "/opt/tempest_3/tempest/scenario/manager.py", line 966, in _get_router
    resp, body = client.show_router(router_id)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 139, in _show
    resp, body = self.get(uri)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 80, in get
    return self.rest_client.get(uri, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 235, in get
    return self.request('GET', url, extra_headers, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 434, in request
    resp, resp_body)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 478, in _error_checker
    raise exceptions.NotFound(resp_body)
NotFound: Object not found
Details: {"NeutronError": {"message": "Router 412c8b09-14d0-4c5a-840f-60d8529d80fb could not be found", "type": "RouterNotFound", "detail": ""}}
Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2015-02-17 22:18:33,395 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 200 POST http://172.29.236.100:5000/v2.0/tokens
2015-02-17 22:18:33,535 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-groups 0.139s
2015-02-17 22:18:34,014 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.477s
2015-02-17 22:18:34,072 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.057s
2015-02-17 22:18:34,139 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.065s
2015-02-17 22:18:34,194 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.054s
2015-02-17 22:18:34,244 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/networks 0.049s
2015-02-17 22:18:34,264 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 404 GET http://172.29.236.100:9696/v2.0/routers/412c8b09-14d0-4c5a-840f-60d8529d80fb 0.020s
2015-02-17 22:18:34,338 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/networks/60ffa252-62e5-413d-9e33-46d9305d27ef 0.072s
2015-02-17 22:18:34,376 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/b29687dc-a43c-42c5-93dc-e9c1ce89dd22 0.037s
2015-02-17 22:18:34,416 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/c7167fdd-2e5e-4a9b-92bc-f4f27cf325e8 0.038s
2015-02-17 22:18:34,454 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/533b43dc-7b3c-46e5-850b-98c2faceffdc 0.037s
2015-02-17 22:18:34,493 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/2bf20591-c35d-4860-bbfd-3d8caec672d2 0.038s
2015-02-17 22:18:34,544 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-groups/9245500d-2e6b-4aa2-a42a-50591e800270 0.049s
}}}

Traceback (most recent call last):
  File "/opt/tempest_3/tempest/test.py", line 112, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 365, in test_network_basic_ops
    self._setup_network_and_servers()
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 108, in _setup_network_and_servers
    self.network, self.subnet, self.router = self.create_networks()
  File "/opt/tempest_3/tempest/scenario/manager.py", line 1012, in create_networks
    router = self._get_router(client=client, tenant_id=tenant_id)
  File "/opt/tempest_3/tempest/scenario/manager.py", line 966, in _get_router
    resp, body = client.show_router(router_id)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 139, in _show
    resp, body = self.get(uri)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 80, in get
    return self.rest_client.get(uri, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 235, in get
    return self.request('GET', url, extra_headers, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 434, in request
    resp, resp_body)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 478, in _error_checker
    raise exceptions.NotFound(resp_body)
NotFound: Object not found
Details: {"NeutronError": {"message": "Router 412c8b09-14d0-4c5a-840f-60d8529d80fb could not be found", "type": "RouterNotFound", "detail": ""}}

Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2015-02-17 22:18:33,395 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 200 POST http://172.29.236.100:5000/v2.0/tokens
2015-02-17 22:18:33,535 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-groups 0.139s
2015-02-17 22:18:34,014 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.477s
2015-02-17 22:18:34,072 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.057s
2015-02-17 22:18:34,139 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.065s
2015-02-17 22:18:34,194 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/security-group-rules 0.054s
2015-02-17 22:18:34,244 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://172.29.236.100:9696/v2.0/networks 0.049s
2015-02-17 22:18:34,264 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 404 GET http://172.29.236.100:9696/v2.0/routers/412c8b09-14d0-4c5a-840f-60d8529d80fb 0.020s
2015-02-17 22:18:34,338 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/networks/60ffa252-62e5-413d-9e33-46d9305d27ef 0.072s
2015-02-17 22:18:34,376 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/b29687dc-a43c-42c5-93dc-e9c1ce89dd22 0.037s
2015-02-17 22:18:34,416 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/c7167fdd-2e5e-4a9b-92bc-f4f27cf325e8 0.038s
2015-02-17 22:18:34,454 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/533b43dc-7b3c-46e5-850b-98c2faceffdc 0.037s
2015-02-17 22:18:34,493 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-group-rules/2bf20591-c35d-4860-bbfd-3d8caec672d2 0.038s
2015-02-17 22:18:34,544 12517 INFO [tempest.common.rest_client] Request (TestNetworkBasicOps:_run_cleanups): 204 DELETE http://172.29.236.100:9696/v2.0/security-groups/9245500d-2e6b-4aa2-a42a-50591e800270 0.049s
}}}

Traceback (most recent call last):
  File "/opt/tempest_3/tempest/test.py", line 112, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 365, in test_network_basic_ops
    self._setup_network_and_servers()
  File "/opt/tempest_3/tempest/scenario/test_network_basic_ops.py", line 108, in _setup_network_and_servers
    self.network, self.subnet, self.router = self.create_networks()
  File "/opt/tempest_3/tempest/scenario/manager.py", line 1012, in create_networks
    router = self._get_router(client=client, tenant_id=tenant_id)
  File "/opt/tempest_3/tempest/scenario/manager.py", line 966, in _get_router
    resp, body = client.show_router(router_id)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 139, in _show
    resp, body = self.get(uri)
  File "/opt/tempest_3/tempest/services/network/network_client_base.py", line 80, in get
    return self.rest_client.get(uri, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 235, in get
    return self.request('GET', url, extra_headers, headers)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 434, in request
    resp, resp_body)
  File "/opt/tempest_3/tempest/common/rest_client.py", line 478, in _error_checker
    raise exceptions.NotFound(resp_body)
NotFound: Object not found
Details: {"NeutronError": {"message": "Router 412c8b09-14d0-4c5a-840f-60d8529d80fb could not be found", "type": "RouterNotFound", "detail": ""}}

Ran 1 test in 3.278s

FAILED (failures=1)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-ansible-deployment (master)

Fix proposed to branch: master
Review: https://review.openstack.org/158467

Changed in openstack-ansible:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (master)

Reviewed: https://review.openstack.org/158467
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=b0242bdfc82f00394418e868197510605fa0f925
Submitter: Jenkins
Branch: master

commit b0242bdfc82f00394418e868197510605fa0f925
Author: Nolan Brubaker <email address hidden>
Date: Mon Feb 23 17:13:13 2015 -0500

    Remove conflicting public_router_id option

    Removes the public_router_id option from tempest.conf, which results in
    RouterNotFound failures before the NetworkbasicOps tests can actually commence.
    Removing this option allows the tests to proceed.

    According to the tempest.conf.sample file, this should only ever be set
    when neutron's `allow_overlapping_ips` setting is False. (It is set to
    True currently in os-ansible-deployment).

    Closes-Bug: #1422936
    Change-Id: I87b9102c15d44a1f8861e6e21c7fbd76b3e71600

Changed in openstack-ansible:
status: In Progress → Fix Committed
Julian Montez (jpmontez)
tags: added: juno-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-ansible-deployment (juno)

Fix proposed to branch: juno
Review: https://review.openstack.org/161440

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-ansible-deployment (icehouse)

Fix proposed to branch: icehouse
Review: https://review.openstack.org/161443

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (icehouse)

Reviewed: https://review.openstack.org/161443
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=876437881467948f9a204c5f09b2f863d058f86c
Submitter: Jenkins
Branch: icehouse

commit 876437881467948f9a204c5f09b2f863d058f86c
Author: Nolan Brubaker <email address hidden>
Date: Mon Feb 23 17:13:13 2015 -0500

    Remove conflicting public_router_id option

    Removes the public_router_id option from tempest.conf, which results in
    RouterNotFound failures before the NetworkbasicOps tests can actually commence.
    Removing this option allows the tests to proceed.

    According to the tempest.conf.sample file, this should only ever be set
    when neutron's `allow_overlapping_ips` setting is False. (It is set to
    True currently in os-ansible-deployment).

    Closes-Bug: #1422936
    Change-Id: I87b9102c15d44a1f8861e6e21c7fbd76b3e71600
    (cherry picked from commit b0242bdfc82f00394418e868197510605fa0f925)

    Conflicts:
     rpc_deployment/roles/tempest/templates/tempest.conf.j2
     rpc_deployment/roles/tempest_resources/tasks/main.yml

    Change-Id: I0939d979482af17605f5bc0489bc09aae8c45a8c

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (juno)

Reviewed: https://review.openstack.org/161440
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=57bebf121ae271339670caf9ca4b4dbe4ba94f97
Submitter: Jenkins
Branch: juno

commit 57bebf121ae271339670caf9ca4b4dbe4ba94f97
Author: Nolan Brubaker <email address hidden>
Date: Mon Feb 23 17:13:13 2015 -0500

    Remove conflicting public_router_id option

    Removes the public_router_id option from tempest.conf, which results in
    RouterNotFound failures before the NetworkbasicOps tests can actually commence.
    Removing this option allows the tests to proceed.

    According to the tempest.conf.sample file, this should only ever be set
    when neutron's `allow_overlapping_ips` setting is False. (It is set to
    True currently in os-ansible-deployment).

    Closes-Bug: #1422936
    Change-Id: I87b9102c15d44a1f8861e6e21c7fbd76b3e71600
    (cherry picked from commit b0242bdfc82f00394418e868197510605fa0f925)

    Conflicts:
     rpc_deployment/roles/tempest/templates/tempest.conf.j2
     rpc_deployment/roles/tempest_resources/tasks/main.yml

    Change-Id: I8a45a99c68db0ddce6d8fdf8cb3f7a79316d55e8

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.