Lots of test methods do not clean up after themselves. One example is all the test cases in test_servers* in the tempest test suite that call the servers_client.create_server method but don't clean up that server instance after running into a failure.
What this leads to is lots of the following:
======================================================================
ERROR: The server name should be changed to the the provided value
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/stack/tempest/storm/tests/test_servers.py", line 83, in test_update_server_name
self.flavor_ref)
File "/home/stack/tempest/storm/services/nova/json/servers_client.py", line 62, in create_server
error=body)
CreateServerFailure: Failed to create server. Response status code: 413. API call returned the following: {"overLimit": {"message": "Instance quotas have been exceeded", "code": 413, "retryAfter": 0}}
Since previous test cases had failures and didn't delete server instances that the test case created.
What this means is lots of false negative test case failures.
The solution for this is to have the tearDown() method handle cleanup from failed test cases and have the test methods inform the test class of things the test method creates..
test_list_images.py has no tearDownClass method to clean up created images and servers.