test_negative_servers in tempest is disabled because of nova bugs

Can you narrow this down a bit? Which specific tests are failing? What do those tests do? What does compute.log have in it? You mentioned stack traces, can you share them?

Please follow the link in the description and go to the bottom of the console log. You will see the errors. Note that the current
versions of test_images.py and test_servers_negative.py have skips. The problem seems to be more general than one particular test. If you follow the link and go into the logs directory you can see the stacktraces. Like in

http://logs.openstack.org/11936/13/check/gate-tempest-devstack-vm/10571/logs/screen-n-cpu.txt

Looks like all errors are due to the same cause:

2012-09-06 06:00:56 TRACE nova.compute.manager Traceback (most recent call last):
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 167, in decorated_function
2012-09-06 06:00:56 TRACE nova.compute.manager return function(self, context, *args, **kwargs)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 202, in decorated_function
2012-09-06 06:00:56 TRACE nova.compute.manager kwargs['instance']['uuid'], e, sys.exc_info())
2012-09-06 06:00:56 TRACE nova.compute.manager File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-06 06:00:56 TRACE nova.compute.manager self.gen.next()
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 196, in decorated_function
2012-09-06 06:00:56 TRACE nova.compute.manager return function(self, context, *args, **kwargs)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 816, in run_instance
2012-09-06 06:00:56 TRACE nova.compute.manager do_run_instance()
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/utils.py", line 751, in inner
2012-09-06 06:00:56 TRACE nova.compute.manager retval = f(*args, **kwargs)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 815, in do_run_instance
2012-09-06 06:00:56 TRACE nova.compute.manager admin_password, is_first_time, instance)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 512, in _run_instance
2012-09-06 06:00:56 TRACE nova.compute.manager self._set_instance_error_state(context, instance['uuid'])
2012-09-06 06:00:56 TRACE nova.compute.manager File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-06 06:00:56 TRACE nova.compute.manager self.gen.next()
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 477, in _run_instance
2012-09-06 06:00:56 TRACE nova.compute.manager self._start_building(context, instance)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 687, in _start_building
2012-09-06 06:00:56 TRACE nova.compute.manager None))
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/compute/manager.py", line 254, in _instance_update
2012-09-06 06:00:56 TRACE nova.compute.manager context, instance_uuid, kwargs)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/db/api.py", line 675, in instance_update_and_get_original
2012-09-06 06:00:56 TRACE nova.compute.manager values)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 130, in wrapper
2012-09-06 06:00:56 TRACE nova.compute.manager return f(*args, **kwargs)
2012-09-06 06:00:56 TRACE nova.compute.manager File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1768, in instance_update_and_get_original
2012-09-06 06:00:56 TRAC...

Seems to no longer happen in recent runs...

Thierry, maybe you missed the bit in comment #2 that these tests are now skipped in tempest so that they do not continue to fail and block other checkins. I tried enabling them again and the same failures happen.

Even with these tests commented out the n-cpu log is strewn with ERRORs and backtraces, some of them on the scary side at least to me. Is any one concerned about this?

Hi David,

From the last comment I understand that the errors seen are not related to the tests that are commented. I see the tempest tests to be skipped even in top of the tree.

Would it make sense to enable them now. If you are fine with enabling the tests, I will raise a bug in launchpad and get the tests enabled.

Please let me know.

Regards
Nithya

Fix proposed to branch: master
Review: https://review.openstack.org/23199

Fix proposed to branch: master
Review: https://review.openstack.org/23203

Reviewed: https://review.openstack.org/23199
Committed: http://github.com/openstack/nova/commit/d6999fbe8c545292a98c4e3d20f3b80e3e66d901
Submitter: Jenkins
Branch: master

commit d6999fbe8c545292a98c4e3d20f3b80e3e66d901
Author: Sean Dague <email address hidden>
Date: Thu Feb 28 14:58:42 2013 -0500

    don't stack trace if long ints are passed to db

    If we pass a integer, which is longer than the id field in the
    database, but a valid python integer into any of the REST API
    calls SQLalchemy throws a DataError, and the user sees a 500
    instead of InstanceNotFound. This can be triggered by using a
    id of 9223372036854775808 (for instance).

    This issue is seen on postgresql, mysql and sqlite don't hit it.

    Instead, catch the DataError exception, and return an InvalidID
    exception. If InvalidID is thrown catch that at the API layer
    and return the expected not found.

    This was caught when reenabling tempest tests for bug #1046870
    and is one of serveral patches in nova needed to enable
    test_servers_negative.py in tempest.

    Change-Id: I6130863a95567eabd3ae05a09e4a9f26bb5f5098

Reviewed: https://review.openstack.org/23203
Committed: http://github.com/openstack/nova/commit/1c6cc18b0cef25bfff8566c2a646cd8bcbbe132a
Submitter: Jenkins
Branch: master

commit 1c6cc18b0cef25bfff8566c2a646cd8bcbbe132a
Author: Sean Dague <email address hidden>
Date: Thu Feb 28 15:32:33 2013 -0500

    validate security_groups on server create

    security_groups were getting passed all the way into the database
    without ever checking for their existance. This allowed for crudding
    up the db with bad data, as well as succeeding when an admin typoed
    a security group, and getting no active security group.

    Also, security_group is actually an array, which is not implied by
    it being a singular variable. Make it a plural variable to cause
    less confusion in the future.

    And given that we multiply change security_groups from None =>
    ['default'], get rid of a utility function that serves no purpose
    to handle the None or string cases.

    To test this correctly, stub out the db call, so we manage to get
    all the way down to the bottom of the stack sanely. Also ensure that
    the negative case gets us the expected HTTPBadRequest.

    Partial fix for nova bugs needed to reenable test_servers_negative
    in Tempest. Fixes bug #1046870

    Change-Id: I5b477f7bc1992abb47e014f137b8010cc76b2113

Reviewed: https://review.openstack.org/23110
Committed: http://github.com/openstack/tempest/commit/e623f75ca687db6ada2efded3509d7be1bf738fe
Submitter: Jenkins
Branch: master

commit e623f75ca687db6ada2efded3509d7be1bf738fe
Author: Sean Dague <email address hidden>
Date: Wed Feb 27 14:52:15 2013 -0500

    enable test_servers_negative

    this was disabled when we were seeing flakey fails during Folsom
    release, however all but one of these tests current pass. And the
    failure is a new nova bug. Pushing to CI to verify that bug on both
    databases before providing the fixes to nova (of which there will
    be at least 2)

    fix xml clients for negative tests

    the xml clients were failing additional tests because the clients
    didn't support all the features as the json tests. This enables
    security_groups and networks in the xml client for create.

    It also skips the test_create_numeric_server_name as that can't be
    tested in xml, it always turns it into a string.

    Fixes bug #1046870

    Change-Id: I8a3a0bef9b1d134da369242a0cf14c3dcb61c6de