Migration test locking is broken

Bug #1255608 reported by Ben Nemec
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo-incubator
Fix Released
High
Ben Nemec

Bug Description

The locking in openstack/common/db/sqlalchemy/test_migrations.py is currently broken. It hard-codes a relative path for the lock file, which does not work in Jenkins runs because we don't have permission to create files in that location. It's wrong to do that anyway, so we need to switch it to use the OSLO_LOCK_PATH value.

See https://review.openstack.org/#/c/56516 for an example of this failing.

Ben Nemec (bnemec)
Changed in oslo:
importance: Undecided → High
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo-incubator (master)

Fix proposed to branch: master
Review: https://review.openstack.org/58830

Changed in oslo:
assignee: nobody → Ben Nemec (bnemec)
status: Triaged → In Progress
Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

This is copied from http://logs.openstack.org/16/56516/13/check/gate-ironic-python26/c7798a4/testr_results.html.gz
Showing the detail:

ft1.304: ironic.tests.db.sqlalchemy.test_migrations.TestMigrations.test_postgresql_opportunistically_StringException: Empty attachments:
  pythonlogging:'ironic.openstack.common'
  stderr
  stdout

traceback-1: {{{
Traceback (most recent call last):
  File "ironic/openstack/common/db/sqlalchemy/test_migrations.py", line 160, in tearDown
    self._reset_databases()
  File "ironic/openstack/common/db/sqlalchemy/test_migrations.py", line 218, in _reset_databases
    self._reset_pg(conn_pieces)
  File "ironic/openstack/common/db/sqlalchemy/test_migrations.py", line 105, in wrapper
    with lock:
  File "/home/jenkins/workspace/gate-ironic-python26/.tox/py26/lib/python2.6/site-packages/lockfile/__init__.py", line 226, in __enter__
    self.acquire()
  File "/home/jenkins/workspace/gate-ironic-python26/.tox/py26/lib/python2.6/site-packages/lockfile/linklockfile.py", line 20, in acquire
    raise LockFailed("failed to create %s" % self.unique_name)
LockFailed: failed to create /home/jenkins/workspace/gate-ironic-python26/pgadmin/centos6-5.slave.openstack.org-4134c700.6738
}}}

Traceback (most recent call last):
  File "ironic/tests/db/sqlalchemy/test_migrations.py", line 296, in test_postgresql_opportunistically
    self._test_postgresql_opportunistically()
  File "ironic/tests/db/sqlalchemy/test_migrations.py", line 275, in _test_postgresql_opportunistically
    self._reset_databases()
  File "ironic/openstack/common/db/sqlalchemy/test_migrations.py", line 218, in _reset_databases
    self._reset_pg(conn_pieces)
  File "ironic/openstack/common/db/sqlalchemy/test_migrations.py", line 105, in wrapper
    with lock:
  File "/home/jenkins/workspace/gate-ironic-python26/.tox/py26/lib/python2.6/site-packages/lockfile/__init__.py", line 226, in __enter__
    self.acquire()
  File "/home/jenkins/workspace/gate-ironic-python26/.tox/py26/lib/python2.6/site-packages/lockfile/linklockfile.py", line 20, in acquire
    raise LockFailed("failed to create %s" % self.unique_name)
LockFailed: failed to create /home/jenkins/workspace/gate-ironic-python26/pgadmin/centos6-5.slave.openstack.org-4134c700.6738

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo-incubator (master)

Reviewed: https://review.openstack.org/58830
Committed: http://github.com/openstack/oslo-incubator/commit/4c47d3e759f15f1f3b230323ca723fbb1dd90a4f
Submitter: Jenkins
Branch: master

commit 4c47d3e759f15f1f3b230323ca723fbb1dd90a4f
Author: Ben Nemec <email address hidden>
Date: Wed Nov 27 17:41:25 2013 +0000

    Fix locking in migration tests

    The migration test code included an external lock with a hard-coded
    relative path that breaks when run in Jenkins. It wasn't being hit
    in Oslo because it was only locking for the postgres code path,
    which isn't currently tested in Oslo.

    This change moves the lock so that it applies to all db types and
    removes the relative path. For the locks to work properly it was
    also necessary to add the lockutils wrapper to the Oslo tox.ini.

    Closes-Bug: 1255608
    Change-Id: I333b833022f208e48e3b7e82e8cfad2764be5ea0

Changed in oslo:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in oslo:
milestone: none → icehouse-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in oslo:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.