removal of testresources from setup.cfg has broken this dependency for oslo.db 4.34

Bug #1756352 reported by Mike Bayer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.db
Fix Released
Undecided
Mike Bayer

Bug Description

this is an issue that keeps coming up, for example at https://bugs.launchpad.net/keystone/+bug/1572202.

oslo.db is in this funny position where it is not only a library that other projects use for database access, it is also a library that is used to support running test suites. And as such, it includes modules within its normal library functions that themselves are dependencies on test-oriented libraries like testresources and testscenarios.

This implies that oslo.db really needs to have testresources and testscenarios in its normal requirements.txt, but for whatever reason, they keep getting removed. For a long time they found themselves in setup.cfg within the "test" section which seemed to fix things but then in https://review.openstack.org/#/c/550842/ this was removed again and we are broken again.

To illustrate the issue:

git clone git://git.openstack.org/openstack/keystone
cd keystone
# remove oslo.db from upper-requirments so they get the latest (4.34, not 4.33)
curl https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt | grep -v "oslo.db" > upper-constraints.txt
TOX_TESTENV_PASSENV="UPPER_CONSTRAINTS_FILE" UPPER_CONSTRAINTS_FILE=upper-constraints.txt tox -v -r --pre -e py27 -- 'keystone.tests.unit.*sql.*'

output:

=========================
Failures during discovery
=========================
--- import errors ---
Failed to import test module: keystone.tests.unit.identity.backends.test_sql
Traceback (most recent call last):
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/identity/backends/test_sql.py", line 14, in <module>
    from oslo_db.sqlalchemy import test_fixtures as db_fixtures
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/oslo_db/sqlalchemy/test_fixtures.py", line 18, in <module>
    import testresources
ImportError: No module named testresources

Failed to import test module: keystone.tests.unit.test_sql_banned_operations
Traceback (most recent call last):
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_sql_banned_operations.py", line 22, in <module>
    from oslo_db.sqlalchemy import test_fixtures as db_fixtures
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/oslo_db/sqlalchemy/test_fixtures.py", line 18, in <module>
    import testresources
ImportError: No module named testresources

Failed to import test module: keystone.tests.unit.test_sql_upgrade
Traceback (most recent call last):
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_sql_upgrade.py", line 53, in <module>
    from oslo_db.sqlalchemy import test_fixtures as db_fixtures
  File "/home/classic/Desktop/tmp/keystone/.tox/py27/lib/python2.7/site-packages/oslo_db/sqlalchemy/test_fixtures.py", line 18, in <module>
    import testresources
ImportError: No module named testresources

solution:

testresources and testsceanrios need to be in requirements.txt, *not* test-requirements.txt.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.db (master)

Fix proposed to branch: master
Review: https://review.openstack.org/553801

Changed in oslo.db:
assignee: nobody → Mike Bayer (zzzeek)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.db (master)

Reviewed: https://review.openstack.org/553801
Committed: https://git.openstack.org/cgit/openstack/oslo.db/commit/?id=acd1ceb69906b8357ec7c60257651d4731aad08e
Submitter: Zuul
Branch: master

commit acd1ceb69906b8357ec7c60257651d4731aad08e
Author: Mike Bayer <email address hidden>
Date: Fri Mar 16 10:35:54 2018 -0400

    Add testresources / testscenarios to requirements.txt

    oslo.db is consumed by the test suites of downstream libraries
    as a test dependency; they make use of the oslo_db.sqlalchemy.test_fixtures
    module which imports testresources and testscenarios, therefore
    these are normal requirements of oslo.db; they are not limited
    to test requirements.

    Change-Id: I240fbc28ddf238614ff37e742a1415a5566bcd2d
    Closes-bug: #1756352

Changed in oslo.db:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.db 4.35.0

This issue was fixed in the openstack/oslo.db 4.35.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.