testsuite and tox
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
python-swiftclient |
Incomplete
|
Undecided
|
Unassigned |
Bug Description
tox in gentoo is simply toxic when it comes to running tests. It woud be so much more sane to assure that the testsuite will pass on the system installed dependencies. While tox has its place, I avoid it like the plague in testsuites in python based packages as a gentoo dev package maintainer. Gentoo also has a rule of no connecting to the internet in a test phase. Sheeeeeeesh.
Sure nosetests is just another test runner, but it's my first port of call.
testuser@archtester /mnt/gen2/
GLOB sdist-make: /mnt/gen2/
py27 create: /mnt/gen2/
py27 installdeps: -r/mnt/
py27 inst: /mnt/gen2/
py27 runtests: commands[0]
running testr
running=
running=
running=
running=
running=
Ran 75 tests in 0.796s
PASSED (id=0)
_______
py27: commands succeeded
congratulations :)
GLOB sdist-make: /mnt/gen2/
pep8 create: /mnt/gen2/
pep8 installdeps: -r/mnt/
pep8 inst: /mnt/gen2/
pep8 runtests: commands[0]
pep8 runtests: commands[1]
_______
pep8: commands succeeded
congratulations :)
says tox worked and the testsuite passed. Well, nice, but for breaking gentoo policy in running tests at the very first step.
testuser@archtester /mnt/gen2/
F......
=======
FAIL: tests.test_
-------
_StringException: Traceback (most recent call last):
File "/usr/lib64/
return func(*args, **keywargs)
File "/mnt/gen2/
self.
File "/usr/lib64/
self.
File "/usr/lib64/
raise MismatchError(
MismatchError: !=:
reference = <StringIO.StringIO instance at 0x1305f38>
actual = <StringIO.StringIO instance at 0xdcd680>
-------
Ran 75 tests in 1.699s
FAILED (failures=1)
says we have a problem possibly with system installed mock or testtools and it did in fact run 75 tests.
testuser@archtester /mnt/gen2/
=======
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 75 items
tests/test_
tests/test_
says let's take py.test out of the equation.
Now you could take the easy way out here and dismiss this whole entry since your 'virtual environment making' sees the testsuite pass. That leaves something amiss here though.
Now here is a classic conflict.
./.tox/
says the virtual env has installed python-subunit. Gentoo's python-subunit is in fact called subunit. We have
/usr/lib64/
oooops.
I am guessing the use of a virtual env. assumes the instances of the virtual installed deps will perfectly match the system installed packages. Sorry to shatter that one for you.
So, could you consider making a testsuite that runs perfectly under nose or pytest or
$ python tests/test_all.py?
It's really quite a sane request. 'We' can't use tox as a test runner for all the various reasons above. At this point I have the tox run testsuite declaring the tests fine and who knows what will happen on using system installed deps once installed. Only a few days ago I got a different result again where the issue was the version of testtools.
well after the neutron fiasco, forget it