$ yum install python-pip rhnlib $ sudo pip install -r test-requirements.txt
Jump in to your subscription-manager check-out and run
nosetest. The tests will run. However, if you are in an SSH session, then some of the GTK tests will fail. Make sure that you have the DISPLAY environment variable set:
export DISPLAY=:0.0. Or you can use Xvfb and never have to worry about the problem again. See the nose-xvfb section below.
Nose is really great at letting you zero in on just what you want to run.
Want to run just one module? Just provide the path to that module. You can also use the Python dotted notation (test.my_test), but using the path lets you use tab completion in your shell
$ nosetests test/my_test.py
How about a single class?
$ nosetests test/my_test.py:TestClass
And now let’s get really specific and run just one test
$ nosetests test/my_test.py:TestClass.test_method
It can be helpful to configure nose to assign numeric IDs to each test making it easier to run specific tests.
In your global
When you run Nose, to create the ids, run with
--with-id. Print all the IDs
-v option. Then use the numbers for additional runs.
$ nosetests --with-id -v tests/my_test.py #1 test_blah (test.my_test.TestClass) ... ok #2 test_blah2 (test.my_test.TestClass) ... ok $ nosetests 1 2
Be aware, however, that the
--with-id option causes issues when Nose is run in
verbose mode and you are using YANC to print the output in color. There is an
unmerged PR around the issue but
Nose is no longer actively maintained.
IDs also allow Nose to run the tests that failed during the previous run
$ nosetests --failed
You can configure Nose to use certain options by default via a
See the docs
for more information.
Ordinarily, Nose will run all the tests in the same order every time. This can
cause problems where a test pollutes the test environment (e.g. via
monkey-patching) and other tests are written that unwittingly rely on this
pollution. To combat this issue, we use nose-randomly which randomly orders
both the test modules and the test methods within a module. Nose-randomly is
setup.cfg to run automatically, so you don’t need to do anything
to use it.
When you run the tests, Nose-randomly will output the RNG seed it is using. By default it uses the epoch.
$ nosetests Using --randomly-seed=1468345257 ....
If you have a failure, you can use the
--randomly-seed option to specify the
same seed so you get the same order. Note that the seed will keep the same
order within any subset you stipulate. E.g.
nosetests --randomly-seed=10 and
nosetests --randomly-seed=10 test/test_certificate.py will both provide the
same order when running the tests in
test_certificate.py. This property is
very useful when you’ve narrowed down the issue to one particular module.
When you submit a PR, our Jenkins instance will run the tests in random order. To ensure that the same random order is used for the life of the PR, the Jenkins sets the seed to the PR number. You can see this in the Jenkins output. If you are seeing Jenkins failures that don’t appear when you run in another order, you can reproduce them by setting the seed to your PR number.
This plugin is essential for us in my opinion. It lets you run all the tests with out using your own X server, so you don’t get a bunch of dialogs popping up or failures because you don’t have DISPLAY set. Unfortunately, the plugin isn’t in PyPI and has some dependencies so you have to do a little more of work. The xorg-x11-server-Xvfb is available with no problem in Fedora. In RHEL 6, things are a little weird because the package was moved to a new channel: RHEL Server Optional. You can grab the package from there.
$ yum install xorg-x11-server-Xvfb $ nosetests --with-xvfb
And since this option is so useful, I put it in my
.noserc like so:
By default, Nose captures all the output to stdout. If your test fails, then it will show the output, otherwise it goes in the trash. We also use a plugin that captures all stderr too since the tests are very noisy otherwise. If you want to see the output anyway, use
ProTip: If you are invoking the Python debugger in code (
pdb.set_trace()), you need to run with
-s. Otherwise, Nose will intercept
all of PDB’s output and the tests will just appear to be hanging even though PDB
has started and is waiting for input.
We can add a little color to our tests with the YANC plugin which is installed with
$ sudo pip install -r dev-requirements.txt
Now you can use the –with-yanc option to get color results for your test. I find this so useful, that I just set it as
.noserc. Keep in mind that YANC and
--with-id -v are mutually exclusive
Normally Nose prints out a period for each successful test and an E or and F for errors and failures. That’s pretty nice, but if you’re short on space, you can use nose-progressive which consolidates all the output into a progress bar. Error information is printed prettily and printed in such a way that you can open your editor right to the line you need (Make sure you have $EDITOR set to what you want). This plugin works in tmux too.
$ sudo pip install nose-progressive $ nosetests --with-progressive
This plugin drops into pdb whenever there is a test with errors or failures.