py.test with xdist is not executing tests parametr

问题:

Does anybody noticed the following strange behaviour for pytest and xdist.

When trying to run the test that is parametrized with some randomly selected values the test are not actualy run. The same test is executed without any problems if xdist is not used.

Following code can be used to reproduce this.

import pytest
import random

PARAMS_NUMBER = 3
PARAMS = []

for i in range(PARAMS_NUMBER):
    PARAMS.append(random.randrange(0, 1000))

@pytest.mark.parametrize('rand_par', PARAMS)
def test_random_param(rand_par):

    assert 500 > rand_par

Without xdists it works fine.

With xdist no test is executed at all with the following output

============================= test session starts =============================
platform win32 -- Python 2.7.3 -- py-1.4.24 -- pytest-2.6.2
plugins: xdist
gw0 [3] / gw1 [3] / gw2 [3] / gw3 [3]
scheduling tests via LoadScheduling

==============================  in 1.93 seconds ===============================

Versions I'm using:

• python 2.7.3
• pytest 2.6.2
• pytest-xdist 1.11

Additional note:

With some older versions (xdist 1.8 and pytest 2.4.X or 2.5.X do not remember exactly) the xdist was stopping on assertion in dsession.py

assert collection == col

Thanks in advance for any help how to solve it or at least workaround it :)

回答1:

A workaround would be to parametrize with reproducible (but meaningless) values, like all numbers between 0 and PARAMS_NUMBER-1. Then each test can individually pick a random value when it runs. In order to know which random value it picked (to reproduce in case of crash), each test should print it first. At least that's what I do with some non-xdist tests; I hope it works too with xdist, i.e. the prints are correctly propagated to the parent process and only shown if the individual test fails (or py.test is run with -s).

回答2:

So here is the code after Armins hint :)

import pytest
import random

PARAMS_NUMBER = 3
PARAMS = []

for i in range(PARAMS_NUMBER):
    PARAMS.append(1000)

@pytest.mark.parametrize('rand_par', PARAMS)
def test_random_param(rand_par):

    par_val = random.randrange(0, rand_par)
    assert 500 > par_val

And it runs test 3 times with randomly selected value.

Update: I have created an issue for xdist project and it is resolved in the meaning of returning reasonable info for user.

More info can be found here py.test with xdist is not executing tests parametrized with random values