Pytest: Using fixtures in pytest.mark.parametrize

[pytestbot]

Originally reported by: Florian Rathgeber (BitBucket: frathgeber, GitHub: frathgeber)

---

I often have a use case like the following contrived example:

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.mark.parametrize('arg', [a, b])
def test_foo(arg):
    assert len(arg) == 1

This doesn't currently do what's intended i.e. arg inside the test function is not the fixture _value_ but the fixture _function_.

I can work around it by introducing a "meta fixture", but that's rather ugly:

@pytest.fixture(params=['a', 'b'])
def arg(request, a, b):
    return {'a': a, 'b': b}[request.param]

def test_foo(arg):
    assert len(arg) == 1

It would be convenient if a syntax like in the first case was supported.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/349

[pytestbot]

_Original comment by_ Matthias Geier (BitBucket: geier, GitHub: geier):

---

This would be a great feature!

I found another (I don't know if more or less ugly) work-around:

#!python

@pytest.mark.parametrize('arg', ['a', 'b'])
def test_foo(arg, request):
    val = request.getfuncargvalue(arg)
    assert len(val) == 1

This doesn't work, however, with parametrized fixtures.

BTW, it would also be great if fixtures were supported in the params argument of pytest.fixture.

[pytestbot]

_Original comment by_ Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

Tentatively assigning this to me as I think I might be able to come up with a reasonable patch. It'll probably take me a long while though so don't let that discourage anyone else from working on this, assigning it more as a way of not forgetting about it.

[pytestbot]

_Original comment by_ Praveen Shirali (BitBucket: praveenshirali, GitHub: praveenshirali):

---

The quoted examples work because functions a and b are part of the same module as test_foo, and within the scope of the example, the parametrization should work even if @pytest.fixture decorator isn't present around functions a and b. They are getting used as regular python functions and not as pytest fixtures. Note that fixtures can also be defined in external modules like conftest.py.

Another alternative to the above example is to directly call these functions in the list.

#!python

@pytest.mark.parametrize('arg', [a(), b()])
def test_foo(arg):
    assert len(arg) == 1

[pytestbot]

_Original comment by_ BitBucket: dpwrussell, GitHub: dpwrussell:

---

This would be an awesome feature.

@praveenshirali I don't think your alternative is used as a fixture, it just calls the fixture function. So it would be run repeatedly. You would also have to specify the arguments to the fixture if there were any which could begin the cycle over again if they are also fixtures.

[mgeier]

Here's a related stackoverflow question: http://stackoverflow.com/questions/24340681/how-to-concatenate-several-parametrized-fixtures-into-a-new-fixture-in-py-test

[jlmenut]

Yes, I would like very much to have this feature also. Maybe a line in the doc explaining it's not possible for the moment would be useful also.

[kevincox]

It would also be killer if this supported parameterized fixtures generating the product of the fixtures. Although this might be a little much.

@pytest.fixture(params=["1", " ", 1, True, [None], {1:2}])
def truthy(request):
    return request.param

@pytest.fixture(params=[False, None, "", 0, [], {}])
def falsey(request):
    return request.param

@pytest.mark.parameterize("val,res", [
    (truthy, True),
    (falsey, False),
])
def test_bool(val, res)
    assert bool(val) is res

[SUNx2YCH]

+1 for this feature.
BTW, combining Florian Rathgeber and Matthias Geier solutions we can get a bit nicer "meta fixture":

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.fixture(params=['a', 'b'])
def arg(request):
    return request.getfuncargvalue(request.param)

def test_foo(arg):
    assert len(arg) == 1

[rabbbit]

+1 on this.

I'm currently writing tests that look like:

@pytest.fixture
def my_fixture():
      return 'something'

@pytest.fixture
def another_fixture(my_fixture):
      return {'my_key': my_fixture}

def yet_another_fixture():
     return {'my_key': None}

@pytest.mark.parametrize('arg1, arg2', [
    (5, another_fixture(my_fixture())),
    (5, yet_another_fixture()),
)
def my_test(arg1, arg2):
    assert function_under_test(arg2) == arg1

and that's rather ugly.

[RonnyPfannschmidt]

@rabbbit your example is structurally wrong and runs fixture code at test importation time

[rabbbit]

@RonnyPfannschmidt I know - and that's why I'd like to be able to use fixtures in parametrize? And that would be awesome.

My example is wrong, but it follows the guideline of "always use fixtures". Otherwise we'd end up with fixtures in normal tests, and workarounds in parametrized tests.

Or is there a way of achieving this already, outside of dropping parametrize and doing 'if/elses' in the test function?

[RonnyPfannschmidt]

There is a upcoming proposal wrt "merged" fixtures, there is no implementation yet

[nicoddemus]

For reference: #1660

[rabbbit]

ok, I don't understand python.

If the below works:

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

wouldn't the below be simpler? And an exact equivalent?

@pytest.fixture
def my_fixture
    return 1

@pytest.mark.parametrize('fixture', [my_fixture])
def test_me(fixture):
    assert 1 == my_fixture

Am I wrong to think that mark.parametrize could figure out whether an argument is a pytest.fixture or not?

[RonnyPfannschmidt]

atm parametrize cannot figure it, and it shouldnt figure it
there will be a new object to declare a parameter will request a fixture/fixture with parameters

some documentation for that is in the features branch

[rabbbit]

yeah, I read the proposal.

I'm just surprised you're going with pytest.fixture_request(' default_context')it feels very verbose?

after all,

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

could also turn to

@pytest.fixture
def my_fixture
    return 1

def test_me(pytest.fixture_request(my_fixture)):
    assert 1 == my_fixture

but that's not the plan, right?

On 5 July 2016 at 16:17, Ronny Pfannschmidt [email protected]
wrote:

atm parametrize cannot figure it, and it shouldnt figure it
there will be a new object to declare a parameter will request a
fixture/fixture with parameters

some documentation for that is in the features branch

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pytest-dev/pytest/issues/349#issuecomment-230490966,
or mute the thread
https://github.com/notifications/unsubscribe/AARt0kUxVSL1GMeCtU-Kzy3Pg80mW7Ouks5qSmdmgaJpZM4FEMDj
.

[RonnyPfannschmidt]

thats not even valid python syntax

the fixture-request would be used as a parameter value to tell py.test "use the value of a fixture you create later on"

there are already parametzw using examples, and its not overly verbose

[kibernick]

It would be really convenient to have this functionality, perhaps along the lines of "LazyFixture" in pytest-factoryboy

[RonnyPfannschmidt]

@kibernick we already did put a outline of possible implementations into the documentation

we just need time or a person implementing it

[Brachi]

@RonnyPfannschmidt can you link to that part of the documentation you mention? Can't find it.
Edit: nevermind. http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html

[TvoroG]

@RonnyPfannschmidt, can you please check this out https://github.com/TvoroG/pytest-fixture-mark? Need some feedback

[Brachi]

@TvoroG good work.
Currently this seems to be failing. Is it possible to support it as well?

import pytest                                                                  


@pytest.fixture(params=[1, 2, 3])                                              
def one(request):                                                              
    return str(request.param)                                                  


@pytest.fixture                                                                
def two():                                                                     
    return 4                                                                   


@pytest.fixture(params=[                                                       
    pytest.mark.fixture('one'),                                                
    pytest.mark.fixture('two')                                                 
])                                                                             
def some(request):                                                             
    return request.param                                                       


def test_func(some):                                                           
    assert some in {'1', '2', '3', 4}

[TvoroG]

@Brachi, thanks for catching it! It works now, but more nested structures need some dependency sorting to instantiate fixtures in correct order. I'll update the plugin code when i'm done.

[TvoroG]

@Brachi, I fixed it. Let me know if there is more such cases when plugin is failing

[Brachi]

@TvoroG great, thanks for the quick reply. I tested it a little more and here's another contrived example that doesn't work, based on a real use case (where one actually returns an object)

import pytest


@pytest.fixture(params=[1, 2, 3])
def one(request):
    return request.param


@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_str(request):
    return str(request.getfixturevalue('one'))


@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_hex(request):
    return hex(request.getfixturevalue('one'))


def test_as_str(as_str):
    assert as_str in {'1', '2', '3'}


def test_as_hex(as_hex):
    assert as_hex in {'0x1', '0x2', '0x3'}


# fails at setup time, with ValueError: duplicate 'one'
def test_as_hex_vs_as_str(as_str, as_hex):
    assert int(as_hex, 16) == int(as_str)

[nicoddemus]

@TvoroG, pytest-fixture-mark seems very nice! I hope I get the chance to try it soon! 😄

Perhaps discussion specific to it should be moved to https://github.com/TvoroG/pytest-fixture-mark thought? 😉

[rabbbit]

Hoping that this could get merged into pytest would be too much, right? :)

On 4 Oct 2016 23:24, "Bruno Oliveira" [email protected] wrote:

@TvoroG https://github.com/TvoroG, pytest-fixture-mark seems very nice!
I hope I get the chance to try it soon! 😄

Perhaps discussion specific to it should be moved to
https://github.com/TvoroG/pytest-fixture-mark thought? 😉

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pytest-dev/pytest/issues/349#issuecomment-251518117,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AARt0vvrzeWro3wD9-G1nKn1H89OrLCJks5qwsP8gaJpZM4FEMDj
.

[nicoddemus]

I like how it played out, but @hackebrot and @hpk42 had something else in mind during the sprint. 😁 Perhaps they can comment on this.

[Brachi]

Perhaps discussion specific to it should be moved to

I was thinking the same thing before posting, but probably the use cases and limitations are useful as well for other implementations.

[RonnyPfannschmidt]

i think its critical to make the fixture request NOT be a mark object as well

[frol]

@TvoroG I have tried your extension and it works great! Thank you!

[brianbruggeman]

I have a less contrived example....

@pytest.fixture
def today():
    import datetime as dt
    today = dt.datetime.utcnow()
    return today.strftime('%Y%m%d')


@pytest.fixture
def yesterday():
    import datetime as dt
    today = dt.datetime.utcnow()
    yesterday = today - dt.timedelta(days=1)
    return yesterday.strftime('%Y%m%d')

And I kind of expected/wanted to use them this way:

@pytest.mark.usefixtures("today", "yesterday")
@pytest.mark.parametrize("date, expected", [
    (None, False),
    (today, False),
    (yesterday, True),
])
def test_my_date_function(date, expected):
    from my_package import date_function

    assert date_function(date) == expected

In the example above, mypackage.date_function actually keys off of the current date to determine the return value. I have a few of these functions I'd like to test like the above.

[The-Compiler]

@brianbruggeman for that example, you could make them normal functions and simply call them in @pytest.mark.parametrize.

[lopter]

What about a similar example where the assertion would have to be is instead of ==?

On January 5, 2017 6:50:39 AM GMT+01:00, Florian Bruhin [e-mail] wrote:

@brianbruggeman for that example, you could make them normal functions
and simply call them in @pytest.mark.parametrize.

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/pytest-dev/pytest/issues/349#issuecomment-270569273

--
Louis Opter

[RonnyPfannschmidt]

we already have a some documented future plans at http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html

[Alex-Bogdanov]

Sorry, when this 2 features will be released?http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html
pytest.define_combined_fixture()
pytest.fixture_request()

[The-Compiler]

@Alex-Bogdanov it's nothing more than an idea/proposal currently.

[Alex-Bogdanov]

@The-Compiler any ideas how to push forward it? cause it would be very helpful feature

[nicoddemus]

@Alex-Bogdanov the best way would be to start a PR yourself; AFAIK all maintainers are working in other features/bug fixes for 3.1.

[Alex-Bogdanov]

@nicoddemus Got you. Starting

[binary10]

I had to find a workaround for this issue in order to share a resource in my module. The idea is to dynamically generate a list from which test cases are then generated. The following things didn't work:

• Using pytest fixtures and providing the fixture name to pytest.mark.parametrize (original issue description)
• Calling the fixture function in pytest.mark.parametrize (won't work for fixtures that call other fixtures)
• Setting up a variable in setup_module, then calling it in pytest.mark.parametrize (it appears the decorator is called before the module is initialized)
• Using conftest.py and metafunc (too convoluted and meta)

What worked:
The simplest approach I found is to forget using pytest fixtures at all and to create a dumb "fixture". I simply created a "constant" variable (global to the module) for my resource. I can now pass it to the decorator without issues. I lose the smart aspects of pytest fixtures (like collecting tests that relate to that fixture), but getting the test output I need is more important than this.

To only support fixtures in the test body seems like a big oversight. It should be possible to use fixtures anywhere. Wouldn't fixtures need to be initialized before the tests are collected anyway? If this isn't the case then why?

[phobson]

Here's how I worked around this:

class Thing:
    def __init__(self, datadir, errorfile):
        self.datadir = datadir
        self.errorfile = errorfile


@pytest.fixture(params=['test1.log', 'test2.log'])
def thing(request):
    with tempfile.TemporaryDirectory() as datadir:
        errorfile = os.path.join(datadir, request.param)
        yield Thing(datadir=datadir, errorfile=errorfile)


def test_thing_datadir(thing):
    assert os.path.exists(thing.datadir)

[Sup3rGeo]

Related to this may be my question on stackoverflow.

Basically it would be nice to parametrize depending on a fixture like this:

datasetA = [data1_a, data2_a, data3_a]
datasetB = [data1_b, data2_b, data3_b]

@pytest.fixture(autouse=True, scope="module", params=[datasetA, datasetB])
def setup(dataset):
    #do setup (only once per dataset)
    yield
    #finalize

#dataset should be the same instantiated for the setup
@pytest.mark.parametrize('data', [data for data in dataset]) 
def test_data(data):
    #do test

It doesn't seem to be possible to do this as of today and I think it is related to parametrizing on fixtures too.

[RonnyPfannschmidt]

@Sup3rGeo if the dataset size is always the same you can do something like have a independent parameter thats just the index of the element

you can even account for different sizes by going for the max size and skipping the missing elements if the cost is manageable

[offbyone]

I'm looking at something that I ... think is related to this? What I want to do is this:

@pytest.fixture(scope='session')
def bootstrap():
    # do something super slow

@pytest.fixture
def config_dir(bootstrap, tmpdir):
    # do something faster with the bootstrap data

def pytest_generate_tests(metafunc):
    # generate some tests based off the current value of the `config_dir` fixture.

I don't see anything in here that is quite what I'm looking for, but I'm open to hearing otherwise.

[ndevenish]

My use case is: I want to parametrize on every file in a regression data folder. Except this folder can move, so the location is (usually) passed to tests as a fixture. Contrived example of what I'm currently doing to work around this:

@pytest.fixture
def regression_data():
  return "some_folder_location"

# Generate list of all files in the regression folder
all_files = [os.path.join(regression_data(), x) for x in os.listdir(regression_data())]

# Convert each parameter value into a fixture
@pytest.fixture(params=all_files)
def regression_image(request):
  return request.param

def test_every_file(regression_image):
  # Test the single image regression_image

e.g. I have to either duplicate code or from ..conftest import regression_data or similar to share the logic for location.

What i feel would be ideal is if I could pass through the parametrization in the fixture (you can send it in, but not out), where other fixtures are available. Perhaps e.g. something like

@pytest.fixture
def regression_image(regression_data):
  all_files = [] #Generate list of files based on regression_data
  return pytest.parametrize(params=all_files) 

or even, to borrow the initial example:

@pytest.fixture
def a():
  pass

@pytest.fixture
def b():
  pass

@pytest.fixture
def regression_image(request, a, b):
  return pytest.parametrize(params=[a, b]) 

or maybe even aggregating other parametrizations?

@pytest.fixture(params=["fileA", "fileB"])
def example_file(location, param):
  return pytest.parametrize(params=[os.path.join(location, param)])

I know people seem to be leaning towards the (with implementation) lazy-fixtures approach, but would this even be possible, or is there e.g. some ordering problem where parameters are resolved before fixtures, or something similar.

[nicoddemus]

Hi @ndevenish,

Your examples are look nice and well written, thank you.

Under the current design, pytest needs to generate all items during the collection stage. Your examples generate new items during the session testing stage (during fixture instantiation), which is not currently possible.

[nluetzge]

Hi!
Is there any development in this issue? I would really love to be able to use fixtures in the parametrization. At the moment it is not possible and the workarounds are not optimal.

Cheers,
Nora

[goodusermf]

I have a not so pretty work around that works for me; hope someone finds it useful.

class MyTests():
     @staticmethod
     def get_data()
           items_list = ["a", "b", "c", "d"]
            return items_list

      @pytest.mark.parametrize('letter', get_data.__func__())
      def test_my_test(letter)
           # consume letter in your test

[dtomas]

Possible workaround, involving indirect parametrization and passing pytest.mark.usefixtures() to pytest.param via its marks argument:

import pytest

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.fixture
def arg(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('arg', [
    pytest.param('a', marks=pytest.mark.usefixtures('a')),
    pytest.param('b', marks=pytest.mark.usefixtures('b')),
], indirect=True)
def test_foo(arg):
    assert len(arg) == 1

Tested with pytest 3.2.5 and 3.8.0.
Can some pytest dev confirm that this is supposed to work (and continue to work), or are there some issues one could run into further down that road?

[RonnyPfannschmidt]

@dtomas at first glance this workaround looks reasonably safe, however due to the inner details i wont make promises

but i do believe it unlikely to break - we'd have to find a very good reason to do that

[dtomas]

Sorry, my suggested workaround does not really work (or work as I thought it would). It just appears to work, because the fixture function is called using request.getfixturevalue(). The usefixtures() marks have no effect here and could as well have been omitted. So the following example does not work:

import pytest

@pytest.fixture
def c():
    return []

@pytest.fixture
def a(c):
    c.append('a')

@pytest.fixture
def b(c):
    c.append('b')

@pytest.mark.parametrize('expected', [
    pytest.param(['a'], marks=pytest.mark.usefixtures('a'), id='a'),
    pytest.param(['b'], marks=pytest.mark.usefixtures('b'), id='b'),
])
def test_foo(c, expected):
    assert c == expected

I don't know if this is expected behavior, or how hard it would be to support the usefixtures mark in pytest.param, or if that would even be possible.

[dtomas]

https://github.com/dtomas/pytest/commit/20c98040993c3f07abfb5eb0d930ebd0f0e40761 makes this work (using usefixtures as a mark in pytest.param), though I'm not sure what unintended side-effects it may cause.

[RonnyPfannschmidt]

there must be a completely ludicrous bug at some other place if it cant be passed in as marker that way ^^

the location that fix is in however is not the correct place to fix this

[dtomas]

I've looked into this a bit, and it seems to have something to do with the Function items for the parametrized functions sharing a FuncFixtureInfo object
(see PyCollector._genfunctions() at https://github.com/pytest-dev/pytest/blob/3.9.0/src/_pytest/python.py#L418-L421), so the fixtures of the usefixtures marks of the parametrized Function items are missing from their _fixtureinfo and fixturenames.

[RonnyPfannschmidt]

@dtomas great job at researching - also omg that one will be a total pain to resolve

[dtomas]

I've created a new issue for the usefixtures bug: https://github.com/pytest-dev/pytest/issues/4112

[jraygauthier]

We will need to add a new scope for fixtures then: @fixture(scope="param") (there might be a better name for this tough). This is so that the fixture can be initialized / de-initialized on a per case/param basis.

It should be the default in my opinion as it might be essential that some state are restored in the fixture before moving the next param. Otherwise, one might be quite surprised when using this new feature.

[RonnyPfannschmidt]

@jraygauthier what you just described would be a completely different kind of scope that's absolutely mismatched to current fixtures - in which a parameter outlines it users

netiher having it spelled like the current system nor having it as default would be desirable

[jraygauthier]

Did you mean "outlives" by "outlines" ?

If so, the user of the fixture(s) in this case would be a given pytest::param entry taken as a parameter by @pytest.mark.parametrize. I would expect by default that my fixture's lifespan exeed that on one of these entry but does not encompass 2 of these.

Would the current "function" scope encompass all the pytest::param entries or a single of these?

[RonnyPfannschmidt]

@jraygauthier i don't follow, please elaborate an example

i did indeed mean outlives

[kielpins]

+1 on this issue, it is core functionality for anyone doing tests of numerical code.

[wjordan213]

+1 on this issue as well, accessing a fixture in pytest.parametrize would be amazing for some tests I'm currently writing.

[wjordan213]

did https://docs.pytest.org/en/latest/proposals/parametrize_with_fixtures.html ever get anywhere?

[brunoais]

@wjordan213 You can solve that kind of thing with current API.
If you use indirect and then using request.getfixturevalue() to run the right fixture.
You made an indirect to the target fixture to parametrize it and then an indirect to another fixture to tell which fixture to call for testing.

[kielpins]

@brunoais Would you be willing to provide a minimal code example? This thread has been going for a long time without resolution, it would be great to have a clear and straightforward solution that everyone can use. Thanks!

[brunoais]

I'll have to build it myself so it will take a while and I'll need to find time in my schedule because it would be my first time calling fixtures dynamically

[dmtucker]

Something like this?

import pytest

@pytest.fixture
def fixture1():
    return 1

@pytest.fixture
def fixture2():
    return 2

@pytest.fixture
def x(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('x', ['fixture1', 'fixture2'], indirect=['x'])
def test_indirect(x):
    assert 0 < x < 3

[brunoais]

From what I understood, there's more missing when parameterization is needed....
From your code, I can make an untested version.

import pytest

@pytest.fixture
def fixture1(request):
    return request.param * 2

@pytest.fixture
def fixture2(request):
    return request.param * 4

@pytest.fixture
def x(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('x,fixture1,fixture2', [('fixture1',1,0), ('fixture2',0,4)], indirect=['x','fixture1','fixture2'])
def test_indirect(x):
    assert 0 < x < 5

Thank you dmtucker for starting this.

[dmtucker]

Eh, looks like that doesn't quite work...

In test_indirect: function uses no fixture 'fixture1'

I thought this could be acheived by parametrizing the fixtures themselves, but that doesn't appear to work either:

import pytest

@pytest.fixture(params=[1, 0])
def fixture1(request):
    return request.param * 2

@pytest.fixture(params=[0, 4])
def fixture2(request):
    return request.param * 4

@pytest.fixture
def x(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('x', ['fixture1', 'fixture2'], indirect=['x'])
def test_indirect(x):
    assert 0 < x < 5

================================================= ERRORS =================================================
_______________________________ ERROR at setup of test_indirect[fixture1] ________________________________
The requested fixture has no parameter defined for test:
    test_indirect3.py::test_indirect[fixture1]

Requested fixture 'fixture1' defined in:
test_indirect3.py:4

Requested here:
test_indirect3.py:13
_______________________________ ERROR at setup of test_indirect[fixture2] ________________________________
The requested fixture has no parameter defined for test:
    test_indirect3.py::test_indirect[fixture2]

Requested fixture 'fixture2' defined in:
test_indirect3.py:8

Requested here:
test_indirect3.py:13
======================================== short test summary info =========================================
ERROR test_indirect3.py::test_indirect[fixture1]
ERROR test_indirect3.py::test_indirect[fixture2]
======================================== 2 error in 0.10 seconds =========================================

[brunoais]

Does it work like this? (changed first param of parametrize)

import pytest

@pytest.fixture
def fixture1(request):
    return request.param * 2

@pytest.fixture
def fixture2(request):
    return request.param * 4

@pytest.fixture
def x(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('x', [('fixture1',1,0), ('fixture2',0,4)], indirect=['x','fixture1','fixture2'])
def test_indirect(x):
    assert 0 < x < 5

[RonnyPfannschmidt]

@brunoais i believe both attribute names need to be put in the argument name list, and only the values need to be in the parameter list

[CAM-Gerlach]

@RonnyPfannschmidt This is the most-commented open issue on the Pytest Github, and the second-most commented of any issue, open or closed (and that by a narrow margin), yet there appears to be no serious effort to implemented a proper solution aside from ugly, unofficial, user-created workarounds involving multiple layers of other fixtures, or simply not using fixtures for such altogether.

Obviously, the dev team has limited resources, but given dozens of other PRs are getting merged every month, is there a reason addressing this still isn't a priority for pytest? Is this not seen as a valid use case for fixtures and/or pytest.parameterize?

[RonnyPfannschmidt]

@CAM-Gerlach its a valid use-case, but not a priority for me personally - there is various really nasty details in the fixture system that will take a while to correctly figure (like for example covariant parameterize), layering of dependencies and other issues before we can hope to make more than a implementation that will simply break down once a requested fixture is parameterized

its not like we don't want this to happen, but with our personal and mental resources we have to be realistic - if we cant hope to make it work well enough we don't start, what we put into pytest should beat the external plugins/workaroudns by a order of magnitude - we currently cant do that so the workarounds are already enough

personally id rather do this never than just pulling a nasty incomplete workaround into pytest we will then regret for the next 5-10 years

[carver]

There are a bunch of different proposed workarounds in the thread. Many of them have bugs, or follow-up comments that are hard for people to quickly compile into a single working example.

Maybe someone on the team could do a writeup in the docs with a "blessed" workaround (and mention some limitations of the approach, like you did here).

:heart: for all your work

[RonnyPfannschmidt]

personally i consider https://pypi.org/project/pytest-lazy-fixture/ a valid workaround (with the caveat that it currently doesn't handle parameterization - its also closest to what we consider pulling into core

my personal plan to drive it forward is to enable the pytest internals to provide whats needed to enable parameterize support in that, and from there expanding towards working on a inclusion

[Corleo]

I can use fixtures as arguments in parametrize using the getfixture fixture that returns a function that calls another fixture:

import pytest

@pytest.fixture
def a():
    return 'a123'

@pytest.fixture
def b():
    return 'b321'

@pytest.fixture
def getfixture(request):
    def _getfixture(name):
        return request.getfixturevalue(name)
    return _getfixture

@pytest.mark.parametrize(
    '_fixture, expect', [
        pytest.param('a', 'a123'),
        pytest.param('b', 'b321'),
    ]
)
def test_foo(getfixture, _fixture, expected):
    assert getfixture(_fixture) == expected

[smarie]

My two cents: it seems to me that there are two core concepts around fixtures and parametrization that people want from pytest. This post refers to the second but it is good to look at both for consistency of the architecture.

feature 1: cartesian product of parametrized fixtures each test is run for each combination of parameters present in the set of fixtures it relies upon. Note: the @pytest.mark.parametrize on the test function itself can be considered "just" dummy function-scoped fixtures here ; actually in another post we suggested to entirely implement the concept of test function parameters by function-scope fixture, so as to simplify things to a single concept: "(parametrized) fixture".

Feature 1 is today correctly provided by pytest, and in pytest-cases we even proposed a way to transform all parameter marks into fixtures, so as to begin the transition to this simpler system.

Feature 2: aggregation/union of parametrized fixtures this is what this post seems to be about: the possibility to combine two or more fixtures into a single one, by union of all their instantiated (parametrized) values. This is what I have started to provide in pytest-cases with @cases_data but it is focused on sourcing parameter values from several parameter functions, not to union fixture values from several fixtures. I think that it is still useful, but it does not answer the main need from this post. I'll have a look today, to see if I can provide a workaround.

Edit: I'm wondering if the second feature is actually a superset of the OP's... I might have been mistaken here. Anyway I'll continue to work on it to see if it makes sense.

Edit 2 have a look here it brings some funny cases for fixture unions + parameters on the table... ideas more than welcome to find the "correct desired" behaviour to implement. (@RonnyPfannschmidt @nicoddemus ?)

[toonarmycaptain]

My use case (please point it out if I'm off topic) is:

@pytest.fixture()
def test_my_custom_class():
test_name = 'Arthur, King of the Britons'
yield my_custom_class(test_name)

@pytest.mark.parametrize(
'argument, result',
[test_my_custom_class, 'some evaluated value',
'not my custom class', 'something else',
])

def test_my_class(self, argument, result):
    assert some_func(argument) == result

Is this what's so hard to figure out because of fixture/parametrize decorator magic, or is there a simple alternative to what I want to do, which is simply use a fixture (or preferably several different fixtures for different testing cases) returning an example of my class as a parameter to a test.

[smarie]

From what @RonnyPfannschmidt explained, the issue is to control what happens in case of parametrization. This case basically is a specific case of "fixture unions" where a fixture (here your parameter) depends on the union of some other fixtures. In your example: what happens when test_my_custom_class has a parameter ? Would 'some evaluated value' be the expected outcome for each parameterized value ? And of course all the ids generation is impacted too, if you do not provide an explicit ids list.

If you do not need parametrization, then as suggested above the pytest-lazy-fixture plugin should solve your problem, from what I understand (I did not test though)

[smarie]

For those of you who wish to try, there is now a draft implementation of fixture union available in pytest-cases 1.6.0. It matches the scope request for proposal, so you'll have to wait a bit more for the support inside pytest.mark.parametrize. But that will be quite easy now that the callspec engine is modified (in this draft) to accept both unions and cartesian products.

[smarie]

Update on this topic: this feature is now available in the new release of pytest-cases. (I remind that pytest-lazy-fixture does not handle the case where referenced fixtures are parametrized, so if you need this, use pytest-cases instead)

It can handle ridiculously complex cases where several references are used from several places. See for example this test.

The principle of the solution to handle union fixtures (and therefore fixture references in parameters) was to replace the concept of "fixture closure" (=set of all fixtures required at a node) with a "fixture closure tree". In other words, to describe all the alternate branches and their (different) local closures. It has been tested on a couple of nasty examples such as this one.

Please play with it if you're interested, and if the global behaviour seems appropriate I believe that it could constitute a solid foundation for a PR, replacing the getfixtureclosure and parametrize functions. Note that it would really help that parameters be considered as "private-scoped" fixtures, rather than being "discarded arg names" (the new ignore_args input to getfixtureclosure that appeared in 4.6+).

EDIT: I updated the URLs, they were obsolete, and added an explicit mention of the difference with pytest-lazy-fixture so that this post can be read standalone and used as a reference.

[smarie]

Quick update: I just pushed a 1.8.0 of pytest-cases. The main difference is that there is now by default an explicit style for the ids corresponding to alternatives in a union. Alternate styles are also proposed, see fixture_union API doc.

[slavaatsig]

Any thought about the lazy load approach from here https://github.com/ionelmc/python-lazy-object-proxy
Worked flawlessly for me in parametrize for fixture that needed to messup with database.

[last-partizan]

@smarie Thank you for pytest-cases, i've managed to get what i wanted, and it looks almost fine, except for fixture names :)

import pytest
from pytest_lambda import lambda_fixture
from pytest_cases import fixture_ref, pytest_parametrize_plus

from . import models


def lazy_fixture_generator(mod):
    """
    Lazy = lazy_fixture_generator(globals())
    """
    ids = (x for x in range(1000))

    def Lazy(fun):
        """
        Lazy fixture loading from lambda

        Usage:
        @pytest_parametrize_plus("args", [Lazy(lambda f1, f2: (f1.atrr, f2.attr))])
        """
        name = f"lazy{next(ids)}"
        mod[name] = lambda_fixture(fun)
        return fixture_ref(name)
    return Lazy


Lazy = lazy_fixture_generator(globals())


@pytest.fixture
def book1():
    return models.Book(name="Moby Dick")


@pytest.fixture
def book2():
    return models.Book(name="Alice in Wonderland")


@pytest_parametrize_plus("name, expects", [
    (Lazy(lambda book1: book1.name), "Moby Dick"),
    (Lazy(lambda book2: book2.name), "Alice in Wonderland"),
])
@pytest.mark.django_db
def test_get_or_create_book(name, expects):
    book = models.Book.objects.get_or_create(name=name)[0]
    assert book.name == expects

test output

 pipenv run pytest -vv --disable-warnings
Test session starts (platform: linux, Python 3.8.0, pytest 4.5.0, pytest-sugar 0.9.2)
cachedir: .pytest_cache
Django settings: bookstore.settings (from ini file)
rootdir: /home/serg/work/git/example/src/bookstore, inifile: pytest.ini
plugins: django-3.7.0, sugar-0.9.2, lambda-1.1.0, cases-1.11.9
collecting ... 
 core/tests.py::test_get_or_create_book[test_get_or_create_book_name_expects_is_fixtureproduct__0] ✓50% █████     
 core/tests.py::test_get_or_create_book[test_get_or_create_book_name_expects_is_fixtureproduct__1] ✓100% ██████████

Results (0.45s):
       2 passed

[smarie]

@last-partizan I opened an issue concerning your comment in pytest-cases: https://github.com/smarie/python-pytest-cases/issues/69

Feel free to discuss it there

[smarie]

For information thanks to @last-partizan pytest-cases 1.12.0 has much more readable test ids when a fixture_ref() is used inside a @pytest_parametrize_plus (and therefore when fixture unions are created behind the scenes). Feel free to test it !

Note: when we'll collectively feel that this reference implementation is mature enough, it will probably be time to try to migrate it to pytest core. Indeed manipulation of the fixture closures is something too low-level to stay in a plugin for ever : sooner or later it will break. Somethink to think about for 2020 ? :)

[andreabisello]

sorry i'm not able to understand.
can i @parametrize test reading data from a fixture?
i would like to read an array of data from a json with a fixture and then parametrize a test in order to generate a test for every element of the list, but it looks like i cannot use a fixture as argsvalues of https://docs.pytest.org/en/latest/reference.html#pytest-mark-parametrize-ref
thanks sorry if i'm off topic.

[smarie]

@andreabisello , please check first this post : https://stackoverflow.com/questions/42228895/how-to-parametrize-a-pytest-fixture/56871701#56871701

It clarifies the role and responsibilities of fixtures and parameters.

Then fixture_ref can be used if you wish to "reuse" a (parametrized or not) fixture in a list of parameters. Nothing "more", really.

[Lewiscowles1986]

Would the alternative not be making a factory fixture, that uses other fixtures and based on a simple parameter returns the fixture needed? or is this also inferring that that would have problems with mutually exclusive fixtures (which seems like an issue of poor design).

It is a lot of work, when a fixture could hashmap an internal key to a value of many fixtures it uses internally. It's then down to those individual fixtures to defer or launch on-call or by default. An inner method would give access to the fixtures.

If you wanted an immediate-mode fixture and a deferred, write the deferred one, then have the immediate one use that fixture and call it.

@pytest.fixture
def a():
    return "AdminUser"


@pytest.fixture
def b():
    return "GuestUser"


@pytest.fixture
def c():
    return "SupplierUser"


@pytest.fixture
def a_or_b_or_c(a, b, c):
    mapping = {
        "A": a,
        "B": b,
        "C": c,
    }

    def inner(fixture_identifier):
        answer = mapping.get(fixture_identifier)
        if not answer:
            raise ValueError("Invalid Input")
        return answer
    return inner


@pytest.mark.parametrize('fixture_identifier', ['a', 'b'])
def test_some_thing(a_or_b_or_c, fixture_identifier):
    data = a_or_b_or_c(fixture_identifier)
    # do something

What am I not understanding about this problem that it's been standing for nearly 7 years?

[RonnyPfannschmidt]

Right now it's trivial doable for non-parameterized fixtures but it requires covariant parameterization for parameterized fixtures

That one is something really tricky, and the internals of the fixture system currently make it harder

There easily is quit the amount of hard work and a number of refactoring missing before we can hope to do it

It's certainly not something doable easily with the currently available volunteer time

And it's also not well defined enough to simply contract out part of that work to well established community members

[toonarmycaptain]

Question: why not make it work for non-parameterized fixtures in the shorter term? I'm sure there are many use cases not needing parameterization, and a lot that do which frequently use particular parameters such that creating specific fixtures for those common cases isn't an undue burden (particularly over crafting every case by hand) eg my_parameterized_fixture_case_one, my_parameterized_fixture_case_two, my_parameterized_fixture_case_three each calling my_parameterized_fixture with 1, 2, 3 as parameters?

This would at least allow a larger portion of testing following this pattern to be DRY, even if it conveys meaning via naming rather than explicit args. My use case would be a couple of database backends, plus those in different states, (empty, full, plus a couple of other states), so...empty_sqlite, full_sqlite, empty_mysql, full_msql, empty_postgresql, full_postgresql fixtures would work for me, rather than building up each case within the test, or providing many fixtures as arguments with the one used chosen by a text parameter, incurring the full cost of loading all of them for every test.

[Lewiscowles1986]

@RonnyPfannschmidt I apologize if you took my words as an attack. They were not meant as such. Pytest is very awesome.

[RonnyPfannschmidt]

@Lewiscowles1986 I did not take your message as attack, i believe i just stated the issues surrounding the problem

@toonarmycaptain personally I haven't gotten into that because getfixturevalue works for that use case already, same goes for @smarie made pytest-cases (if i recall correctly)

And it safes a support burden, if we only implement the trial case the feature will ruin the day off many users running into that limit and we will have to tell them "sorry we only have it halfway"

The trivial case already has trivial workarounds /3rd party helpers

The prospect of shipping a feature that is likely severely limited for potentially years to come is not exactly attractive

[toonarmycaptain]

@RonnyPfannschmidt Interesting, I didn't know about getfixturevalue, thanks, I'll look into it. EDIT: That vastly simplified my setup, thankyou.

With respect to "sorry we only have it halfway" - that's already how the idea of fixtures that can only be passed as direct test arguments, and not via parametrize feels to me. Admittedly I haven't been around for many years of pytest dev, but it makes sense to me as a user that anything that can be passed into a test as an argument should be able to be passed via parametrize, whose function I understand as passing a variety of different values as arguments to a test.
I do understand the issue of volunteer time. If there was a roadmap with tasks set out, I'd be willing to contribute to the best of my ability. I'd imagine more people would be interested if the answer to "why doesn't it work for parametrized fixtures" was "not yet but here's the list of tasks, please help us!" But maybe I'm naiive.

[RonnyPfannschmidt]

Parametrize is a mechanism to declare parameter sets for tests

Fixtures consume those

For parameters without fixture to consume them, a fixture is made up

It's that way because of history (funcarg hooks and generate tests where very different in the early stages, combinatorial parameterization was easily added, however covariant parameterization is rather hard,

[smarie]

@RonnyPfannschmidt , @toonarmycaptain , @Lewiscowles1986

Right now it's trivial doable for non-parameterized fixtures but it requires covariant parameterization for parameterized fixtures
That one is something really tricky, and the internals of the fixture system currently make it harder
There easily is quit the amount of hard work and a number of refactoring missing before we can hope to do it
It's certainly not something doable easily with the currently available volunteer time

This feature is implemented in pytest-cases since april 2019 and it has been working fairly well since. Recent move to v2 has clarified a little bit the mechanisms that are needed and have been created. See this detailed explanation page.

When I wrote the page, that made me realize that I could simplify even more the implementation in the plugin. When this is done, maybe we'll be able to discuss whether this is something that should be migrated into pytest ? Until then feel free to experiment and post feedback in pytest-cases.

[RonnyPfannschmidt]

Pytest cases looks nicely along the road we also want to have in pytest core

Im still heavily invested in the idea to clean up fixtures internals before integration of your good work to core

[Lewiscowles1986]

I'll try to grab some time to play with that. Thanks so much @smarie and the pytest team.

[smarie]

Thanks @RonnyPfannschmidt for your kind words. Sure, the cleaner the internals are, the easier evolutions can be made ! @Lewiscowles1986 you're welcome ;)

[tim-oh]

Does pytest-lazy-fixture provide a solution to the original issue?

[The-Compiler]

@tim-oh See https://github.com/pytest-dev/pytest/issues/349#issuecomment-457355792 and https://github.com/pytest-dev/pytest/issues/349#issuecomment-478915992

[smarie]

@tim-oh : see https://github.com/pytest-dev/pytest/issues/349#issuecomment-501796965 , I updated it a bit so that it contains all information at once.

[tim-oh]

Thanks @The-Compiler @smarie

TLDR; pytest-lazy-fixture works for un-parametrised fixtures (it did the job nicely for me) whereas pytest-cases can handle more complex scenarios.

[jxramos]

Looking forward a bit, when the day comes where we can parametrize over fixtures as parameter values there should be a means to inject an id to those fixtures at construction time so we don't have to repeat id definitions wherever those fixtures get used as parameters. Something like

@pytest.fixture(id="MySuperFixturePresentationString")
def my_super_fixture():
     return "foo"

That way we separate the presentation representation of the fixture from its usage in source. With that we could handily filter by pytest keyword expressions on the fixture id alone...

pytest -k MySuperFixturePresentationString