Testcafe: Using "--test-meta" doesn't take the tests with meta from fixtures

[blaryjp]

Hello,

What is your Test Scenario?

    fixture('Fixture 1')
      .meta({
        type: "smoke"
      })
      .page('https://devexpress.github.io/testcafe/example/');

    test
      .meta({
        foo: 'bar'
      })('Text 1', async (t) => {
      await t.expect(true).eql(true);
    });

    test
      .meta({
        type: 'regression'
      })('Text 2', async (t) => {
      await t.expect(true).eql(true);
    });

What is the Current behavior?

If I want to launch all the tests of type "smoke" (so, using the meta "type=smoke"),
I use the CLI command:
testcafe chromium:headless tests/test.js --test-meta type=smoke

It fails with:
ERROR No tests to run. Either the test files contain no tests or the filter function is too restrictive.

If I launch the command using "--fixture-meta type=smoke", it runs successfully the two tests: but that's not what I want because the second test have "type=regression".

What is the Expected behavior?

Regarding the doc :

All metadata entries specified for a fixture are applied to tests included in this fixture.

But it seems to not works with the CLI arguments, the meta are not inherited.

Your Environment details:

• testcafe version: 1.3.1
• node.js version: 10.6.0
• command-line arguments: testcafe chromium:headless tests/test.js --test-meta type=smoke
• browser name and version: chromium:headless (65)
• platform and version: Linux Ubuntu 14.04
• other: /

[LavrovArtem]

Thank you for the detailed description. This behavior seems strange. Our team will research it and find a suitable solution.

[miherlosev]

Hi @blaryjp

It's a documentation mistake. There are two separate flags to filter tests by meta information: test-meta and fixture-meta.

@VasilyStrelyaev

Please do the following:

• remove the 'All metadata entries specified for a fixture are applied to tests included in this fixture' sentence from the Specify testing metadata paragraph.
• remove the example that shows how to override meta information. It's wrong because the fixture's meta doesn't override the test meta.

[blaryjp]

Hello @miherlosev ,
Thank you for your response.
I understand. For my use case, can we maybe create a parameter like "--meta", to run tests for both?

[LavrovArtem]

Hello @blaryjp,
We are not planning to create a parameter like --meta. We split meta information in two parts: the fixture and test. Command line parameters (--test-meta, --fixture-meta) have simple logic and check only key/value pairs. If you need more complex filtering logic (or, and, contains conditions, etc.) use the programming interface. See the example below:

const createTestCafe = require('testcafe');
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe     = tc;
        const runner = testcafe.createRunner();

        return runner
            .src(['tests/fixture1.js', 'tests/func/fixture3.js'])
            .filter((testName, fixtureName, fixturePath, testMeta, fixtureMeta) => testMeta.type === 'smoke' || fixtureMeta.type ===  'smoke')
            .browsers(['chrome', 'safari'])
            .run();
    })
    .then(failedCount => {
        console.log('Tests failed: ' + failedCount);
        testcafe.close();
    });

[lock[bot]]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.