Testcafe: We need to figure out `test` extensions syntax

[inikulin]

Some planned features require the way to attach methods to test, e.g. before or skip. For skip many frameworks use test postfix:

test.skip('Some name', t=> {..});

This approach is quite ergonomic since you don't need to modify test code much: just add dot and skip. But the problem is that our directive may have parameters, e.g. before. skip can also have parameter to specify predicate that will allow you to choose in which browsers you need to skip tests. I've tried some variations, but none of them satisfies me:

1. Move body afterwards

test('Some test')
   .before(async t => {...})
   .skip(browserInfo => {...})
   .fn(async t => {
        // test code here
    });

_Pros:_ Looks OK, kinda...
_Cons_: If you have existing test you need a lot of modifications to the existing test code. fn looks a little bit akward.

2. Directives afterwards

test('Some test', async t => {
        // test code here
    })
   .before(async t => {...})
   .skip(browserInfo => {...});

_Pros:_ Minimal modifications to the existing code to add directive
_Cons:_ Prerequisites of the test goes after the test.

3. Options object

test('Some test', {
    before: async t => {...},
    skip: browserInfo => {...}
}, async t => {
   // test code here
});

_Pros:_ Minimal modifications to the existing code to add directive. Prerequisites goes before test.
_Cons:_ Looks a little bit weird. Inconsistent with fixture directive where we have chained syntax.

If you have any other ideas, please, share them.

\cc @DevExpress/testcafe

[VasilyStrelyaev]

fn looks a little bit akward.

last time we called it body

[inikulin]

@VasilyStrelyaev fn is shorter and named after the parameter we have currently

[kirovboris]

The fourth option we can add t.skip action.

test('Some test', async t => {
    t.skip(browserInfo => {...})

    // test code here
});

I like the second option because it doesn't complicating the test reading.

[inikulin]

@kirovboris We can't add directive in body, since we already executing the test.

[helen-dikareva]

I think first variant looks better than others, but where will be after after fn or before?

[inikulin]

but where will be after after fn or before

It doesn't matter, you can use the way you prefer

[helen-dikareva]

can we use someting more descriptive than fn?
like just variant like in mocha:

it('Some test')
   .before(async t => {...})
   .skip(browserInfo => {...})
   .test(async t => {
        // test code here
    });

Or may be do

test('Some test')
   .before(async t => {...})
   .skip(browserInfo => {...})
   .do(async t => {
        // test code here
    });

or may be smtng better.

[inikulin]

@helen-dikareva https://github.com/DevExpress/testcafe/issues/802#issuecomment-246989277

We need to keep it somehow consistent with the existing API.

[inikulin]

One more

test
  .before(async t => {...})
  .skip(browserInfo => {...})
  ('Some test', async t => {
      // test code here
  });

[churkin]

test('Some test')
   .before(async t => {...})
   .skip(browserInfo => {...})
   (async t => {
        // test code here
   });

[inikulin]

https://github.com/DevExpress/testcafe/issues/802#issuecomment-247000714

This one looks kinda good, but have problem with easy code modification

[churkin]

@inikulin The last variant - the best

[inikulin]

@churkin Bad thing about it is that title is separated from test directive

[AndreyBelym]

mocha allows skip usage in the test body. When you use this.skip() inside a test, it throws a special exception, that aborts the test.

Also, in mocha you do not declare before and after directly in the it test block, you use another describe block.

So what's about

context('context', () => { //maybe allow to omit the context name
   before(async t => {});
   after(async t => {});
   skip(...args => {}); 
   only(...args => {});   

   test('test 1' , async t => {});
})

but it doesn't have the fancy chained look.

For chaining, I really like the @churkin's variant. If it is hard to implement, the first one from https://github.com/DevExpress/testcafe/issues/802#issue-176882339 looks very good.

PS. Can we use something like that

test('The test') (
    async t => {}
);

for simple tests in order to add modifiers easily afterwards? You just need to add your modifier before the second opening bracket.

[AndreyBelym]

Or maybe even

test `Test` (async t => {
})

[inikulin]

PS. Can we use something like that

It looks awkward without modifiers.

mocha allows skip usage in the test body. When you use this.skip() inside a test, it throws a special exception, that aborts the test.

It's an option, but it's inconsistent and it's easy to shoot yourself in the leg with that approach.

Also, in mocha you do not declare before and after directly in the it test block, you use another describe block.

Yeah, but we don't have nested fixtures yet. Also from my own experience you don't need a new fixture for some tests, you just need some test-specific teardown code and introduction of new fixture feels awkward in that case.

[inikulin]

https://github.com/DevExpress/testcafe/issues/802#issuecomment-249877463 - this one looks interesting

[inikulin]

import { expect } from 'chai';

fixture `Click`
    .page `http://localhost:3000/fixtures/api/es-next/click/pages/select.html`;


test `Click on an "option" element` (async t => {
    await t
        .click('#simple-select')
        .click('[value=Second]');

    var select = await t.select('#simple-select');

    expect(select.value).eql('Second');
});


test `Click on an invisible "option" element` (async t => {
    await t
        .click('[value=Second]');
});

Nice! But should we deprecate old API in that case? Or just force usage of the new one and change all examples in documentation to this one.
\cc @DevExpress/testcafe

[inikulin]

Hmm, unfortunately with directives this looks a bit awkward:

test `Click on an invisible "option" element`
    .before(async t => {})
    .skip(browserInfo => {})
    (async t => {
        await t
            .click('[value=Second]');
    });

[miherlosev]

I like this variant:

fixture `Click`
         .page `http://example.com`.
         .before(async t => {})
         .after(async t => {});

test `click submit button`.
     .skip(browserInfo => {})
     .before(async t => {})
     .do(async t = {}) //maybe another word
    .after(async t => {})

Proc:

• consistent api for fixture and test
• minimum code to skip test
• at once I understand where test body is placed

[inikulin]

@miherlosev How test will look like without directives?

[miherlosev]

test `click submit button`.
     .do(async t = {}) //.run or .body or .perform etc.

versus

test `Click on an invisible "option" element` 
    (async t => {

     });

[inikulin]

test `click submit button`.
     .do(async t = {}) //.run or .body or .perform etc.

Unfortunately it's too excessive for basic case

[miherlosev]

Unfortunately it's too excessive for basic case

Absolutely.

Requiments:

• possibility skip test using minimum code
• test definition should be simplest

before, after will used seldom. skip will used very seldom.

So I vote for directive Directives afterwards (see https://github.com/DevExpress/testcafe/issues/802#issue-176882339)

Also I don't like syntax with template string.
fixture('grid sorting').page('http://example.com') - it is simple chain of the two function calls.

fixture `grid sorting`.page `http://example.com` 

I dont understand logic of this.
New keyword fixture, then extention method of template string that redirect page, then again templated string - it is very strange and it is not clear

[miherlosev]

In AVA test runner (https://github.com/avajs/ava) possible use any order of test modifiers.
see https://github.com/avajs/ava#chaining-test-modifiers.

I think it is a good solution.

Also I found features that we can to do in testscafe:
https://github.com/avajs/ava#test-placeholders-todo
https://github.com/avajs/ava#failing-tests

[inikulin]

@miherlosev

In AVA test runner (https://github.com/avajs/ava) possible use any order of test modifiers.

We're obviously gonna support the same and it's irrelevant to question raised here.

https://github.com/avajs/ava#test-placeholders-todo
https://github.com/avajs/ava#failing-tests

I might be wrong, but I find this features enforcing bad practices. I have no idea why and where they can be useful. Do you have any real life scenarios for this?

[miherlosev]

Do you have any real life scenarios for this?

See description for AVA's specified features.

[inikulin]

@miherlosev I've read them, but I still don't remember that I've ever needed something like this. That's why I'm asking you for scenarios from your experience.

[inikulin]

So I propose:

test `My test name` (async t => {
  // Test code
});

test `My other test` 
    .skip
    .before(() => { ... })
(async t => {
  // Test code
}); 

As a new recommended test declaration syntax (we'll enforce it in the docs as well).
Please add 👍 or 👎 on this comment if you aggree or disagree so I'll be able to move it from proposal stage.

\cc @DevExpress/testcafe

[FDiskas]

what about nested tests?
:black_small_square:
-> :black_small_square:
-> -> :black_small_square:

[AndreyBelym]

@FDiskas, there is an issue about nested fixtures and BDD interface: https://github.com/DevExpress/testcafe/issues/792

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs or feature requests. For TestCafe API, usage and configuration inquiries, we recommend asking them on StackOverflow.