Jest: option to run Jest in band and sort tests alphabetically

[aaronabramov]

the fact that Jest executes tests in different order was always a pain when testing.

can we implement an option (at least internally) to disable sorting by test running time and sort them alphabetically?

Projects using Jest might also benefit from it (debugging CI is a pain sometimes when you don't know which test is run when)

[cpojer]

No, Jest should absolutely not make any guarantees about the order in which it runs tests, and especially not publicly exposed. Tests are run the way Jest wants to run them, and integration tests should be able to account for them. We shouldn't add features to make testing Jest itself easier.

[aaronabramov]

fair enough. Testing features in production code is not the best practice :)

[danvc]

@aaronabramov

What are the features for this purpose? Could you please cite them?

[aaronabramov]

@DanZeuss not sure what you mean, but we solved the problem eventually by parsing the output and sorting it later

[danvc]

@aaronabramov I thought there were a way to define the ordering, or filenames that should run firstly.

[lukepighetti]

@cpojer are there any published best practices for writing integration test suites with Jest that rely on a specific order of operations?

[SimenB]

Not sure what you mean, but you can have jest output json and parse that. You can look at the integration tests in this repo for inspiration

[lukepighetti]

await load page
expect page to have loaded

await find button
expect button to exist

await click button
expect cart count to increment 1

How do we sequence these events with Jest without making one huge test with tons of expect()

[SimenB]

Tests inside of a single file is executed in definition order

[lukepighetti]

Thanks for the quick response. :)

first.spec.js
  describe
    async test 1
    async test 2
  describe
    async test 3
    async test 4

second.spec.js
  describe
    async test 5
    async test 6

is 1,2,3,4 guaranteed?
is 5,6 guaranteed?
do they need --runInBand to be guaranteed?

is there a way to guarantee 1,2,3,4,5,6 ?

[SimenB]

We do not guarantee between first.spec.js and second.spec.js and there is no way to do so. Within those, yes, order is guaranteed

[markpradhan]

I do some integration testing with puppeteer within jest. Having the ability to guarantee the order of your tests is paramount in some cases. I have noticed this pattern works:

first.specpartial.js

export default () => {
  describe('First', () => {
    test('First', () => {})
  })
}

second.specpartial.js

export default () => {
  describe('Second', () => {
    test('Second', () => {})
  })
}

third.specpartial.js

export default () => {
  describe('Third', () => {
    test('Third', () => {})
  })
}

test.spec.js

import first from './first.specpartial'
import second from './second.specpartial'
import third from './third.specpartial'

first()
second()
third()

Terminal output

$ export BABEL_ENV=test NODE_ENV=test && jest --verbose --runInBand
 PASS  utils/test.spec.js
  First
    ✓ First (2ms)
  Second
    ✓ Second
  Third
    ✓ Third (1ms)

[rostgoat]

@markpradhan how did u get em to run?

i can't seem to following your approach npx jest --forceExit --runInBand --detectOpenHandles "test/e2e/tests/all.js"

import {first} from './super/clients/add.spec'
import {second} from './super/portals/add.spec'
import {thirds} from './super/stores/add.spec'
import {fourth} from './super/users/edit.spec'

first();
second();
thirds();
fourth();

always get No tests found, exiting with code 1

[markpradhan]

@rostgoat can you post one of the imported test files as well?

[rostgoat]

yessir

import puppeteer from "puppeteer";
const CheckMethods = require('../../../methods/common/checks');
const config = require('../../../../../config/config');
require('jest-extended');

let page;
let browser;

export default () => {
  describe('Super - Client', () => {
    const checks = new CheckMethods();
    beforeEach(async () => {
      browser = await puppeteer.launch(config.puppeteer);
      page = await browser.newPage();
      await page.setViewport(config.browser);
      await page.goto(`${config.ui_endpoint}/super/#login`, {
        waitUntil: 'networkidle2'
      })
    });

    afterEach(async () => {
      await browser.close();
    });

    it('should be able to create a client in super', async () => {
      ...
    }, 10000)
  });
}

[markpradhan]

- import {first} from './super/clients/add.spec'
+ import first from './super/clients/add.spec'

Because you are exporting as default.
But jest should actually throw an error because you are calling functions that do not exist.

Try running a simple test in test/e2e/tests/all.js itself to check if the file is targeted correctly.

[rostgoat]

good point, u were totally right! my jest.config was only looking for files with spec hence all.js didn't run anything.

also, good catch on export default, totally missed that

thanks :+1: