Quick: Allow tests to have a specific order

[J-Rojas]

Some asynchronous tests require multiple tests to be executed in a particular order to be easier to read, maintain, and understand.

I've noticed that Quick appears to execute tests within contexts (and between contexts) in alphabetical order of the labels. I'm not sure if this is expected or not, but the order is deterministic at the moment.

It would be helpful to add some way to specify order or priority, or at least respect the order of the tests as defined in each describe/context definition as opposed to the alphabetical order of labels.

I've worked around this problem by prefixing the it() definitions with numbers: "1 - does foo...", "2 - does foobar...", etc, but this of course is crude and isn't scalable.

• Quick: 1.2.0
• Xcode Version: 9.0
• Swift Version: 3.1

[modocache]

Thanks for the suggestion, @J-Rojas! It is deterministic, and based on alphabetical order. Quick doesn't do anything special in this regard, this behavior comes from XCTest. So a test case A_TestCase is executed before a test case named B_TestCase, and within that test case A_TestCase.test_01 is executed before A_TestCase.test_02.

I agree that Quick should allow tests to be executed within a specific order. Perhaps we could do so using a new function:

ordered() {
  it("runs this test first") {}
  it("runs this test second") {}
}

Or maybe there's another, better way to do this. Do RSpec or Ginkgo, or some other testing framework, have a similar feature?

[J-Rojas]

Thanks for commenting on this quickly with an explanation of why its ordered the way it is.

I'm not an expert on the other BDD systems, but some quick research yields some details on order configuration for both RSpec and Ginkgo:

RSpec (order can be configured per group using the :order parameter with predefined or custom ordering)
https://relishapp.com/rspec/rspec-core/docs/configuration/overriding-global-ordering

Ginkgo Spec Permutation (a hybrid of default ordering with top-level randomness)
https://onsi.github.io/ginkgo/#spec-permutation

One possible translation of this RSpec feature to Quick that I can think of would maybe look like this:

//this enum would be part of the Quick SDK
enum Order {
     case defined
     case random
}

describe("MyTests", order: Order.defined) {
    it("should run test1() first") {...}
    it("should run test2() second") {...}

    // func CustomAlphabetical(tests: [TestCase]) ->  [TestCase] 
    // is a user defined function where the return value is the sorted array of tests
    context("when a context is used", order: CustomAlphabetical) {
          it("test 'z' should run last") {...}
          it("test 'a' should run first") {...}
    }
}

[modocache]

Yeah, that API looks really great! Thanks for researching the other testing frameworks, too. Would you like to try implementing this?

[jonchui]

@modocache - looks like @jwfriese has a PR for this in https://github.com/Quick/Quick/pull/766

Any idea what's blocking ?