Quick: itBehavesLike is not invoking beforeEach

[nandin-borjigin]

• [x] I have read CONTRIBUTING and have done my best to follow them.

What did you do?

I'm using shared examples and find that itBehavesLike doesn't behave like it -- itBehavesLike won't trigger beforeEach itself, so any forced unwrapping variables shouldn't be accessed directly within sharedExample closure. One should move that unwrapping either into it closure or beforeEach closure inside the sharedExample

The code follows demonstrates the problem

class FooSharedExampleConf: QuickConfiguration {
    override class func configure(_ conf: Configuration) {
        sharedExample("foo") {
            (context: SharedExampleContext) in
            let bar = context()["bar"] as! Bar // This line is causing crash
            let baz = context()["baz"] as! Baz // This line would crash too
            it("some test") {
                expect(bar.baz) == baz
            }
            it("another test") {
                expect(bar.baz2) == baz
            }
        }
    }
}

class BarBazSpec: QuickSpec {
    override class func spec() {
        var bar: Bar!
        var baz: Baz!
        beforeEach {
            bar = Bar()
            baz = Baz()
        }
        describe("bar baz") {
            itBehavesLike("foo") { 
                ["bar": bar, "baz": baz] 
                // crashes here because of unwrapping nil, bar == nil & baz == nil, i.e. beforeEach hasn't executed yet
            }
        }
    }
}

beforeEach closure in BarBazSpec won't be called for itBehavesLike untill it reaches a it closure, so when the code runs into commented lines in FooSharedExampleConf the beforeEach closure hasn't been executed thus bar and baz is still nil.

class FooSharedExampleConf: QuickConfiguration {
    override class func configure(_ conf: Configuration) {
        sharedExample("foo") {
            (context: SharedExampleContext) in
-            let bar = context()["bar"] as! Bar
-            let baz = context()["baz"] as! Baz
+            var bar:Bar!
+            var baz:Baz!
+            beforeEach {
+                bar = context()["bar"] as! Bar
+                baz = context()["baz"] as! Baz
+            }
            it("some test") {
                expect(bar.baz) == baz
            }
            it("another test") {
                expect(bar.baz2) == baz
            }
        }
    }
}

Making some change like this will correct the problem, but I consider it as unintuitive.
First of all, itBehavesLike has an "it" in it and that's why I thought it should trigger beforeEach like it closure does. And as it can be seen from the code above, adding another beforeEach closure inside the sharedExample is verbose.

I suppose itBehavesLike actually behaves more like describe and context because it is meant to have several it in it. From this perspective, this makes sense, but I'm still wondering if there is better solution to this.

What did you expect to happen?

I thought itBehavesLike itself triggers beforeEach closure itself

What actually happened instead?

itBehavesLike doesn't trigger beforeEach closure

Environment

List the software versions you're using:

• Quick: 1.0
• Nimble: 1.0
• Xcode Version: Version 8.2.1 (8C1002)
• Swift Version: Xcode Default
• Package Manager: none, git submodule only

[jeffh]

Hey @Nandiin,

Thanks for filing an issue 😄 . The behavior of itBehavesLike is expected, although I agree that the wording of itBehavesLike creates ambiguity. If you have any suggestions that would be greatly appreciated!

[CaioSym]

@Nandiin
I'm no Quick specialist, but iI think the issue here comes from the fact that unlike beforeEach and it, itBehavesLike is used for setting up the test suite and not during its execution, just as context and describe. Hence, as jeffh said it is expected that anything not inside a runtime closure in itBehavesLike will be executed before the run time closures such as itBehavesLike. This is why you are forced to declare context as escaping if you use it inside beforeEach btw.

@jeffh
I think a possible way around this issue is to have sharedExample have a structure more like the following:

class FooSharedExampleConf: QuickConfiguration {
    override class func configure(_ conf: Configuration) {
        sharedExample("foo") {

            var bar:Bar!
            var baz:Baz!

            setupExample() { context: SharedExampleContex in
                 let bar = context()["bar"] as! Bar // This line is causing crash
                 let baz = context()["baz"] as! Baz // This line would crash too
            }

            it("some test") {
                expect(bar.baz) == baz
            }

            it("another test") {
                expect(bar.baz2) == baz
            }
        }
    }
}

where the Quick framework would have to be responsible for running the setupExample similarly to a before each closure at the appropriate point in time

This way one cannot run the SharedExampleContex closure in a non escaping way, or at least not without a lot of counter intuitive code writing.

Let me know your thoughts or if I'm missing something

[jeffh]

Hey @CaioSym, thanks for commenting.

That's an interesting solution, although it does cause problems if one wants to use it inside another closure. An example is passing along constants through a context:

class FooSharedExampleConf: QuickConfiguration {
    override class func configure(_ conf: Configuration) {
        sharedExample("foo") { context in
            var bar: Bar!
            var baz: Baz!

            beforeEach {
                bar = context()["bar"] as! Bar
                baz = context()["baz"] as! Baz
            }

            it("some test") {
                expect(bar.baz) == context()["expectedBaz"] as! Baz
            }

            it("another test") {
                expect(bar.baz2) == context()["expectedBaz"] as! Baz
            }
        }
    }
}

Maybe it's not worth having that available to it closures. But I can imagine something like that for multiple beforeEach closures.

[CaioSym]

@jeffh I may be missing something crucial here but it seems to at worst case one could simply do as follows:

class FooSharedExampleConf: FooSharedExampleConf {
    override class func configure(_ conf: Configuration) {
        sharedExample("foo") {
            var context: SharedExampleContex!
            var staticContextResult: [Hashable: Any]! // or specific type in case of strongly typed contexts
            var bar:Bar!
            var baz:Baz!

            setupExample() { ctx: SharedExampleContex in
                 context  = ctx
                staticContextResult = ctx()
                 bar = ctx()["bar"] as! Bar // This line is causing crash
                 baz = ctx()["baz"] as! Baz // This line would crash too
            }

           beforeEach() {
               //do Something with context
           }

            it("some test") {
                expect(bar.baz) == context()["expectedBaz"] as! Baz
            }

            it("another test") {
                expect(bar.baz) == staticContextResult["expectedBaz"] as! Baz
            }
        }
    }
}

The main point is making it explicit that context() is likely a dependent on a var x: Any! and so should be declared in the same fashion. Again, I may be missing something that would make the above approach unfeasible...

[Arcikbtw]

There is no wonder why did Nandiin expect that behavior of itBehavesLike as it is used this way in the documentation.
https://github.com/Quick/Quick/blob/master/Documentation/en-us/SharedExamples.md

And it would be a good idea, to let it work this way. The shared example could be much more reusable and make more sense for unit tests. Currently, the solution is to create beforeEach in sharedExample and create sut inside. Problem is that its impossible to create subject under test in beforeEach before calling sharedExample and pass it in context, now we can only pass some configuration.

[nickjwallin]

At the very least, the documentation should be updated, since as @Arcikbtw mentions the documentation implies that beforeEach should be called. I think also the current way is very unintuitive. There is even an error that gets hit if you try to use itBehavesLike inside an it block, saying it must be in a describe or context block. But then why wouldn't the beforeEach of each enclosing describe and context not be called, as it is for everything else? I am not sure how the current itBehavesLike can be used for anything but the most trivial of tests.