Quick: How to properly use asynchronous expectations with beforeEach/afterEach beforeSuite/afterSuite

[jgongo]

Hi,

I'm trying to test the network layer of my application, and would like to do something like this:

var networkLayer: NetworkLayer!

beforeEach {
    // Prepare infrastructure for tests, for example, stub some network requests using OHHTTPStubs
    networkLayer = ...
}

afterEach {
    // Tear down infrastructure for tests, including removing all OHHTTPStubs
}

it("should properly do some network operation") {
    var actualResult: Result
    networkLayer.doSomething() { result in actualResult = result }
    expect(actualResult).toEventuallyNot(beNil())
    expect(actualResult.someProperty).toEventually(equal("expectedValue"))
}

I understand that the examples are run asynchronously, so for example afterEach may be invoked _before_ the network operation has been fired, failing because the stubs are not there anymore. Am I right? If so, what is the proper way to do this kind of tests?

Summing up, what I would like to have is some way to delay the execution of afterEach until the asynchronous operations have finished (maybe because the test succeeds or because it timeouts), and be sure that there is no other test running in parallel.

I have found another way to run the tests, but I'm not sure this is correct:

it("should properly do some network operation") {
    networkLayer.doSomething() { result in 
        expect(result).toNot(beNil())
        expect(result.someProperty).to(equal("expectedValue"))
    }
}

This is, running synchronous tests inside the callback of the asynchronous operation. Is this correct? Anyway I understand this doesn't guarantee that the test is run in isolation due to its asynchronous nature.

BTW, the problems that I'm finding in the tests are caused by two things:

• The network layer has state (for example remembers authentication state), and sometimes I chain several operations to test that it's behaving correctly:

it("should forget about authentication information) {
    networkLayer.login(user,pass) { networkLayer.logout() {} }
    // make sure the authentication information is gone for sure
}

In this case I found that networkLayer could have changed when invoking the login callback. I solved this capturing the currently configured networkLayer in a local variable:

it("should forget about authentication information) {
    let localNetworkLayer = networkLayer
    localNetworkLayer.login(user,pass) { localNetworkLayer.logout() {} }
}

• The network layer saves some information in _NSUserDefaults_ and the keychain so upon application restart there's no need to repeat the login. I want to test this behavior, but I don't know how to do it if tests aren't run isolated due to their asynchronous behavior.

I've read in some closed issue that the asynchronous operation should be performed in beforeEach. I understand this wouldn't solve my problems, as afterEach can still run before the asynchronous operation started in beforeEach has even started to run, am I right?

[jgongo]

Would waitUntil in Asynchronous Expectations help me? Would this 'block' the running test until the asynchronous operation ends so I can be sure no other test is running in parallel? Should I include this in beforeEach or in it blocks?

[jgongo]

So waitUntil indeed seems to be the solution to my problems. Just for the sake of documentation, if anybody else has the same problem, the following spec works properly:

class AsyncSpec: QuickSpec {
    override func spec() {
        describe("async") {
            var value: String? = "initial"

            beforeEach {
                waitUntil(timeout: 5.0) { done in
                    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0)) {
                        NSThread.sleepForTimeInterval(3.0)
                        value = "modified"
                        done()
                    }
                }
            }

            afterEach {
                value = nil
            }

            it("should do some async operation") {
                expect(value) == "modified"

                waitUntil(timeout: 5.0) { done in
                    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0)) {
                        NSThread.sleepForTimeInterval(2.0)
                        value = "locally modified"
                        done()
                    }
                }

                expect(value) == "locally modified"
            }
        }
    }
}

Thanks and sorry for the noise.

[DanielAsher]

not sure this is noise. this probably need to be address. tricky semaphores can't be idiomatic Quick.

[jeffh]

@danielasher, maybe because I'm on my phone, but I don't see any semaphores.

[DanielAsher]

@jeffh, @jgongo: hi!

perhaps I was being colloquial with language, but I was referring to the role:
var value: String? = "initial" adopts.

Wikipedia has this in its second paragraph on semaphores:

_A trivial semaphore is a plain variable that is changed (for example, incremented or decremented, or toggled) depending on programmer-defined conditions._

Anyhow, var value: String? is shared mutable state. 'Nuff said :)

Quick doesn't seem to have the ambition to become a fully monadic system, but it's so very expressive!

I would like to see expect support Observable. In the test below, I fake it :)

import RxSwift

class AsyncSpec: QuickSpec 
{
    override func spec() 
    {
       describe("async") {

       let machineState = Variable("Initial")

       machineState >- debug("machineState:") >- subscribeNext { _ in return }

       let createService = { (resultState: String) -> Disposable in

            machineState.next("Awaiting")

            return interval(3.0, MainScheduler.sharedInstance) 
                >- take(1) // take only the next async `result`. 
                           // here we ignore that and simply set to `modified`.
                >- debug("service:")                 
                >- subscribeNext { result in machineState.next(resultState) } 
        }

        beforeEach { createService("Modified") }

        afterEach { machineState.next("Idle") }

        fit("should do some async operation") {

            expect(machineState.value).toEventually(beginWith("Modified"), timeout: 5)                

            createService("LocallyModified")

            expect(machineState.value).toEventually(beginWith("LocallyModified"), timeout: 5) 
          }
      }
   }
}

Which also boasts this output into the test log:

[machineState:] -> Event Next(Awaiting)
[service:] subscribed
[service:] -> Event Next(0)
[machineState:] -> Event Next(Modified)
[service:] -> Event Completed
[machineState:] -> Event Next(Awaiting)
[service:] subscribed
[service:] -> Event Next(0)
[machineState:] -> Event Next(LocallyModified)
[service:] -> Event Completed
[machineState:] -> Event Next(Idle)

[jeffh]

Since I forgot to mention earlier, @jgongo, thanks for posting your solution for others in the future. It's greatly appreciated :+1:

---

@DanielAsher,

Quick doesn't aim to provide monadic types. While monads are a versatile, expressive construct, they also add conceptual overhead with little gain to what a mutable variable satisfies for most users here. While it's technically a shared mutable variable, the use of beforeEach effectively clears it's state. And this doesn't require that it's shared between tests (Quick's implementation could dictate creation of the enclosed variable for each test if it chose).

While I agree a lot with the functional style, I don't think it's something worth dictating to users of Quick/Nimble, ignoring the possibility of depending on RxSwift.

Please move any further discussion to issue #365. Thanks!

[jgongo]

:thumbsup: :smile: