Nimble: toEventually blocks the main thread

[Blackjacx]

I found an issue that I need to solve. Consider the following code snippet:

if !isExecuted {
  isExecuted = true
  let delayTime = dispatch_time(DISPATCH_TIME_NOW, Int64(0.5 * Double(NSEC_PER_SEC)))
  dispatch_after(delayTime, dispatch_get_main_queue()) {
    self.centerMapOnUserLocation()
  }
}

I want to center the map on the users location using a delay or I want to simply dispatch something else on the main thread using Grand Central Dispatch.

Now when I test that piece of asynchronous code using toEventually the dispatched task is blocked by the toEventually on the main thread which causes test pollution I think. See the following example:

run my piece of code ----o----- dispatches a task on the main thread after toEventually() (task2)
run toEventually ---------o----- polling regularly on mainThread (task1)

So it comes that task2 blocks task1 and produces a deadlock. Am I doing something wrong?

[modocache]

Hello! Hmm, sounds like a problem, but not one I've ever encountered. It's also not caught by our test suite. Could you submit a pull request to add a test to Nimble's test suite that demonstrates the failure you're seeing? Alternatively, could you create a small sample project that demonstrates the failure? That'd really help us solve this issue quickly! :)

[Blackjacx]

Hmmm I cannot reproduce that issue with a clean project but I get the following failure message:

timed out, but main thread was unresponsive

I searched for it and it indicates a blocked main thread. This leads to randomly failing test cases that use toEventually.

But maybe I found a solution in our code for that. I do a performSelectorOnMainThread with waitUntilDone set to true. This may cause some deadlocks.

So I think it is a failure in our code and you can close this issue.

[jeffh]

Closing, feel free to reopen if you think it's a problem with Nimble.

[asalom]

We have this issue for some time now. We have a big test suite with 4000 tests, 122 of which make use of toEventually. When we run them locally, everything is good but whenever we use a slower machine for CI (either a local mac mini with Jenkins or a virtual machine with Travis), tests with toEventually randomly fail and it only happens some times, not every time. With Travis we realized that whenever we have several jobs running in parallel, the virtual machines we get are slower than if only one job is running. So whenever we spawn 4-5 jobs at once, some tests with toEventually are more prone to fail.

[jeffh]

Hey @asalom,

Sadly, timeout tweaking is inherit to asynchronous tests (I personally prefer to avoid them whenever possible because of this). But you can increase the default timeout as documented in the README as a hacky-fix:

// Increase the global timeout to 5 seconds:
Nimble.AsyncDefaults.Timeout = 5

// Slow the polling interval to 0.1 seconds:
Nimble.AsyncDefaults.PollInterval = 0.1

Alternatively, you can set specific timeouts for specific expectations:

// Evaluate someValue every 0.2 seconds repeatedly until it equals 100, or fails if it timeouts after 5.5 seconds.
expect(someValue).toEventually(equal(100), timeout: 5.5, pollInterval: 0.2)

[asalom]

Thank you, we gave a small try to this but didn't work very well. A soon as we migrate to travis some other project where we kind of abused eventually clauses, we'll try again and I'll post the results here

[jreyesv]

Hey @asalom,

Sadly, timeout tweaking is inherit to asynchronous tests (I personally prefer to avoid them whenever possible because of this). But you can increase the default timeout as documented in the README as a hacky-fix:

// Increase the global timeout to 5 seconds:
Nimble.AsyncDefaults.Timeout = 5

// Slow the polling interval to 0.1 seconds:
Nimble.AsyncDefaults.PollInterval = 0.1

Thanks for sharing that. I know this issue is old but seems to be happening for us now with Xcode 11.1 running on iOS 13.1 sims.

Just out of curiosity, where can you override those values? Is there a specific place where you can override those Nimble settings?

Thanks

[guillermomartinez-accenture]

@jeffh as mentioned by @jreyesv, could you indicate where to change the defaults?

[ikesyo]

I assume that a test bundle's NSPlincipalClass could be the entry point: https://developer.apple.com/documentation/xctest/xctestobservationcenter

If an NSPrincipalClass key is declared in the test bundle's Info.plist file, XCTest automatically creates a single instance of that class when the test bundle is loaded. You can use this instance as a place to register observers or do other pretesting global setup before testing for that bundle begins.