Detox: Videos recording sometimes causes simulator to freeze when a test fails

[d4vidi]

Describe the bug
There seems to be a problem with video recording (i.e. --record-videos arg) on iOS: The simulator succeeds in running and recording the first test, but after that the simulator just freezes, and the generated .mp4 artifact is an empty file.

In the beginning I thought this had something to do with jest-circus, but it appears to also reproduce when using Jest+Jasmine2 and also Mocha. It typically reproduces immediately when a test fails, but only once every few runs.

Additional note: I haven't managed to have this reproduced on Android (standard emulator).

To Reproduce

• [x] I have tested this issue on the latest Detox release and it still reproduces

1. Set up a project with either Mocha or Jest 24.8.0 as the test runner (default setup).
2. Set up as single e2e test suite with just one e2e test that fails.
3. Run the suite a few times with --record-videos all; eventually the simulator will freeze upon run completion.

Expected behavior
Should generate a proper artifact and keep the simulator alive.

Screenshots
N/A

Environment (please complete the following information):

• Detox: 14.3.1
• React Native: 0.59.9
• Node: 10.15.3
• Device: iPhone X
• Xcode: 10.12
• iOS: 12.1
• macOS: 10.14.6

Device and Verbose Detox Logs
N/A

iOS Framework Build Logs
N/A

[d4vidi]

@noomorph I would highly appreciate your help with this one.

[LeoNatan]

If the file is empty, it sounds like stop recording for previous was not performed. Lifecycle issues, as I feared.

[noomorph]

I was able to reproduce the problem just by this script:

#!/usr/bin/env bash

for i in {1..6}
do
   yes > /dev/null &
done

for i in {1..25}
do
  REC=/tmp/ios-sim-$RANDOM.mp4
  timeout 5 /usr/bin/xcrun simctl io booted recordVideo $REC
  du -h $REC
  rm -rf $REC
done

killall yes

The main idea is to load the CPU, about 75-80% of its capacity.
Then you take 10-25 video recordings, 5 seconds each.
While the script is running, try doing some stupid actions on iOS simulator, swipe back and forth on the Springboard.
When the script finishes, it kills all "yes" processes that were creating that CPU load.
The simulator remains unresponsive for an infinite amount of time even after that.

[d4vidi]

@noomorph I really appreciate your help on this!
Your script _does_ show then that video recording can be buggy. I'm still bewildered though as to what makes our integration with circus stress-test it into the edge.

[d4vidi]

Well, here's a newsflash: I've managed to reproduce this quite solidly running the usual Jest+jasmine setup (over our jest example app). Looks like a bug - not related specifically to jest-circus. I've only seen it happen when tests fail.
'Will change the title accordingly.

[d4vidi]

@LeoNatan I don't know about lifecycle, according to the logs it seems everything is run in place. I've also checked runtimes by the millisecond and they seems quite similar comparing a successful and erroneous run.

[d4vidi]

Well hurray, this also happens with Mocha.

[d4vidi]

@LeoNatan I've assigned this to @noomorph on an instinct, but on 2nd thought, as this fails on all runners and not on android, perhaps it'd be better if you try to look for iOS-ish insights as to what adjustments or even workarounds we could implement in order to stabilize this. If that's ok, I'm assigning it to you for the time being...

[LeoNatan]

Sorry, I don’t have time to look at this.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

[d4vidi]

@LeoNatan you want to take a look at this nevertheless?

[LeoNatan]

Not really, but let’s keep open. Not sure what I can do if it’s a bug in Xcode.

[vonovak]

just a note - the situation seems to be better in xcode 11.2.
In Xcode 11, recording does not work at all (https://developer.apple.com/documentation/xcode_release_notes/xcode_11_release_notes?language=objc) but in 11.2 beta 2 this has been fixed (https://developer.apple.com/documentation/xcode_release_notes/xcode_11_2_beta_2_release_notes?language=objc). Still, migrating our test suite to latest xcode & detox is quite a pain (to summarize, a lot of pretty random failures) :/

[LeoNatan]

@vonovak Thanks for letting us.

If you are seeing issues on our end, please open issues. Thanks

[dmarkowski]

We are facing a similar issue in our environment. If we add --record-videos to our test running script

detox test -c ios.sim.release-int --cleanup --record-videos all

and one of the tests will fail because of the bug in code there is a random timeout error (“Timeout - Async callback was not invoked within the 360000ms timeout specified by jest.setTimeout.Error….”) on other tests and simulator stays booted even the tests are finished.

After all tests there is information on the console:

`Jest did not exit one second after the test run has completed.

This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with --detectOpenHandles to troubleshoot this issue.`

Without --record-videos option we never get any Timeout errors if there are any test failures.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

[stale[bot]]

The issue has been closed for inactivity.