Detox is not showing which expectation failed

[Elijen]

Describe the bug
When a test fails I want to be able to know why (what expectation was not met). Instead I only see which test failed.

To Reproduce

• [x] I have tested this issue on the latest Detox release and it still reproduces

Provide the steps necessary to reproduce the issue. If you are seeing a regression, try to provide the last known version where the issue did not reproduce.

1. Write a test with multiple expectations, where one will fail
2. Run tests
3. Observe error log

Expected behavior
I see an error message explaining why test failed.

Screenshots

Environment (please complete the following information):

• Detox: 12.1.3
• React Native: Expo v32
• Node: 8.15.1
• Device: iPhone 8
• Xcode: 10.1
• iOS: 12.1
• macOS: 10.14.4

Device and Verbose Detox Logs

detox test --loglevel verbose
detox[72264] INFO:  [test.js] configuration="ios.sim" loglevel="verbose" artifactsLocation="artifacts/ios.sim.2019-04-10 07-05-02Z" recordLogs="none" takeScreenshots="none" recordVideos="none" node_modules/.bin/jest --config=e2e/config.json --maxWorkers=1 '--testNamePattern=^((?!:android:).)*$'
● Deprecation Warning:

  Option "setupTestFrameworkScriptFile" was replaced by configuration "setupFilesAfterEnv", which supports multiple paths.

  Please update your configuration.

  Configuration Documentation:
  https://jestjs.io/docs/configuration.html

detox[72265] INFO:  [DetoxServer.js] server listening on localhost:50310...
detox[72265] DEBUG: [AsyncWebSocket.js/WEBSOCKET_OPEN] opened web socket to: ws://localhost:50310
detox[72265] DEBUG: [DetoxServer.js/LOGIN] role=tester, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=tester, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_CMD, #0] /usr/bin/xcrun simctl list -j
detox[72265] DEBUG: [exec.js/EXEC_CMD, #1] applesimutils --list --byType "iPhone 8" --byOS "12.1"
detox[72265] DEBUG: [exec.js/EXEC_TRY, #1] Searching for device matching iPhone 8...
detox[72265] DEBUG: [exec.js/EXEC_CMD, #2] applesimutils --list --byId "493FBB48-9A18-4A8C-85C9-5C5310EE5BA7"
detox[72265] DEBUG: [exec.js/EXEC_CMD, #3] /usr/bin/xcrun simctl uninstall 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #3] Uninstalling host.exp.Exponent...
detox[72265] DEBUG: [exec.js/EXEC_SUCCESS, #3] host.exp.Exponent uninstalled
detox[72265] DEBUG: [exec.js/EXEC_CMD, #4] /usr/bin/xcrun simctl install 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 "/Users/elijen/Code/xps-client/bin/Exponent.app"
detox[72265] DEBUG: [exec.js/EXEC_TRY, #4] Installing /Users/elijen/Code/xps-client/bin/Exponent.app...
detox[72265] DEBUG: [exec.js/EXEC_TRY, #4] Installing /Users/elijen/Code/xps-client/bin/Exponent.app...
detox[72265] DEBUG: [exec.js/EXEC_SUCCESS, #4] /Users/elijen/Code/xps-client/bin/Exponent.app installed
detox[72265] DEBUG: [exec.js/EXEC_CMD, #5] /usr/bin/xcrun simctl terminate 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #5] Terminating host.exp.Exponent...
detox[72265] DEBUG: [exec.js/EXEC_SUCCESS, #5] host.exp.Exponent terminated
detox[72265] DEBUG: [exec.js/EXEC_CMD, #6] /bin/cat /dev/null >/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.out 2>/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/elijen/Library/Detox/ios/851f44e610e30166ef9a5155c0aa6618f7564749/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent --args -detoxServer ws://localhost:50310 -detoxSessionId a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_TRY, #6] Launching host.exp.Exponent...
detox[72265] INFO:  [AppleSimUtils.js] host.exp.Exponent launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F /Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.{out,err}
detox[72265] DEBUG: [DetoxServer.js/CANNOT_FORWARD] role=testee not connected, cannot fw action (sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d)
detox[72265] DEBUG: [DetoxServer.js/LOGIN] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_CMD, #7] /usr/bin/xcrun simctl terminate 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #7] Terminating host.exp.Exponent...
detox[72265] DEBUG: [exec.js/EXEC_SUCCESS, #7] host.exp.Exponent terminated
detox[72265] DEBUG: [DetoxServer.js/DISCONNECT] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_CMD, #8] /bin/cat /dev/null >/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.out 2>/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/elijen/Library/ExpoDetoxHook/ios/efc4c80d1c0afa30121fb80ef512e9b058ef8680/ExpoDetoxHook.framework/ExpoDetoxHook:/Users/elijen/Library/Detox/ios/851f44e610e30166ef9a5155c0aa6618f7564749/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent --args -detoxServer ws://localhost:50310 -detoxSessionId a39f36c5-dea4-757e-d654-7b23f07a7e7d -EXKernelDisableNuxDefaultsKey true -detoxURLBlacklistRegex \("http://192.168.0.94:19001/onchange","https://e.crashlytics.com/spi/v2/events"\) -detoxURLOverride exp://192.168.0.94:19000 -detoxSourceAppOverride host.exp.exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #8] Launching host.exp.Exponent...
detox[72265] INFO:  [AppleSimUtils.js] host.exp.Exponent launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F /Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.{out,err}
detox[72265] DEBUG: [DetoxServer.js/CANNOT_FORWARD] role=testee not connected, cannot fw action (sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d)
detox[72265] DEBUG: [DetoxServer.js/LOGIN] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_CMD, #9] /usr/bin/xcrun simctl terminate 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #9] Terminating host.exp.Exponent...
detox[72265] DEBUG: [exec.js/EXEC_SUCCESS, #9] host.exp.Exponent terminated
detox[72265] DEBUG: [DetoxServer.js/DISCONNECT] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [exec.js/EXEC_CMD, #10] /bin/cat /dev/null >/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.out 2>/Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/elijen/Library/ExpoDetoxHook/ios/efc4c80d1c0afa30121fb80ef512e9b058ef8680/ExpoDetoxHook.framework/ExpoDetoxHook:/Users/elijen/Library/Detox/ios/851f44e610e30166ef9a5155c0aa6618f7564749/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err 493FBB48-9A18-4A8C-85C9-5C5310EE5BA7 host.exp.Exponent --args -detoxServer ws://localhost:50310 -detoxSessionId a39f36c5-dea4-757e-d654-7b23f07a7e7d -EXKernelDisableNuxDefaultsKey true -detoxURLBlacklistRegex \("http://192.168.0.94:19001/onchange","https://e.crashlytics.com/spi/v2/events"\) -detoxURLOverride exp://192.168.0.94:19000 -detoxSourceAppOverride host.exp.exponent
detox[72265] DEBUG: [exec.js/EXEC_TRY, #10] Launching host.exp.Exponent...
detox[72265] INFO:  [AppleSimUtils.js] host.exp.Exponent launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F /Users/elijen/Library/Developer/CoreSimulator/Devices/493FBB48-9A18-4A8C-85C9-5C5310EE5BA7/data/tmp/detox.last_launch_app_log.{out,err}
detox[72265] DEBUG: [DetoxServer.js/CANNOT_FORWARD] role=testee not connected, cannot fw action (sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d)
detox[72265] DEBUG: [DetoxServer.js/LOGIN] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
 FAIL  e2e/firstTest.spec.js (61.135s)
  Example
    ✓ should have login form (18583ms)
    ✕ should be able to login (17419ms)

  ● Example › should be able to login



      13 |   });
      14 | 
    > 15 |   it("should be able to login", async () => {
         |   ^
      16 |     await element(by.id("usernameField")).typeText("demo player");
      17 |     await element(by.id("passwordField")).typeText("demo player");
      18 |     await element(by.id("logInButton")).tap();

      at Env.it (../node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:91:24)
      at Suite.it (firstTest.spec.js:15:3)
      at Object.describe (firstTest.spec.js:3:1)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 passed, 2 total
Snapshots:   0 total
Time:        61.239s
Ran all test suites with tests matching "^((?!:android:).)*$".
detox[72265] DEBUG: [DetoxServer.js/DISCONNECT] role=tester, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72265] DEBUG: [DetoxServer.js/DISCONNECT] role=testee, sessionId=a39f36c5-dea4-757e-d654-7b23f07a7e7d
detox[72264] ERROR: [cli.js] Error: Command failed: node_modules/.bin/jest --config=e2e/config.json --maxWorkers=1 '--testNamePattern=^((?!:android:).)*$'

[LeoNatan]

I don't think this is Detox related. The test runner is responsible for displaying this information. I can't help you with that, but it sounds like you should read the configuration of the test runner you are using (by default, mocha, but we also support Jest).

[LeoNatan]

@rotemmiz Is this correct?

[Elijen]

@LeoNatan I think you are right. Although this setup is generated using detox init -r jest so either the template or documentation should be updated.

Mocha shows all errors properly.

[Olovorr]

Having the same issue, did you manage to solve this somehow @Elijen? If so, could you share please? (even though that this looks like jest issue).

I used detox init -r jest as well.

[Elijen]

@Olovorr I have solved this by switching to Mocha as I know Jest has been plagued by similar issues for years, so I didn't want to waste hours trying to solve it. Perhaps it's an easy configuration fix, though.

See: https://github.com/facebook/jest/issues/3893 and https://github.com/facebook/jest/issues/2441
Perhaps passing verbose: false to Jest would help?

[buzzb0x]

Having the same issue and it used to display. I'd rather not switch to Mocha. Does anyone have a clue?

[Olovorr]

@Elijen thanks, I couldn't solve it by switching Mocha but I was able to solve it in the end :)

@buzzb0x I managed to make it work by downgrading jest to version 23.6.0.

It was also kind of working if I wrapped the test in try & catch statement and just consoled the error instead of throwing. But obviously, in this case the test was false positive, I just could read the actual error in the console.

[noomorph]

@Elijen, @Olovorr, @buzzb0x, thanks for bringing up the issue. It is going to be resolved by #1336.