Detox: [SimulatorDriver] semantics of device.name has been unexpectedly changed since 7.4.0

Created on 24 Jul 2018 · 13Comments · Source: wix/Detox

Description

After upgrading to Detox 8.0.0 from 7.3.3, detox is not able to find the same iOS simulator using the name of the simulator. After downgrading, it works again, with no changes to the simulator config in between.

Error: a BeforeAll hook errored, process exiting: e2e/support/index.js:48: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"

Output of xcrun simctl list

== Devices ==
-- iOS 11.2 --
    DETOX (4E4CECC2-AA47-437F-B9B6-EC5917751329) (Booted)
-- iOS 11.3 --
    iPhone 7 (621A1BEF-EEF1-4017-BE73-A46DDAC3A0DF) (Shutdown)
-- tvOS 11.3 --
-- watchOS 4.3 --
== Device Pairs ==

``` Detox config in package.json
"detox": {
    "test-runner": "cucumber",
    "configurations": {
      "iphone": {
        "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/xx.app",
        "build": "yarn ios-detox-build",
        "type": "ios.simulator",
        "name": "DETOX"
      },
      "android": {
        "binaryPath": "./android/app/build/outputs/apk/debug/app-debug.apk",
        "build": "yarn android-detox-build",
        "type": "android.emulator",
        "name": "Nexus_5X_API_27"
      }
    }
  }

Steps to Reproduce

Create a working detox configuration with version 7.3.3
Upgrade to 8.0.0
Run tests

Detox, Node, Device, Xcode and macOS Versions

Detox: 8.0.0
React Native: 0.55
Node: v8.11.3
Device: iOS 11.2 simulator
Xcode: 9.3
macOS: 10.13.4

Device and verbose Detox logs

$ yarn e2e-test e2e/features/
$ ENVFILE=.env.e2e export $(cat .env.e2e | xargs -I{} echo 'DETOX_{}') && ./node_modules/.bin/cucumber-js --require-module babel-core/register --require-module babel-preset-stage-2 -f node_modules/cucumber-pretty --artifacts-location ./artifacts --configuration iphone e2e/features/
Started mock server on PORT 9000
detox info 13:03:50: server listening on localhost:64969...
detox info 2: Searching for device matching DETOX... 
VError: a BeforeAll hook errored, process exiting: e2e/support/index.js:48: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at /Users/xx/dev/proj/x/mobile-app/node_modules/cucumber/src/runtime/index.js:36:15
    at Generator.next (<anonymous>)
    at Generator.tryCatcher (/Users/xx/dev/proj/x/mobile-app/node_modules/bluebird/js/release/util.js:16:23)
    at PromiseSpawn._promiseFulfilled (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/generators.js:97:49)
    at Promise._settlePromise (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:574:26)
    at Promise._settlePromise0 (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/Users/xx/dev/proj/xx/mobile-app/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
caused by: Error: Can't find a simulator to match with "DETOX", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at AppleSimUtils.findDevicesUDID (/Users/xx/dev/proj/xx/mobile-app/node_modules/detox/src/devices/AppleSimUtils.js:50:13)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
error Command failed with exit code 1.

acceptebug 📌 pinned

Source

bartolkaruza

👍3

Most helpful comment

Yes, this the issue. We discussed this internally and it's an issue we intend to fix. @rotemmiz @noomorph

LeoNatan on 5 Aug 2018

👍2

All 13 comments

It appears the semantics of the name in configuration has changed. Where in 7.3.3 the name field corresponded to the AppleSimUtils --byName flag, it now only corresponds to the nested deviceType.name field, which is really a deviceType, not a name. After changing the name to iPhone 7 it was able to access a simulator, but the wrong one.

The reason that it is very important to us to be able to select a specific simulator for the UI tests, is because the UI test simulator has configurations that are not ergonomic for normal development use simulator; eg no hardward keyboard.

output of applesimutils --list

[
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "iPhone 7",   <---- Development device
    "udid" : "621A1BEF-EEF1-4017-BE73-A46DDAC3A0DF",
    "os" : {
      "version" : "11.3",
      "availability" : "(available)",
      "name" : "iOS 11.3",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-3",
      "buildversion" : "15E217"
    }
  },
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "DETOX", <---- UI Test device
    "udid" : "4E4CECC2-AA47-437F-B9B6-EC5917751329",
    "os" : {
      "version" : "11.2",
      "availability" : "(available)",
      "name" : "iOS 11.2",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-2",
      "buildversion" : "15C107"
    }
  }
]

output of applesimutils --list --byName DETOX

[
  {
    "deviceType" : {
      "name" : "iPhone 7",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    "state" : "Shutdown",
    "availability" : "(available)",
    "name" : "DETOX",  <--- UI test device
    "udid" : "4E4CECC2-AA47-437F-B9B6-EC5917751329",
    "os" : {
      "version" : "11.2",
      "availability" : "(available)",
      "name" : "iOS 11.2",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-2",
      "buildversion" : "15C107"
    }
  }
]

If this was the intended behaviour before 8.0.0 and the functionality we rely on was a bug to begin with, can you let us know how we could specify a specific simulator using the configuration?

bartolkaruza on 24 Jul 2018

It does look like a not documented breaking change, but not sure yet. I'll advise with @rotemmiz on this.

noomorph on 25 Jul 2018

This is not good and must be resolved. There is no reason to break this.

LeoNatan on 25 Jul 2018

👍1

@LeoNatan, I agree and I'll speak to Rotem at the first opportunity.

noomorph on 25 Jul 2018

etaiklein on 30 Jul 2018

Yeah, this also broke our hand-rolled parallelization of Detox tests we are using on pre-8.0 versions. We are using xcrun to script creation of 'n' simulator devices with unique names (but same underlying device type and iOS version) and using gulp to split our tests to run against them, then using an env var to tell Detox which simulator instance to use for each batch.

Detox 8 seems to have broken this; I believe because the semantics of AppleSimUtils's search for devices by UDID has been completely changed to use a 'byType' flag, with an attempt to find the latest OS version for the given name: https://github.com/wix/detox/commit/94348fc480813c94dce7707e4f45107ecfb77183#diff-2ec04c0d120ceb57a189afbc6a59a9e8

It feels to me that the new behaviour is designed around the assumption your Simulators are using "standard" names, like out-of-the-box simulators.

Unfortunately this change in semantics is preventing an incremental upgrade and conversion of our hand-rolled parallelization support to use Detox's inbuilt support :(

chadlwilson on 5 Aug 2018

Yes, this the issue. We discussed this internally and it's an issue we intend to fix. @rotemmiz @noomorph

LeoNatan on 5 Aug 2018

👍2

is it still planned to be fixed? The pull request seems to be ready but it is not yet merged.

DanielBoening on 7 Dec 2018

👍1

The same error happens when I try to run the react-native ios example. How can detox be used until the PR is merged? Older versions fail with the post install script.

vujevits on 14 Feb 2019

Similar issue happened to me. I tried to create a dedicated simulator for my e2e test on CI using
xcrun simctl create "iPhone detox" com.apple.CoreSimulator.SimDeviceType.iPhone-X com.apple.CoreSimulator.SimRuntime.iOS-12-2

The entry in my package.json looks like this:

"ios.sim.release": {
  "binaryPath": "build/kollex/Build/Products/Release-iphonesimulator/kollex.app",
  "build": "xcodebuild -workspace ios/kollex.xcworkspace -scheme kollex -configuration Release -sdk iphonesimulator -derivedDataPath build/kollex",
  "type": "ios.simulator",
  "name": "iPhone detox"
}

But detox can't find the device:

detox[51803] ERROR: [index.js/DETOX_INIT_ERROR]
 Error: Can't find a simulator to match with "6E69D11C-6814-468F-92EC-F30A183EAC72", run 'xcrun simctl list' to list your supported devices.
      It is advised to only state a device type, and not to state iOS version, e.g. "iPhone 7"
    at AppleSimUtils.findDevicesUDID (/Users/ullrich/Projects/kollex-app/node_modules/detox/src/devices/ios/AppleSimUtils.js:47:13)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)

even though applesimutils finds it:

~ applesimutils --list --byName "iPhone detox"
[
  {
    "availabilityError" : "",
    "os" : {
      "bundlePath" : "\/Applications\/Xcode.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Developer\/Library\/CoreSimulator\/Profiles\/Runtimes\/iOS.simruntime",
      "availabilityError" : "",
      "buildversion" : "16E226",
      "availability" : "(available)",
      "isAvailable" : true,
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-12-2",
      "version" : "12.2",
      "name" : "iOS 12.2"
    },
    "availability" : "(available)",
    "deviceType" : {
      "name" : "iPhone X",
      "bundlePath" : "\/Applications\/Xcode.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Developer\/Library\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone X.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-X"
    },
    "udid" : "6E69D11C-6814-468F-92EC-F30A183EAC72",
    "isAvailable" : true,
    "state" : "Booted",
    "name" : "iPhone detox"
  }
]

Env

detox: 12.8.0
applesimutils: 0.6.4

stigi on 21 May 2019

What's the workaround for this? We're trying to add E2E testing with Detox, but not being able to specify a device name is pretty much a blocker for us.