Carthage: Unable to find iOS Simulators (Post Xcode 10.2 install)

[msaps]

• carthage install method: [ ] .pkg, [x] homebrew, [ ] source
• which carthage: /usr/local/bin/carthage
• carthage version: 0.31.2
• xcodebuild -version: 10.1

After installing Xcode 10.2, Carthage fails to build any project due to Could not find any available simulators for iOS. I've tried a number of different things to resolve it:

• Build a project in Xcode 10.2 on a simulator (suggested in #2690)
• Use Xcode.app and Xcode-beta.appfor xcode-select.
• Reset all my simulators etc.

I think the problem is the keys that are in the output of simctl. Looking through Simulator.swift in CarthageKit I can see that it expects to find a platform prefix for the key (i.e. iOS) - my keys however are formatted like this: com.apple.CoreSimulator.SimRuntime.iOS-12-1.

Is this a problem with my installation? Or is it something that has changed in Xcode 10.2 and should be supported in Carthage?

Example xcrun simctl list devices --json output:

"com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
      {
        "availability" : "(available)",
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 5s",
        "udid" : "9AC9332D-7DB0-4F2E-8615-61EE86E363CA",
        "availabilityError" : ""
      },
      {
        "availability" : "(available)",
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 6",
        "udid" : "B1DA5CFF-D918-467B-8E49-04B6172B9456",
        "availabilityError" : ""
      }
]

[tmspzz]

Are you installing Xcode via MAS or developer portal?

I don't think it's and Xcode 10.2 specific issue but something related to your installation. I have no idea however why something like this would happen.

[msaps]

@blender I run the stable version of Xcode through the MAS - I'll try nuking everything and see if that resolves it. Also maybe avoid the MAS in future 🤣

[tmspzz]

Yeah, I personally don't use MAS and I have never so far encountered issues like this. I don't want to correlate the two but...

[mattgallagher]

This type of problem has affected Carthage before. It is not your fault: Carthage needs to be updated to handle the new JSON format used by the simctl that ships with the Xcode 10.2 beta.

FYI, after encountering the same problem, the issue persisted for me, even after reverting to Xcode 10.1 and deleting all Carthage build artifacts and caches.

I needed to run:

xcrun simctl delete unavailable

to delete unreadable simulators and reset the JSON format.

EDIT I think my analysis here was flawed. I had also deleted the /Library/Developer directory at the same time, forcing a re-install of the Xcode 10.1 simctl tool. I think this was the true fix for me.

[msaps]

Tried nuking my entire Xcode installation (including ~/Library/Developer/,~/Library/Caches/com.apple.dt.Xcode and the CLI tools), reinstalling 10.1 (from the dev portal) and the Simulator keys are still the same.

@mattgallagher also tried xcrun simctl delete unavailable to no avail, both before and after reinstalling Xcode.

I'm not sure what causes this format to occur but does seem like something that might be useful for Carthage to handle? What do you think @blender?

[tmspzz]

I don't think it's a carthage problem honestly.

So @msaps you have installed 10.1, ran xcrun simctl delete unavailable and now the problem is solved?

This seems to be like it's triggered by installing 10.2.

I think the right way here is to open a radar and probably add __yet another ad-hoc patch for a non final version of Xcode__...

[taher-mosbah]

Same problem for me after installing 10.2 via xcversion and switching to it using xcversion select "10.2" , switching back to 10.1 did not help.

xcrun simctl delete unavailable did not solve the issue.

What can we do to help ?

[tmspzz]

try checking what version of simclt is actually being invoked

[msaps]

@blender unfortunately not. Completely removed my installation, reinstalled 10.1 and the problem still exists. xcrun simctl delete unavailable didn't seem to do much and recreating the simulators with Fastlane fastlane snapshot reset_simulators also didn't work (both before and after reinstalling Xcode).

simctl version: @(#)PROGRAM:simctl PROJECT:CoreSimulator-587.24

[msaps]

One potentially interesting point is that I still have the 10.2 Simulator display names in my simulators list (Believe the R and S are rendered differently in <10.2).

Unless I missed something stupid, I did remove every Xcode cache/directory I could think of. Had to reinstall the Command Line Tools etc.

[tmspzz]

xcrun simctl --version on Xcode 10.1 returns @(#)PROGRAM:simctl PROJECT:CoreSimulator-581.2

Xcode 10.2 Beta yields:

@(#)PROGRAM:simctl PROJECT:CoreSimulator-587.24

so @msaps you're still running the new simctl

[tmspzz]

I see two device outputs formats currently from xcrun simctl list devices --json

one has the devices under the key in the format like platform <version> and the other one has the format com.apple.CoreSimulator.SimRuntime.<platform>-<major_version>-<minor_version>

[msaps]

I only get the verbose format.

How can my simctl version be out of sync with the current Xcode version? 🤔

[tmspzz]

@msaps do you have 10.1 CLI tools installed?

[tmspzz]

Open 10.1 and run a project on the simulator.

Now xcrun simctl list devices --json will output the format the carthage understands

[msaps]

@blender

CLI Tools look okay.

Running on the 10.1 Sim makes no difference to my simctl list output. I am really confused as to how I have the 10.2 version of simctl with no trace of it installed on my machine.

[apoltix]

I am having the same issue. I've checked out Carthage 0.31.2 and run it in Xcode 10.1 to see where it goes wrong, and have found the Simulator class checks the platform's name by prefix (e.g. "iOS"), but the prefix has (seemingly) changed with Xcode 10.2 so it's now "com.apple.CoreSimulator.SimRuntime.iOS-12-2".

After changing hasPrefix to contains in Simulator.swift's selectAvailableSimulator function, it correctly reads the simulators. For some reason they're all marked as isAvailable = false, so I'm looking into that now.

[tmspzz]

I can reliably switch versions of simctl and reset the output by doing:

• sudo xcode-select -s /Applications/Xcode_10.1/Contents/Developer
• Running any app that boots the simulator
• xcrun simctl list devices --json

[tmspzz]

please dupe:

https://openradar.appspot.com/radar?id=5058478808760320

[tmspzz]

@msaps did you xcode-select 10.1 ?

[msaps]

@blender yeah I was on 10.1 - switching between that and 10.2 didn't seem to make any difference to what simctl my machine was using weirdly.

[apoltix]

So I've managed to run Carthage 0.31.2 successfully with a single code patch, the aforementioned change of hasPrefix to contains, building and running from Xcode 10.1 but setting the environment variable DEVELOPER_DIR to the Xcode 10.2 beta developer path, e.g. /Applications/Xcode-beta.app/Contents/Developer. This makes Carthage use the correct xcrun so the simulators are available.

[msaps]

@apoltix yeah I'm currently using a locally patched version of Carthage as well.

[russbishop]

When Xcode is updated CoreSimulator is updated. CoreSimulator always rolls forward so it will be the version installed by the newest Xcode you ever launched. simctl is now bundled with CoreSimulator, that's why you are seeing the newer version.

[drekka]

I definitely have the issues where Carthage cannot see any simulators. But also noticed that the Components tab in Xcode 10.2 beta preferences is empty and won't download any.

It sounds like a lot of people are trying to back up to 10.1. I'd like to get 10.2 going instead.

[tmspzz]

@drekka You can patch carthage to do just that like so https://github.com/Carthage/Carthage/issues/2691#issuecomment-457551603

If you're interested in a little (pointless?) investigation read forward

I have done a little investigation following what @russbishop has said and I found the following.

I am currently using Xcode 9.2 and Xcode 10.2

in Xcode 9.2 simctl is at /Applications/Xcode_9.2.app/Contents/Developer/usr/bin/simctl

this version links to

/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/CoreSimulator (compatibility version 1.0.0, current version 494.33.0)
@rpath/SimulatorKit.framework/Versions/A/SimulatorKit (compatibility version 1.0.0, current version 494.33.0)

in Xcode 10.2 instead simctl is at /Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl

this version links to

/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/CoreSimulator (compatibility version 1.0.0, current version 587.24.0)
/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Versions/A/Frameworks/CoreSimulatorUtilities.framework/Versions/A/CoreSimulatorUtilities (compatibility version 1.0.0, current version 587.24.0)

Now, when I xcode-select Xcode 9.2 the "old" (let's call it 9.2 version) version of simctl is called from xcrun and the opposite when I xcode-select Xcode 10.2

Since the linked version of CoreSimulator is the same and simctl 9.2 and simctl 10.2 both use a common subset of the CoreSimulator API (the other frameworks look inconsequential) __I believe this is a bug (or an arbitrary change) in simctl 10.2__. In fact unsurprisingly if I replace simctl 10.2 with a simlink to simctl 9.2 an xcode-select Xcode 10.2 carthage works just fine.

simctl 10.2 CoreSimulator API:

$ objdump -bind /Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl | grep CoreSimulator
/Library/Developer/PrivateFrameworks/CoreSimulator.framework/Resources/bin/simctl:  file format Mach-O 64-bit x86-64
__DATA   __got              0x1000432F8 pointer         0 CoreSimulatorUtilities _SimPrefKeyRuntimeImageCacheMaxSizeGB
__DATA   __got              0x100043300 pointer         0 CoreSimulatorUtilities _SimPrefKeyRuntimeImageCachePath
__DATA   __objc_const       0x10004A820 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_const       0x10004AC28 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x10004DCD8 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x10004DD68 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimNSPasteboard
__DATA   __objc_classrefs   0x10004DD88 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimPasteboardItem
__DATA   __objc_classrefs   0x10004DD58 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimServiceContext
__DATA   __got              0x100043268 pointer         0 CoreSimulator    _SimAudioHostUseSystemDefaultDeviceUID
__DATA   __got              0x100043270 pointer         0 CoreSimulator    _SimDeviceBootKeyDisabledJobs
__DATA   __got              0x100043278 pointer         0 CoreSimulator    _SimDeviceBootKeyEnvironment
__DATA   __got              0x100043280 pointer         0 CoreSimulator    _SimDeviceBootKeyPersist
__DATA   __got              0x100043288 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyArguments
__DATA   __got              0x100043290 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyEnvironment
__DATA   __got              0x100043298 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyStandardErrPath
__DATA   __got              0x1000432A0 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyStandardOutPath
__DATA   __got              0x1000432A8 pointer         0 CoreSimulator    _SimDeviceLaunchApplicationKeyWaitForDebugger
__DATA   __got              0x1000432B0 pointer         0 CoreSimulator    _SimDeviceNotificationType_BootStatus
__DATA   __got              0x1000432B8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyArguments
__DATA   __got              0x1000432C0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyBinPref
__DATA   __got              0x1000432C8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyEnvironment
__DATA   __got              0x1000432D0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStandalone
__DATA   __got              0x1000432D8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStderr
__DATA   __got              0x1000432E0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStdin
__DATA   __got              0x1000432E8 pointer         0 CoreSimulator    _SimDeviceSpawnKeyStdout
__DATA   __got              0x1000432F0 pointer         0 CoreSimulator    _SimDeviceSpawnKeyWaitForDebugger
__DATA   __got              0x1000433A0 pointer         0 CoreSimulator    _kSimDeviceNotification_NewBootStatus

simctl 9.2 CoreSimulator API

$ objdump -bind /Applications/Xcode_9.2.app/Contents/Developer/usr/bin/simctl | grep CoreSimulator
__DATA   __objc_const       0x10002F850 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x100030D00 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimDevice
__DATA   __objc_classrefs   0x100030D58 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimNSPasteboard
__DATA   __objc_classrefs   0x100030D80 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimPasteboardItem
__DATA   __objc_classrefs   0x100030D40 pointer         0 CoreSimulator    _OBJC_CLASS_$_SimServiceContext
__DATA   __got              0x10002A028 pointer         0 CoreSimulator    _SimAudioHostUseSystemDefaultDeviceUID
__DATA   __got              0x10002A030 pointer         0 CoreSimulator    _SimDeviceNotificationType_BootStatus
__DATA   __got              0x10002A0C0 pointer         0 CoreSimulator    _kSimDeviceNotification_NewBootStatus

[mdiep]

I think we need to suck it up and update our use of the JSON format.

These seemingly random JSON changes from the simulator team are extremely disappointing. This is 2nd Xcode release in a row that includes a breaking change to the JSON API—with no mention in the release notes and seemingly with no reason for it.

I assumed that the point of JSON output was easier consumption by tooling. But these breaking changes make it worse for tooling, not better. It makes me regret switching to the JSON output at all. We would have been better off parsing the normal output since it hasn't changed.

[Nonnus]

@mdiep from your words I assume we need to wait for a Carthage update to fix the issue and allow use of Xcode 10.2?

[mdiep]

Yup!

[drekka]

Thanks @blender I've downloaded the Carthage source with a view to taking a look, but I'm having problems working with it. Could be an issue with 10.2 as well. Who knows.

[qnoid]

@mdiep curious if you you know this to be as intended and not a regression. If so, is that a change in the spec of simctl?

Emphasis mine. According to simctl, the list command is used to "List available devices, device types, runtimes, or device pairs.".

AFAICS the devices are missing. Are devices meant to be inferred by the runtimes starting with Xcode 10.2?

Could you please elaborate @russbishop?

[dive]

Btw, it seems that Apple removed all previous simulator runtimes support from Xcode 10.2b. Perhaps, due to missing dynamically linked libraries for the Swift standard library. I tried to link them manually, but it didn't work to me. I didn't try hard though.
I mean, you'll see the following information if you switch to Xcode 10.2b with xcode-select and try to list the simulators:

$ xcrun simctl list devices --json | grep -A16 12.1
    "com.apple.CoreSimulator.SimRuntime.iOS-12-1" : [
      {
        "availability" : "(unavailable, runtime profile not found)",
        "state" : "Shutdown",
        "isAvailable" : false,
        "name" : "iPhone 5s",
        "udid" : "DDD36346-A76F-42E8-80F4-6F11E1EE4BEB",
        "availabilityError" : "runtime profile not found"
      },
      {
        "availability" : "(unavailable, runtime profile not found)",
        "state" : "Shutdown",
        "isAvailable" : false,
        "name" : "iPhone 6",
        "udid" : "21794717-BC89-45E4-9F57-8CF9D14A87D1",
        "availabilityError" : "runtime profile not found"
      },
...

No runtime for com.apple.CoreSimulator.SimRuntime.iOS-12-1 at all.

[tmspzz]

🥁

Apple Developer Relations
Please know that our engineering team has determined that this issue behaves as intended based on the information provided.
In order to better support pinning, this was the least disruptive option.

[qnoid]

What do they mean by pinning?

[dive]

I can reliably switch versions of simctl and reset the output by doing:

• sudo xcode-select -s /Applications/Xcode_10.1/Contents/Developer
• Running any app that boots the simulator
• xcrun simctl list devices --json

@blender, hmm, this solution doesn't work to me.

Since the linked version of CoreSimulator is the same and simctl 9.2 and simctl 10.2 both use a common subset of the CoreSimulator API (the other frameworks look inconsequential) I believe this is a bug (or an arbitrary change) in simctl 10.2. In fact unsurprisingly if I replace simctl 10.2 with a simlink to simctl 9.2 an xcode-select Xcode 10.2 carthage works just fine.

So, the only way is to re-link simctl explicitly? Even in case when you have Xcode 10.1 and Xcode 10.2?

[shinma]

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.

OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

[taher-mosbah]

Now that #2693 is merged does it make sense to have a release ?
or should we wait for finale version of Xcode 10.2 ?

[Nonnus]

hopefully we won't have to wait for any final version of Xcode 10.2 as this would be terribly disruptive to ongoing testing and development that should be done before the final release

[yapiskan]

This issue is currently breaking our buddybuild CI builds. Would be great if there is a release.

[jdhealy]

Yes, we plan to have a release well before the final version of Xcode 10.2…

[mvnevesbrb]

Please, do you have a solution for this problem ? Or can you point how can I compile the source, because some packages :

error: dependency graph is unresolvable; found these conflicting requirements:

Dependencies:
https://github.com/jdhealy/PrettyColors.git @ 5.0.0..<6.0.0
https://github.com/thoughtbot/Curry.git @ 4.0.0..<5.0.0
'Carthage' ... error: product dependency 'Curry' not found
'Carthage' ... error: product dependency 'PrettyColors' not found
'Carthage' ... error: product dependency 'Curry' not found

[mvnevesbrb]

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.

OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

how did you achieve to compile the code, can you help?

[shinma]

It's not that hard to change one line in Simulator.swift.

let platformName = "com.apple.CoreSimulator.SimRuntime.\(sdk.platform.rawValue)"

...and it works.
OR...

let allTargetSimulators = devices.filter { $0.key.stripping(prefix: "com.apple.CoreSimulator.SimRuntime.").hasPrefix(platformName) }

...with backward compatibility.

how did you achieve to compile the code, can you help?

• git checkout 0.31.2
• make boostrap
• open Carthage.workspace
• build using Xcode 10.1
• make install to replace brew version (after fix is released make uninstall and brew upgrade)

[giginet]

After merging #2693 and #2696 , I tried on my project.
It works both on Xcode 10.1 and 10.2.

Could anyone release a newer version? @Carthage/carthage

[jdhealy]

Fixes released with https://github.com/Carthage/Carthage/releases/tag/0.32.0

〜 Edit: Homebrew binary bottle released: 0.32.0.

Thanks @kmcbride and @DavidBrunow for submitting fixes!

Thank you to @giginet, @taher-mosbah, @ikesyo, and @blender for reviewing pull requests!

[kaushal9678]

if could not found simulator still persists after replacing following lines of code in runSimulator.js

if (!version.startsWith('iOS') && !version.startsWith('tvOS')) { continue; }

with this one

if ( !version.startsWith('com.apple.CoreSimulator.SimRuntime.iOS') && !version.startsWith('com.apple.CoreSimulator.SimRuntime.tvOS') ) { continue; }

comment following lines after replacing

if ( simulator.availability !== '(available)' && simulator.isAvailable !== 'YES' ) { continue; }
It will launch iPhone simulator without any issue.

and 2nd approach after updating to XCode 11

Xcode 11 xcrun returns true or false and not YES or NO as before.

Replace YES with ### true in following code
if ( simulator.availability !== '(available)' && simulator.isAvailable !== 'YES' ) { continue; }
like this
if ( simulator.availability !== '(available)' && simulator.isAvailable !== true ) { continue; }

[RustamKhisamov]

@blender unfortunately not. Completely removed my installation, reinstalled 10.1 and the problem still exists. xcrun simctl delete unavailable didn't seem to do much and recreating the simulators with Fastlane fastlane snapshot reset_simulators also didn't work (both before and after reinstalling Xcode).

simctl version: @(#)PROGRAM:simctl PROJECT:CoreSimulator-587.24

helps me with Xcode 11 beta

[rsalunga29]

If you are experiencing this in Xcode 11 and was surprised that iPhone X is not on the list of the new iOS 13.0 simulators, I think this should explain it all.

"Xcode no longer creates every available iOS simulator device by default. Instead a set of the most commonly used devices are created. To create other devices — or multiple instances of a device — open the Devices window, select Simulators, click the + button, enter a name, and select the relevant device type and OS version. In Terminal, execute the xcrun simctl create command, for example xcrun simctl create "My iPhone 7" "iPhone 7" iOS13.0. (49428617)"

Reference: https://developer.apple.com/documentation/xcode_release_notes/xcode_11_release_notes