Xamarin-macios: Can we detect the command line tools weren't installed?

[VincentDondain]

Problems occur when after downloading and using in the IDEs a new Xcode you didn't open it. I'm opening this issue to gather comments on how to address this, if only it's possible.

From Apple's technical note Xcode includes all of the command-line tools. If it is installed on your system, remove it to uninstall the command-line tools.

I tried completely removing Xcode and it didn't re-ask me to install the command line tools though.

However it's still possible there's something in the Xcode app bundle we could check.

[rolfbjarne]

There are a few answers here: https://stackoverflow.com/questions/15371925/how-to-check-if-command-line-tools-is-installed

We can easily check if the Xcode license has been accepted: https://github.com/xamarin/xamarin-macios/blob/34af7112b56adffc0777989e4c46d61f1e3ad2c4/system-dependencies.sh#L333

We might also want to check if the additional tools are installed too, but that's a bit more complicated:

1. List the tools to install:

   $ ls /Applications/Xcode.app/Contents/Resources/Packages
   MobileDevice.pkg        MobileDeviceDevelopment.pkg XcodeExtensionSupport.pkg   XcodeSystemResources.pkg
   

2. Iterate over each package, and get the package id and its version. pkg files are xar archives, and there's a top-level PackageInfo file inside the xar archive that contains this information:

   $ xar -t  -f /Applications/Xcode.app/Contents/Resources/Packages/MobileDevice.pkg
   Bom
   PackageInfo
   Payload
   Scripts
   

3. Extract the PackageInfo file. This is annoyingly complex, because the xar command-line tool doesn't allow you to select a single file to extract... but you can exclude files:

   $ xar -f /Applications/Xcode.app/Contents/Resources/Packages/MobileDevice.pkg -x --exclude Bom --exclude Payload --exclude Scripts
   $ cat PackageInfo 
   <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
   <pkg-info format-version="2" relocatable="true" overwrite-permissions="true" identifier="com.apple.pkg.MobileDevice" useHFSPlusCompression="true" auth="root" version="4.0.0.0.1.1509560275">
     [...]
   

   The list of files to exclude should probably be constructed first using xar -t -f /file/to/package.pkg to list the actual files in the package instead of hardcoding them.

4. Get the identifier and the version from the PackageInfo file:

   $ csharp -e 'var xml = new System.Xml.XmlDocument (); xml.Load ("PackageInfo"); Console.WriteLine (xml.SelectSingleNode ("//pkg-info").Attributes ["identifier"].Value);'
   com.apple.pkg.MobileDevice
   $ csharp -e 'var xml = new System.Xml.XmlDocument (); xml.Load ("PackageInfo"); Console.WriteLine (xml.SelectSingleNode ("//pkg-info").Attributes ["version"].Value);'
   4.0.0.0.1.1509560275
   

5. Query the system to see what's installed using the identifier we found:

   $ pkgutil --pkg-info com.apple.pkg.MobileDevice
   package-id: com.apple.pkg.MobileDevice
   version: 4.0.0.0.1.1521089107
   volume: /
   location: 
   install-time: 1522915669
   groups: com.apple.findsystemfiles 
   

6. Finally check the version of the installed package (if installed) versus the version in the package.

[dalexsoto]

From @mvirkkunen (https://github.com/xamarin/xamarin-macios/issues/3921)

I just spent many frustrating hours debugging why a project stopped building with completely nonsensical error messages such as:

ArgumentException: Failed to parse PList data type: 
ArgumentException: Must specify valid information for parsing in the string.

Further investigation revealed that ibtoold kept segfaulting on the Mac build host, and I assume that caused something to generate an invalid plist file, which Xamarin's tools then failed to parse. In the end it turned out that somebody had turned on auto-updates on the Mac host and Xcode had been updated to version 9.3 which broke everything. The installed version isn't immediately obvious either, so even if you open Xcode you might not notice what version you're actually running. Everything started working after updating all Xamarin tools.

If a minor version update to Xcode can break things this badly, I think it'd be nice if the build tools checked the installed version of Xcode and issued a big fat warning if the version is newer than what is known to be compatible!

[jstedfast]

The problem isn't that the msbuild tools aren't compatible with a newer Xcode, it's that the command-line tools weren't (properly?) installed.

[rolfbjarne]

Just found out something simpler:

man xcodebuild shows:

           Check if any First Launch tasks need to be performed.

     -runFirstLaunch
           Install packages and agree to the license.

[VincentDondain]

No way!! Well I should have read the entire manual :P

[jstedfast]

Should we invoke this (perhaps in DetectSdkLocations task) in msbuild?

What happens if the user is building on a build host?

[VincentDondain]

DetectSdkLocations sounds like the right place to me however we'll likely need to surface more than a build error (e.g: solution pad tooltip) so it's more visible to the user. However something is better than nothing.

Note: I can't get -checkFirstLaunchStatus to return anything but I don't have an Xcode that needs to install new command lines so... If anyone knows how to remove the Xcode command line so it prompts you to install them again (and make this command work) let me know.

[rolfbjarne]

@VincentDondain if you haven't installed the latest Xcode beta, you can try that (if not, wait for the next one :smile:)

[VincentDondain]

@rolfbjarne just tried on the new Xcode 9.4 beta 2. /Applications/Xcode94-beta2.app/Contents/Developer/usr/bin/xcodebuild -checkFirstLaunchStatus does nothing for me but sudo /Applications/Xcode94-beta2.app/Contents/Developer/usr/bin/xcodebuild -runFirstLaunch worked.

Install Started
1%.........20.........40.........60.........80......Install Succeeded

[rolfbjarne]

@VincentDondain did you check the exit status of -checkFirstLaunchStatus?

/Applications/Xcode94-beta2.app/Contents/Developer/usr/bin/xcodebuild -checkFirstLaunchStatus
echo $?

I think that should show 1 if something needs to be done, and 0 otherwise.

[rolfbjarne]

Actually it returns 69:

> /Applications/Xcode94-beta2.app/Contents/Developer/usr/bin/xcodebuild -checkFirstLaunchStatus
> echo $?
69
> /Applications/Xcode93.app/Contents/Developer/usr/bin/xcodebuild -checkFirstLaunchStatus
> echo $?
0

[VincentDondain]

Ah no I thought it would just output, it works then we can totally use that.