Xamarin.forms: [Docs] Update docs to help users run Xamarin.Forms source against their own projects

[beeradmoore]

Description

Provide updated documentation for someone wanting to add Xamarin.Forms source projects to their own project in order to debug issues.

@PureWeen has a project here that does this but it doesn't currently use the recommended way with props files
https://github.com/PureWeen/ReactShell

The other thing to keep in mind is that users will have 3rd party libraries with transitive dependencies on XF so effort has to be made so that the source projects are used and you don't get exceptions about duplicate targets being imported

https://github.com/PureWeen/SmartHotel360-Mobile/blob/submodule/Source/SmartHotel.Clients/SmartHotel.Clients/SmartHotel.Clients.csproj#L51

Original Description

Currently having an issue within Xamarin.Forms/Xamarin.Forms.Platform.iOS/Renderers/PageContainer.cs and I am wanting to debug Xamarin.Forms to see why I am getting a null reference in there. Building XF will also be useful for making PRs in future so I'd prefer to focus on building XF now rather than focusing on that other issue.

I've tried to follow the steps from here

Steps to Reproduce

1. Download XF source.
2. Change to point to the release-4.1.0-sr1 tag.
3. Execute the build command.

Output:

$ ./build.sh --target vsmac
Downloading packages.config...
Downloading NuGet...
Feeds used:
/Users/somename/.nuget/packages/
https://api.nuget.org/v3/index.json

Restoring NuGet package Cake.0.33.0.
Adding package 'Cake.0.33.0' to folder '/Users/somename/git/Xamarin.Forms/caketools'
Added package 'Cake.0.33.0' to folder '/Users/somename/git/Xamarin.Forms/caketools'
Analyzing build script...
Processing build script...
Installing tools...
Installing addins...
The assembly 'Cake.Xamarin, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.26.1).
For best compatibility it should target Cake.Core version 0.33.0.
The assembly 'Cake.Android.Adb, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.26.1).
For best compatibility it should target Cake.Core version 0.33.0.
The assembly 'Cake.Git, Version=0.19.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.28.1).
For best compatibility it should target Cake.Core version 0.33.0.
Compiling build script...
Error: One or more errors occurred. (GitVersion: Process returned an error (exit code 1).)
GitVersion: Process returned an error (exit code 1).

Expected behaviour is I have some nice .nupkg files ready to add to my app. This may be confusion on my part though. So I continued with alternative steps for building.

1. Tried alternative of opening Xamarin.Forms.sln in Visual Studio 2019 and waiting for NuGet packages to be restored.
2. Execute the msbuild command
   > $ msbuild Xamarin.Forms.Build.Tasks/Xamarin.Forms.Build.Tasks.csproj
   The build was successful.
3. I am not setting any of the API keys that it lists, just wanting to build for iOS.
4. Build Xamarin.Forms.Platform.iOS.
5. Copied the 6 dll and pdb files from the Xamarin.Forms.Platform.iOS/bin/Debug/ directory (unsure if this is confusion on my end of not what I am meant to do)
6. Add Xamarin.Forms.Core.dll, Xamarin.Forms.Platform.dll and Xamarin.Forms.Platform.iOS.dll to my default XF project and attempt to build. 3 errors.

AssemblyInfo.cs(3,12,3,27): error CS0246: The type or namespace name 'XamlCompilationAttribute' could not be found (are you missing a using directive or an assembly reference?)
AssemblyInfo.cs(3,12,3,27): error CS0246: The type or namespace name 'XamlCompilation' could not be found (are you missing a using directive or an assembly reference?)
AssemblyInfo.cs(3,28,3,50): error CS0103: The name 'XamlCompilationOptions' does not exist in the current context

If I comment out [assembly: XamlCompilation(XamlCompilationOptions.Compile)] I now get

MainPage.xaml.cs(18,13,18,32): error CS0103: The name 'InitializeComponent' does not exist in the current context
App.xaml.cs(11,13,11,32): error CS0103: The name 'InitializeComponent' does not exist in the current context

I going back to XF I wondered if I was meant to do the "Build from the Command line" step instead now that I am set up.

1. Execute the given msbuild command for iPhone devices, as I am building for physical device only.

$ msbuild /restore /p:Platform=iPhone Xamarin.Forms.sln

32 Warning(s)
0 Error(s)

Time Elapsed 00:05:40.47

Full build log is here

At this point my previous DLLs didn't change and I don't have any .nupkg files so I am unsure what to do to be able to build and use XF from source.

Curent VS setup.

Visual Studio Community 2019 for Mac
Version 8.1.5 (build 9)
Installation UUID: 1374504d-0801-4cf3-a444-0b2c2c488e02
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 5.6.0.25 (d16-0 / 50f75273)

Package version: 518010028

Mono Framework MDK
Runtime:
Mono 5.18.1.28 (2018-08/223ea7ef92e) (64-bit)
Package version: 518010028

NuGet
Version: 5.0.2.5988

.NET Core
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
2.2.4
2.1.10
2.1.9
2.1.8
2.1.7
2.1.6
2.1.2
SDK: /usr/local/share/dotnet/sdk/2.2.203/Sdks
SDK Versions:
2.2.203
2.1.505
2.1.504
2.1.503
2.1.500
2.1.302
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.18.1/lib/mono/msbuild/Current/bin/Sdks

Xamarin.Profiler
Version: 1.6.10
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Updater
Version: 11

Xamarin.Android
Version: 9.3.0.23 (Visual Studio Community)
Commit: HEAD/d0b48056f
Android SDK: /Users/bradmoore/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
5.0 (API level 21)
6.0 (API level 23)
7.0 (API level 24)
7.1 (API level 25)
8.0 (API level 26)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.1
SDK Build Tools Version: 27.0.3

Build Information:
Mono: mono/mono/2018-08@3a07bd426d3
Java.Interop: xamarin/java.interop/d16-1@5ddc3e3
LibZipSharp: grendello/LibZipSharp/d16-1@44de300
LibZip: nih-at/libzip/rel-1-5-1@b95cf3f
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.27.1@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-1@acabd26

Microsoft Mobile OpenJDK
Java SDK: /Users/bradmoore/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Android Device Manager
Version: 1.2.0.44
Hash: aac645b
Branch: remotes/origin/d16-1
Build date: 2019-05-29 19:55:24 UTC

Apple Developer Tools
Xcode 10.2.1 (14490.122)
Build 10E1001

Xamarin.Mac
Version: 5.10.0.157 (Visual Studio Community)
Hash: 6bd94753
Branch: d16-1
Build date: 2019-06-12 17:28:48-0400

Xamarin.iOS
Version: 12.10.0.157 (Visual Studio Community)
Hash: 6bd94753
Branch: d16-1
Build date: 2019-06-12 17:28:47-0400

Xamarin Designer
Version: 16.1.0.467
Hash: f1657e133
Branch: remotes/origin/d16-1-new-document-model
Build date: 2019-06-18 21:57:42 UTC

Xamarin Inspector
Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1

Build Information
Release ID: 801050009
Git revision: bd0ab28ba941b19b39322247db020dcd0fb305d0
Build date: 2019-07-03 17:15:21+00
Build branch: release-8.1
Xamarin extensions: 8cc25b5cb090e6c23b62a7901000c299977eb08d

Operating System
Mac OS X 10.14.5
Darwin 18.6.0 Darwin Kernel Version 18.6.0
Thu Apr 25 23:16:27 PDT 2019
root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64

Enabled user installed extensions
MFractor 3.8.9
MonoGame Extension 3.7.1.189

[jfversluis]

Hey @beeradmoore thank you for this question!

I’m sorry you had to spend some time on this and still not get it to work. I can see how frustrating that is. From what I got in private messages from you and see in this question is that you want to do basically two things:

• Get XF to build
• Get DLLs/NuGets from it that you can add to your project for debugging

Get XF to build

As discussed in some private messages, ultimately you got XF to build from both the command-line and VSMac, so we’re good there.

Debug your own application

(I have thrown together a couple of things I already mentioned to you privately and tried to put it into an ongoing story)

So, debugging something of your own is a bit of a pickle. This would mean you either need to change something in Xamarin.Forms code, create DLLs, reference that, run your application and see if whatever you did helped. _Or_ get your app into the XF solution, still update the references and be able to debug through the source directly.

You should be able to just reference some the right csproj files into you own solution as well. But I remember that I have tried that at some point and didn’t get it to work directly. But you should be able to get that to work as well.

There are two other things you could do, the first one is not an option for your specific problem in this case I think.

1. Create your own renderer, copy the code you need from the XF repo so it’s in your project and debug through it with that. But this is not a typical renderer but something deeper with accessibility in a PageContainer.

2. Reproduce your problem and add it the Control Gallery app and debug it through there. If you do that, it will also be easier to make it into a PR for us. But that might just be wishful thinking on our part 😉

Please give it another go and if you run into anything please let us know here or to me directly and we’ll get a timeslot to get on a call or something.

Thank you for your patience!

[beeradmoore]

I tried a few things. I've gone back to attempt to build for iPhoneSimulator instead of iPhone for the following steps.

First thing was to add various XF csproj files into my blank forms app project and fixing the references. I didn't have much luck. I'll try repro and get that on Github so you can see if I did anything silly.

The second attempt was adding a new blank forms app to the XF source and then referencing it in my projects. I even copied across most of the properties for the ControlGallery projects to mine (via editing csproj directly) to ensure they match. Same result as above essentially. I'll also try repro this and get my project up so for confirmation that I was attempting to do this the correct way.

I was able to build Xamarin.Forms.ControlGallery.iOS and deploy to my iOS simulator. This will be useful later if I was to work on a new feature on say, the Button class. But as I was not able to reproduce the initial null reference bug outside of my main project then being able to build the ControlGallery wont help me with that.

The next thing I tried was to build XF to dlls and then reference those in my project. I did upload that as its own project, https://github.com/beeradmoore/BlankXamarinFormsAppFromSource.
The readme has information of what I did to build the dlls. This resulted in the same 4 errors I had yesterday (and from memory the same 4 errors I had with the first two methods mentioned above that I tired today). It seems the XAML custom build tool isn't attempting to run, and InitializeComponent is not found. Both issues may be related?

I'll try next with a custom renderer so I can at least go on with debugging and fixing the issue I am facing for work.

[PureWeen]

@beeradmoore

I'm thinking there are two ways you can try to go about this

1) here are a couple projects setup using submodules you can check out that might help you with your structure
https://github.com/PureWeen/ReactShell
https://github.com/PureWeen/SmartHotel360-Mobile/tree/submodule

These look to match what you are trying to do with your source above

2) Unlike the projects above the Control Gallery project uses a global props/targets file which I'm guessing is why your XAML files aren't being processed

I even copied across most of the properties for the ControlGallery projects to mine (via editing csproj directly) to ensure they match.

To model your project after the Control Gallery you'll also need to include the global props and targets files.
https://github.com/xamarin/Xamarin.Forms/blob/master/Directory.Build.props
https://github.com/xamarin/Xamarin.Forms/blob/master/Directory.Build.targets
https://github.com/xamarin/Xamarin.Forms/blob/master/Xamarin.Forms.Controls/Directory.Build.props
https://github.com/xamarin/Xamarin.Forms/blob/master/Xamarin.Forms.Controls/Directory.Build.targets

For personal projects I've had more luck with the first method which you can checkout my projects above and probably replicate to what I have to what you have

[PureWeen]

It looks like your csproj file is missing the props/targets needed to process the Xaml files

https://github.com/PureWeen/ReactShell/blob/master/Source/SmartHotel.Clients/SmartHotel.Clients/Shopanizer.csproj#L11-L12

https://github.com/PureWeen/ReactShell/blob/master/Source/SmartHotel.Clients/SmartHotel.Clients/Shopanizer.csproj#L49-L50

https://github.com/PureWeen/ReactShell/blob/master/Source/SmartHotel.Clients/SmartHotel.Clients/Shopanizer.csproj#L30-L37

Your cross platform file doesn't have any of the props or targets needed to process the XAML files
https://github.com/beeradmoore/BlankXamarinFormsAppFromSource/blob/master/BlankXamarinFormsApp/BlankXamarinFormsApp/BlankXamarinFormsApp.csproj

[beeradmoore]

Thanks @PureWeen, that fixed my issues! I've updated my repo with new samples. One sample builds dlls and has them referenced in the individual projects. The other adds the csproj files into the solution and references those in the individual projects.

The key was indeed adding

<Import Project="..\..\Xamarin.Forms\.nuspec\Xamarin.Forms.DefaultItems.props" />
<Import Project="..\..\Xamarin.Forms\.nuspec\Xamarin.Forms.DefaultItems.targets" />
<Import Project="..\..\Xamarin.Forms\.nuspec\Xamarin.Forms.targets" Condition="'$(BuildingInsideVisualStudio)' == 'true'" />
<Import Project="..\..\Xamarin.Forms\.nuspec\Xamarin.Forms.targets" Condition="'$(BuildingInsideVisualStudio)' != 'true'" />

to my solution. Although I am confused why these last two lines reference != true and == true, I'll investigate later if I can just do away with those and make it one line to be simpler, and reference Xamarin.Forms.targets and Xamarin.Forms.props instead of the DefaultItems props/targets to make this come down to 2 lines.

[PureWeen]

Awesome @beeradmoore !! We're discussing the best way to guide users more efficiently on this topic and we'll get the docs updated with the results.

I'm glad you got it working.

[adrianknight89]

5815 We still need help to be able to create our own custom Nugets.

[kvpt]

@adrianknight89 With cake integration it's now easy.

You have the instruction in the cake file.
https://github.com/xamarin/Xamarin.Forms/blob/2b3b66a5e890c435b87c190d5b86c475b9a38d27/build.cake#L9-L15

It create an artifacts directory where the nupkg is dropped.

However it miss instruction for using the build.sh entry point for non Windows system (perhaps you can use PowerShell Core on non Windows too).

But I agree with you that the instructions are not very easy to found.

[jfversluis]

@adrianknight89 I think I can predict the answer but please humor me. Why do you want to create nugets?

Asking to see if I get it right and see if we can get out with another route/documentation

[adrianknight89]

@kvpt Thanks. I didn't know about the cake integration. ./build.ps1 -Target NugetPack seems to do the trick. Is this the right way to build nugets though? I'd like to see the official recommendation by Xamarin. Wondering if we need to tweak anything prior to running the script.

@jfversluis Sometimes PRs sit in the queue for 6 months to over a year. I fear that when I go to production, I might have an issue that needs fixing but I can't get the fix right away, hence the need for a custom build. Also, sometimes we need to test our projects before submitting PRs. :)

[PureWeen]

Playing around with different options here for loading sources into a project

https://github.com/PureWeen/Xamarin.Forms.Sandbox

If a project is setup with references inside an external props/targets set of files then this method should make it viable to swap the source code in and out of your project.

Feel free to fork and PR if there are areas which you want to cleanup

[kvpt]

Thank you very much @PureWeen for this, lost so much time in the past to test my app with the non nuget source. If I need to do it in the future I will use your bootstap project.

[PureWeen]

I also added some brief instructions how to upgrade an existing project.

Mileage may vary though :-)