Runtime: QUESTION: Enabling New OS for CoreFX

[cydhaselton]

Currently working on the CoreFX section of this issue and I've parsed the following issues for information…

• dotnet/corefx#6770
• 
  
  • dotnet/coreclr#917 (specifically this comment)
    
    …but need additional clarification/information:
  
  

1. Are there published (gist or issue) steps for what is needed for a PR enabling CoreFX for a new platform?
2. Should I be able to generate Android-specific nuget packages for CoreFX as an end goal or is there additional work needed?
3. The main user doing the bulk of the work to bring up CoreFX/dotnet for Alpine was deleted; is there an Alpine "team" or maintainer whom I could contact?
4. Who are the maintainers for packaging and other bits of the CoreFX repo (I'll create a PR for a CODE_OWNERS.TXT file along the lines of this with the info provided)

cc @janvorli, @jkotas and @schellap

[jkotas]

1. Search for tizen in sources. Tizen is similar kind of Linux as Alpine or Android, so it will give you about the righ list of places to modify.

2. The complete .NET Core bundle is produced in http://github.com/dotnet/core-setup repo, so you need that one too. @ellismg has been working on "build from source" script that is meant to allow you to build the whole stack. It may be useful for what you are trying to do.

3. None that I am aware of.

4. area-Infrastructure under https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/issue-guide.md

BTW: @schellap is no longer on the .NET team.

cc @Petermarcu

[cydhaselton]

Thanks @jkotas

The complete .NET Core bundle is produced in http://github.com/dotnet/core-setup repo, so you need that one too.

I was asking specifically about enablement of Android for the CoreFX repo but are you saying that core-setup is needed as part of bringing up CoreFX for Android (and/or other platform)?

[jkotas]

Yes - if you want it to work same way as it works on existing platforms.

[cydhaselton]

@jkotas Thanks. So port core-setup packaging before corefx?

[jkotas]

I think it is better to do corefx first and core-setup second.

[cydhaselton]

@jkotas Thanks. Unfortunately, after modifying the sources accordingly, I'm getting the following error when running the ./build-managed.sh part of the build.

/build/core/corefx/Tools/depProj.targets(83,5): error : Error no assets were resolved from NuGet packages. [/build/core/corefx/external/runtime/runtime.depproj]

I'm running the following for the managed build:

CONFIG_DIR=`realpath cross/android/arm64` ROOTFS_DIR=`realpath /build/core/corefx/cross/android-rootfs/toolchain/arm64/sysroot` ./build-managed.sh -debug -verbose -skiptests -BuildPackages=false -buildArch=arm64

Are there any dependencies I'm overlooking?

[cydhaselton]

Copying @mellinoe and @ericstj on the above question
The entire build-managed.log is too large to pastebin; I can post excerpts if necessary.

[cydhaselton]

Polite re-ping to @mellinoe, @jkotas, @Petermarcu and @ericstj. Specifically I'd like to know if the build error I encountered is due to a) incorrectly modified corefx sources or b) an external dependency that was not found…so that I know where to focus my efforts.

[jkotas]

The project under corefx/external/runtime downloads NuGet package with CoreCLR to build CoreFX against. It fetches it from https://dotnet.myget.org/gallery/dotnet-core NuGet feed by default. There is no Android CoreCLR package published in this feed today, and that's why you are getting this error.

Here are some options that you can use to make progress:

• Override the Android build to use regular Linux runtime package. There are number of similar overrides to support various cross-building scenarios: https://github.com/dotnet/corefx/blob/master/external/runtime/runtime.depproj#L5
  or
• Pass the path to your local Android CoreCLR to corefx build using CoreCLROverridePath: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/developer-guide.md .

[cydhaselton]

Thanks @jkotas
As part this issue I'll need to know the process of getting Android CoreCLR packages published to the dotnet-core feed if appropriate. I have the following related questions:

1. Is there documentation for the process of getting packages published to the dotnet-core feed.
2. If not does the following look about right?

• Enable CoreCLR package build for Android (done)
• Configure pipeline build for Android
• configure CI builds for Android

[Petermarcu]

/cc @weshaggard

Yes, in the current infrastructure, it would be to update the pipeline build to do an android leg. That build is the one that builds packages that go to myget. The CI builds wouldn't publish packages.

[cydhaselton]

Thanks @Petermarcu

Yes, in the current infrastructure, it would be to update the pipeline build to do an android leg. That build is the one that builds packages that go to myget. The CI builds wouldn't publish packages.

I believe I know the answer but I'll ask anyway: Is there any documentation on the process for setting this up for a new OS? I'm currently running a search in the corefx/coreclr repos then picking through the results...which is tedious.

[Petermarcu]

@weshaggard @wtgodbe should be able to help give guidance.

[wtgodbe]

There isn't documentation for this, but I would suggest cloning CoreClr's Linux build (https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build?_a=summary&buildId=851825) and modifying it to run the appropriate commands for your Android build. The current Linux builds run in Docker containers, which may not be applicable for Android - If you already know how to run for Android on top of Linux, you can replace those steps as appropriate (it may also be easier to just clone the mac build - https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build/index?definitionId=5969&_a=completed. That build doesn't use docker, so it may be more straightforward to just edit the build steps, then surround them with any necessary android-related steps). After doing that, @chcosta has a tool you can use to convert that build definition into a .json file. The last step is to add a block in pipelines.json to run your new job, like https://github.com/dotnet/coreclr/blob/master/buildpipeline/pipelines.json#L13-L25.

[jkotas]

@wtgodbe The links you have shared are accessible to Microsoft employees only. @cydhaselton is not Microsoft employee.

@cydhaselton We have been adding the platform to the official "pipeline" build only after the platform became mature enough and generally working and useful. The build machines, storage accounts and other infrastructure for official builds is "closed source" - it is maintained and paid for by Microsoft. We are open to sponsoring community supported ports, but they have to reach certain level before we add official builds for them.

[cydhaselton]

Thanks everyone for the feedback and info. Given the requirements for the official pipeline build and that I'm probably the only one working the Android port at the moment I'm going to go forward with assumption Android won't be an official build.

Knowing this, I have another (hopefully final) set of questions:

1. Are there examples of bring-ups (i.e. open issues, PRs) for unofficial builds?
2. Is there guidance for using/building a complete .NET Core for a new, unofficial platform? If it helps to answer this question, both the coreclr and corefx repos can be built for Android. My goal is to get PowerShell running on same.

Copying @jkotas @Petermarcu @wtgodbe on the above.

[Petermarcu]

@cydhaselton , I have this on my todo list and will try to get you some answers soon. We've been having similar conversations with folks that want to bring Alpine up again. I'd like to make this easier and at least work towards having a checklist of how this can be done.

Are you saying you have android builds of coreclr and corefx? You will need at least the host from the core-setup repo. Once you have those things, it would be pretty easy to construct a working runtime folder that could be used to run an app.

[cydhaselton]

@Petermarcu Both CoreFX and CoreCLR can be built for Android-21. Additionally a PR has been merged to the CoreCLR repo that allows package builds for Android.

You will need at least the host from the core-setup repo.

Are you talking about a native build of this repo?

[Petermarcu]

@cydhaselton yes. Thats where dotnet.exe and hostpolicy.dll are built which know how to launch .NET Core apps.

[cydhaselton]

@Petermarcu Ok I'll see what I can do to get core-setup building for android.
Last question for now: where do package builds fit in with this…i.e. are they necessary for a folder-based working runtime?

[Petermarcu]

The only complication I believe will be with Microsoft.NETCore.App.deps.json. That may need a little "hacking" but otherwise, the folder based layout should work. You need packages to support building a "self-contained" application but the packages aren't the ones from coreclr or corefx, there is a single package from core-setup that you would need that provides the runtime for android. We'll have to figure out where to plug android into the RID graph as well.

What you want in the end is to be able to replicate these files/folders:

dotnet/
       dotnet
       host/*
       shared/*

Those will let you run a basic netcoreapp2.0 application. sdk will get you tooling running on android. I'm not sure if thats a goal at all for you. store would be required as an optimization for ASP.NET Core applications but with a settings change in your project, that can just get put with your application. The only native component there that I'm aware of that you'd need to get building for android would be libuv.

[cydhaselton]

@Petermarcu So I've finally managed to get the following cross-built for Android:

ubuntu@ip-172-31-42-205:/build/core/core-setup$ ls Bin/linux-arm64.Debug/corehost/
total 44M
-rwxrwxr-x 1 ubuntu ubuntu 9.1M Jul 19 00:04 apphost*
-rwxrwxr-x 1 ubuntu ubuntu 9.2M Jul 19 00:04 dotnet*
-rwxrwxr-x 1 ubuntu ubuntu  13M Jul 19 00:04 libhostfxr.so*
-rwxrwxr-x 1 ubuntu ubuntu  14M Jul 19 00:05 libhostpolicy.so*

However the build throws 20 warnings and fails after with the following error:

/build/core/core-setup/src/pkg/projects/Microsoft.NETCore.App/src/Microsoft.NETCore.App.depproj(97,5): error : Required property _runtimePackagePath was not set.

Is this due to not having a plug in location for the android RID or something else?

[Petermarcu]

@weshaggard any thoughts? We need to make sure people can build netcoreapp for their own RIDs.

[weshaggard]

@cydhaselton see https://github.com/dotnet/core-setup/blob/3009b12355fb5829681928aa343f81f2020812d9/src/pkg/projects/Microsoft.NETCore.App/src/Microsoft.NETCore.App.depproj#L56 it looks like it didn't find a coreclr for your build. Did you get any issues during restore?

[cydhaselton]

@weshaggard No issues during restore. Can ReferenceCopyLocalPaths be specified in the ./run.sh build-native command or do I need to modify the code?

@Petermarcu should this issue be moved/re-opened in dotnet/core-setup?

[cydhaselton]

@Petermarcu In response to earlier reply:

Those will let you run a basic netcoreapp2.0 application.

Can I assume this includes PowerShell?

sdk will get you tooling running on android. I'm not sure if thats a goal at all for you.

There was a discussion/blog post about this somewhere and unfortunately I don't remember where, but I believe the opinion was that running tooling on android would be inefficient at best and unusable at worst. I'm sure there are people who would use SDK tooling (which I assume means msbuild, and everything else needed to build .NET programs at the commandline) on Android...after all there are apps that provide complete GCC and Clang toolchains...but my goal is primarily to get PowerShell running on Android.

[weshaggard]

@weshaggard No issues during restore. Can ReferenceCopyLocalPaths be specified in the ./run.sh build-native command or do I need to modify the code?

That should be setup from the restore. Can you do a a build of the Microsoft.NETCore.App.depproj with a diag log and share the log so I can have a look?

@Petermarcu should this issue be moved/re-opened in dotnet/core-setup?

It it turns out we need to do work in core-setup then yes we should file an issue there but if it is just figuring out what to do we can continue the conversation here.

Can I assume this includes PowerShell?

Yes assuming there are no other platform specific libraries that powershell depends on that need to be ported as well.

[cydhaselton]

@weshaggard

That should be setup from the restore. Can you do a a build of the Microsoft.NETCore.App.depproj with a diag log and share the log so I can have a look?

I can. Before I do, should the restore be pulling from a local set of packages or a feed?

[weshaggard]

It is my understanding that you are producing a new runtime which isn't published so I would expect your packages would be coming locally. Another option is to share with me a branch of your changes and how you are building so I can try to repro myself.

[cydhaselton]

@weshaggard Here's my core-setup branch https://github.com/cydhaselton/core-setup/tree/android-cross-build

Will re-run the build with a diag.log in the meantime and attempt to use the ReferenceCopyLocalPaths option

[cydhaselton]

@weshaggard and/or @Petermarcu a better question, is there doc or gist similar to this for core-setup.

@weshaggard You mentioned that ReferenceCopyLocalPaths should be set from restore and that the packages should be coming locally. I'm fairly certain they;re not…where is that option set?

Still working at getting log

[cydhaselton]

@weshaggard: Debug logs posted. This one is from run.sh build-native and this one is from run.sh build-custom (see next comment)
Additional note: Using the build instructions from here, specifically run build-custom -verbose -OSGroup=Linux -- src/corehost/build.proj /p:TargetRid=ubuntu.16.04-x64 runs without errors. However the Bin/ folder is created as android.21-arm64.Debug/ instead of linux-arm64.Debug/

[cydhaselton]

@jkotas

• Override the Android build to use regular Linux runtime package. There are number of similar overrides to support various cross-building scenarios: https://github.com/dotnet/corefx/blob/master/external/runtime/runtime.depproj#L5
  or
• Pass the path to your local Android CoreCLR to corefx build using CoreCLROverridePath: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/developer-guide.md .

After trying several times to implement option 1, I ran the build with the CoreCLROverridePath set. Unfortunately it failed with the same error:

/build/core/corefx/Tools/depProj.targets(83,5): error : Error no assets were resolved from NuGet packages. [/build/core/corefx/external/runtime/runtime.depproj]

[weshaggard]

@weshaggard and/or @Petermarcu a better question, is there doc or gist similar to this for core-setup.

Not that I'm aware of.

@weshaggard: Debug logs posted. This one is from run.sh build-native and this one is from run.sh build-custom (see next comment)
Additional note: Using the build instructions from here, specifically run build-custom -verbose -OSGroup=Linux -- src/corehost/build.proj /p:TargetRid=ubuntu.16.04-x64 runs without errors. However the Bin/ folder is created as android.21-arm64.Debug/ instead of linux-arm64.Debug/

The problem is because we don't have any linux-arm64 binaries for the runtime which is why it doesn't find the coreclr. Are you building a private version of the runtime that has the linux-arm64 binaries? If so that is the version of the coreclr you need to be referencing in your core-setup builds. This property https://github.com/dotnet/core-setup/blob/master/dependencies.props#L21 will need to match the version of your private package and you will need to make sure you setup a local feed/directory with your packages and set the OverridePackageSource property to point at it (https://github.com/dotnet/core-setup/blob/master/dir.props#L98).

[cydhaselton]

@weshaggard I'm not building a private version or changing the version number; I'm using an unofficial RID generate nuget packages for coreclr. If I'm not understanding you correctly let me know.

I'll re-try the core-setup build specifying that property.

[weshaggard]

We don't currently produce linux-arm64 packages from our coreclr builds (https://github.com/dotnet/coreclr/blob/master/src/.nuget/dir.props#L135), so that RID doesn't exist and will need to be built before you can consume it in core-setup.

edit: Looks like we have a PR to produce that RID https://github.com/dotnet/coreclr/pull/12372, once it goes through and we publish a package for it that should fix your issue.

[cydhaselton]

@weshaggard Thanks for clarifying. Just making sure I understand; it's possible to build coreclr for Linux.arm64.Debug, but not package it?

[weshaggard]

That is correct until PR https://github.com/dotnet/coreclr/pull/12372 goes through.

[cydhaselton]

@weshaggard But if I have a private build of a coreclr linux-arm64 package, I should be able to use it and run a successful build by specifying /p:OverridePackagePath=/path/to/Product/Linux.arm64.Debug?

[weshaggard]

Yes in theory which is what I was asking/suggesting earlier.

[cydhaselton]

@weshaggard: Sorry, difficult keeping suggestions for corefx and core-setup straight.
I mentioned here that I did use the /p:OverridePackagePath with my local /path/to/bin/Product/Linux.arm64.Debug during the core-setup build; unfortunately I get the same error mentioned earlier.

Build command:

CONFIG_DIR=`realpath cross/android/arm64` ROOTFS_DIR=`realpath /build/core/core-setup/cross/android-rootfs/toolchain/arm64/sysroot` ./run.sh build -verbose -OSGroup=Linux -CrossBuild=True -DistroRid=android.21-arm64 -SkipTests=True -Architecture=arm64 -TargetArchitecture=arm64 -- /p:TargetRid=android.21-arm64 /p:OverridePackageSource=/build/core/coreclr/bin/Product/Linux.arm64.Debug

Error:

/build/core/core-setup/src/pkg/projects/Microsoft.NETCore.App/src/Microsoft.NETCore.App.depproj(97,5): error : Required property _runtimePackagePath was not set.

[weshaggard]

Are the nupkg files in /build/core/coreclr/bin/Product/Linux.arm64.Debug I think they are under /build/core/coreclr/bin/Product/Linux.arm64.Debug/.nuget/pkg.

[cydhaselton]

@weshaggard They're in /build/core/coreclr/bin/Product/Linux.arm64.Debug/.nuget/pkg, but the docs specifies /build/core/coreclr/bin/Product/Linux.arm64.Debug so I assumed the build had code to locate the package dir.

I'll try with .nuget/pkg added

[cydhaselton]

@weshaggard Same error. I don't know if it matters but the error is always thrown immediately after this part in the build

Microsoft.Win32.Registry -> /build/core/corefx/bin/Unix.AnyCPU.Debug/Microsoft.Win32.Registry/netcoreapp/Microsoft.Win32.Registry.dll

If it helps, this is the warning thrown after the error:

/build/core/core-setup/Tools/packageresolve.targets(86,5): warning : The project '/build/core/core-setup/Bin/obj/android.21-arm64.Debug/Microsoft.NETCore.App/Microsoft.NETCore.App.depproj' was referenced more than once in the ProjectReferencesCreatingPackages property. Ignoring all but the first. [/build/core/core-setup/src/pkg/projects/Microsoft.NETCore.App/src/Microsoft.NETCore.App.depproj]

[weshaggard]

Sorry for not thinking about this earlier but did you update the version of the runtime package you are using in your branch? It should match the version you have in our local build. You will need to update the version here https://github.com/dotnet/core-setup/blob/release/2.0.0/dependencies.props#L22

[cydhaselton]

@weshaggard Updated the property in core-setup and corefx. The managed build still fails in corefx with the same error, but in a different part of the build, but core-setup is failing with a different error

core-setup:

/build/core/core-setup/src/pkg/projects/Microsoft.NETCore.App/src/Microsoft.NETCore.App.depproj(97,5): error : Required property _fxLibDirectory was not set.

corefx:

/build/core/corefx/Tools/dotnetcli/sdk/2.0.0-preview1-005977/NuGet.targets(97,5): warning : Dependency specified was Microsoft.NETCore.Runtime.CoreCLR (>= 2.1.0-preview2-25527-0) but ended up with Microsoft.NETCore.Runtime.CoreCLR 2.1.0-preview2-25527-03. [/build/core/corefx/external/runtime/runtime.depproj] [/build/core/corefx/external/runtime/runtime.depproj]                                                               

/build/core/corefx/Tools/depProj.targets(83,5): error : Error no assets were resolved from NuGet packages. [/build/core/corefx/external/runtime/runtime.depproj]

[weshaggard]

: error : Required property _fxLibDirectory was not set.

That is an indication that you don't have the matching linux-arm64 package from corefx which probably isn't built yet. We are going to need to get that building first. Lets focus on the corefx build until we get it working.

warning : Dependency specified was Microsoft.NETCore.Runtime.CoreCLR (>= 2.1.0-preview2-25527-0) but ended up with Microsoft.NETCore.Runtime.CoreCLR 2.1.0-preview2-25527-03. [/build/core/corefx/external/runtime/runtime.depproj] [/build/core/corefx/external/runtime/runtime.depproj]

That tells me that it didn't find your package. Are you passing the OverridePackagePath to option to corefx build? https://github.com/dotnet/corefx/blob/master/dir.props#L68.

[cydhaselton]

@weshaggard I was using CoreCLROverridePath as per build instructions here. Managed build with OverridePackagePath parameter is running now

[cydhaselton]

@weshaggard same error with corefx build using OverridePackagePath. Retrying with OverridePackageSource

[weshaggard]

OK I've been getting this issue as well as another issue (https://github.com/dotnet/corefx/issues/22509) mixed up in my head. To make sure we are on the same page let me ask a few questions:

• For your work are you based off of master?
• I assume you have successfully built coreclr repo with the linux-arm64 packaging PR?

Assuming that is true we should focus on getting corefx successfully building for linux-arm64 and then we will get core-setup working.

Looks like we have https://github.com/dotnet/corefx/pull/21236 which is an attempt to build the linux-arm64 stuff.

@sdmaclea have you been successful at building linux-arm64 in coreclr and corefx?

I don't have a linux-arm64 machine to use for testing so I'm kind of flying blind and providing general guidance based on other platforms.

[cydhaselton]

@weshaggard: Yes, to both questions. And corefx build works using /p:OverridePackageSource, so we have a successful Linux.arm64.Debug build for corefx.

If you search the coreclr, corefx and core-setup repos for Android you'll find several PRs to bring up Android. I'll post links when I have a sec

[weshaggard]

@weshaggard: Yes, to both questions. And corefx build works using /p:OverridePackageSource, so we have a successful Linux.arm64.Debug build for corefx.

It works now? I thought you said it had the same error earlier.

If it works then the next step is to make sure the corefx build produces the necessary linux-arm64 packages which is PR https://github.com/dotnet/corefx/pull/21236. If you get that fully working then you can start working on the core-setup build making sure the override path you point it at contains but the coreclr and the corefx packages you built.

[cydhaselton]

It works now? I thought you said it had the same error earlier.
That tells me that it didn't find your package. Are you passing the OverridePackagePath to option to corefx build?

If failed when I used OverridePackagePath. I changed it to OverridePackageSource and it worked.

If you get that fully working then you can start working on the core-setup build making sure the override path you point it at contains but the coreclr and the corefx packages you built.

What param is used to specify the corefx packages? Or is it OverridePackageSource as well?

[cydhaselton]

@weshaggard A RID question: I'm working on building and packaging clr, fx and setup for Android. As part of that effort android.21-arm64 has been added as a RID in PR dotnet/corefx#16729 and dotnet/coreclr#10286. Why is it necessary to have this RID if packaging will be using linux-arm64? Where does the android RID fit in to this layout?

[sdmaclea]

have you been successful at building linux-arm64 in coreclr and corefx?

@weshaggard
CoreCLR and CoreFX both cross build successfully using a Ubuntu Xenial ROOTFS.

CoreCLR ubuntu CI is running, but it is not green yet, @jashook has been working on it over the last few months. The failed test list needs to be updated and some unexplained CI weirdness needs to be fixed.

CoreFX CI is not building arm64 native packages automatically, so that piece is missing. No one is working on it yet as far as I can tell.

[cydhaselton]

@sdmaclea Couple of questions:

1. Do your cross-built CoreCLR packages use the ubuntu.16.04-arm64 RID or linux-arm64?
2. If the answer to dotnet/corefx#1 is linux-arm64, would the CoreFX and Core-Setup packages use the same naming conventions?
3. Does the successful corefx/coreclr cross-builds include .nuget package generation?

@weshaggard A few questions/comments

1. Skimmed dotnet/corefx#21236 and it looks like this just covers adding the RID; I don't think packaging work is covered; unless adding the linux-arm64 RID is the only change needed for packaging?
2. If not what effort is needed to enable linux-arm64 packaging for corefx? @sdmaclea will you be tackling this?

Question for anyone:
@sdmaclea mentions in this comment that the patch (adding linux-arm64 RID to corefx) would break release flow because of missing CI build for linux-arm64. I'm confused as to the relationship between CI builds and packaging; i.e. why or if the lack of the former would block the latter; an explanation would be helpful.

[weshaggard]

What param is used to specify the corefx packages? Or is it OverridePackageSource as well?

Yes you will need the OverridePackageSource to build. If you do a root level build on an linux-arm64 machine then the packages should get built automatically for you assuming you have the changes in https://github.com/dotnet/corefx/pull/21236. Once you build you should see two packages of interest under bin/packages/Debug which are Microsoft.Private.CoreFx.NETCoreApp and runtime.linux-arm64. Microsoft.Private.CoreFx.NETCoreApp. Those packages will be needed for your core-setup build.

Does the successful corefx/coreclr cross-builds include .nuget package generation?

Yes.

Skimmed dotnet/corefx#21236 and it looks like this just covers adding the RID; I don't think packaging work is covered; unless adding the linux-arm64 RID is the only change needed for packaging?

Yes just adding it to the OfficialBuildRID list will enable it to build the package.

@weshaggard A RID question: I'm working on building and packaging clr, fx and setup for Android. As part of that effort android.21-arm64 has been added as a RID in PR dotnet/corefx#16729 and dotnet/coreclr#10286. Why is it necessary to have this RID if packaging will be using linux-arm64? Where does the android RID fit in to this layout?

Hopefully we don't need an android specific RID. We are doing our best to keep things portable using linux-arm64 RID to avoid cross-compiling for every distro. We would only build and use the android specific RID if we ended up needing to build native assets specifically for android.

@sdmaclea mentions in this comment that the patch (adding linux-arm64 RID to corefx) would break release flow because of missing CI build for linux-arm64. I'm confused as to the relationship between CI builds and packaging; i.e. why or if the lack of the former would block the latter; an explanation would be helpful.

The thing that will break here is that we don't have official builds for linux-arm64 yet so if we enable this RID it will be a dangling dependency in our root packages because there won't be an official package produced and published. We also need to get CI to ensure we don't break this particular build leg.

[jkotas]

We would only build and use the android specific RID if we ended up needing to build native assets specifically for android.

The current portable Linux design will not work for Android. The portable Linux design assumes glibc. Android does not use glibc - it is similar to Alpine.

cc @janvorli

[cydhaselton]

Hopefully we don't need an android specific RID. We are doing our best to keep things portable using linux-arm64 RID to avoid cross-compiling for every distro. We would only build and use the android specific RID if we ended up needing to build native assets specifically for android.

@weshaggard By 'specifically for android' do you mean components that would only be applicable to the Android OS?

The thing that will break here is that we don't have official builds for linux-arm64 yet so if we enable this RID it will be a dangling dependency in our root packages because there won't be an official package produced and published. We also need to get CI to ensure we don't break this particular build leg.

I'm not sure I understand; android was added to the official RID list but official packages aren't published and CI builds aren't enabled. Wouldn't it be possible to do the same for linux-arm64? Or is there enough support for ARM64 Linux for it to have official CI and published packages?

[janvorli]

@jkotas is right, Android uses Bionic C library. See https://en.wikipedia.org/wiki/Bionic_(software).
So it needs its own RID (android-arm64).

[weshaggard]

@weshaggard By 'specifically for android' do you mean components that would only be applicable to the Android OS?

Yes and it sounds like we will need to do that because of the boinic dependency.

Wouldn't it be possible to do the same for linux-arm64? Or is there enough support for ARM64 Linux for it to have official CI and published packages?

Yes it is possible it just doesn't exist yet. HOwever given the new information linux-arm64 doesn't look like it will be enough. You will likely need a new build just for andriod-arm64 and use that as your RID for each repo build.

[sdmaclea]

@cydhaselton I think you already have these answers.

Do your cross-built CoreCLR packages use the ubuntu.16.04-arm64 RID or linux-arm64?

linux-arm64

If the answer to dotnet/corefx#1 is linux-arm64, would the CoreFX and Core-Setup packages use the same naming conventions?

Yes

Does the successful corefx/coreclr cross-builds include .nuget package generation?

Yes

[cydhaselton]

Just catching up with the responses today; work has been busier than normal.

@sdmaclea you mentioned this:

Do your cross-built CoreCLR packages use the ubuntu.16.04-arm64 RID or linux-arm64?
linux-arm64

If the answer to dotnet/corefx#1 is linux-arm64, would the CoreFX and Core-Setup packages use the same naming conventions?
Yes

and @weshaggard, you mentioned this

Yes it is possible it just doesn't exist yet. HOwever given the new information linux-arm64 doesn't look like it will be enough. You will likely need a new build just for andriod-arm64 and use that as your RID for each repo build.

Based on the above I have some questions before I resume work:

1. My builds of corefx, coreclr and core-setup should use the android.21-arm64 RID. Should the build.sh (or build-native.sh and build-managed.sh) commands for each repo include that RID as a parameter?
2. Should the .nuget packages for each repo use a runtime.android-arm64.Microsoft.NETCore.x naming convention or would they use the runtime.linux-arm64.Microsoft.NETCore.x naming convention?
3. Assuming the answer to 1 and 2 above is 'yes', it would still be possible to do a complete build of coreclr, corefx and core-setup without official CI builds or published packaging, correct? (I assume there won't be either for Android; I don't see it being an official build any time soon)

[sdmaclea]

@cydhaselton, @weshaggard is the expert, but here are my opinions.

Seems like the RID should be the most generic. Therefore android-arm64 is better than android.21-arm64 so that is generalized to run on more android platforms. Unless there is a reason to limit he build to a specific android version, I would avoid it.

For android, runtime.android-arm64.Microsoft.NETCore.x should be the default. For android, runtime.linux-arm64.Microsoft.NETCore.x would only be acceptable if the packages ran correctly on Linux and linux-arm64 was a parent of android-arm64 in the RID hierarchy. I suspect android-arm64 is does not inherit from linux-arm64.

core-setup I haven't had time to figure out how to build core-setup for linux-arm64, it looked like it would be easier if there were nuget packages published. I think there is an option to publish to a private nuget repository, but I have not figured it out yet.

[weshaggard]

I've recently been working on getting RHEL6 builds going so I now have a much better idea of what commands you need to build each of the repo's for a new RID. I'm going to work on publishing some documentation with that information (hopefully over the next week or so). So that will hopefully help with your first question.

I agree with @sdmaclea that you should try to be as general as possible so using android-arm64 would be preferred over android.2.1-arm64, unless you are taking a dependency on something in a specific android version.

You won't need official build or CI build support to produce your private local build I will provide instructions on how to do that with the docs.

[weshaggard]

@janvorli has volunteered to write up some docs on how to build for new RIDs.

[cydhaselton]

Briefly checking in on this issue; work at the day job has increased exponentially since this issue was posted so I've less time to contribute to this effort.

Has there been any progress on new RID build docs?
copying @janvorli and @weshaggard for progress

[weshaggard]

@cydhaselton please take a look at https://github.com/dotnet/source-build/pull/227 where @janvorli has a set of docs and a helper script.

[weshaggard]

@cydhaselton I'm going to close this issue as I think the docs https://github.com/dotnet/source-build/pull/227 should cover what you need. Feel free to continue asking questions if needed.