Core: CentOS 7: aspnetcore-2.2.8 RPM doesn't install Microsoft.AspNetCore

[markashton]

Problem encountered on https://dotnet.microsoft.com/download/linux-package-manager/runtime-aspnetcore-2.2.8
Operating System:
Centos 7.

Yesterday, following the instructions on the above page resulted in the following information being shown when performing a dotnet --info command:

$ dotnet --info

Host (useful for support):
  Version: 2.2.7
  Commit:  b1e29ae826

.NET Core SDKs installed:
  No SDKs were found.

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Today, following the same instructions produces the following output:

$ dotnet --info

Host (useful for support):
  Version: 2.2.8
  Commit:  b9aa1abc51

.NET Core SDKs installed:
  No SDKs were found.

.NET Core runtimes installed:
  Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

... and a number of our services fail to start with an error:
It was not possible to find any compatible framework version The specified framework 'Microsoft.AspNetCore.App', version '2.1.1' was not found.

Looking at https://packages.microsoft.com/centos/7/prod/ I see that a new version (2.2.8) was released yesterday, and that the previous version (2.2.7) did correctly install Microsoft.AspNetCore.App/All whereas 2.2.8 doesn't.

Is this an expected change?

[dagood]

This is really messy... the server (https://packages.microsoft.com/centos/7/prod/) has a mix of RHEL and non-RHEL RPMs, even for servicing releases:

aspnetcore-runtime-2.2.7-x64.rpm                   10-Sep-2019 16:00            31471866
aspnetcore-runtime-2.2.8-rh.rhel.7-x64.rpm         19-Nov-2019 18:48            31476936
aspnetcore-runtime-2.2.8-x64.rpm                   19-Nov-2019 18:49            31473267
aspnetcore-runtime-3.0.0-x64.rpm                   23-Sep-2019 14:39             7806594
aspnetcore-runtime-3.0.1-rh.rhel.7-x64.rpm         19-Nov-2019 22:54             7813169
aspnetcore-runtime-3.0.1-x64.rpm                   19-Nov-2019 22:55             7810887

The indexed one changed from non-RHEL to RHEL for 3.0.1:

[root@0e81ea137e09 /]# yumdownloader --urls aspnetcore-runtime-3.0-3.0.0
...
https://packages.microsoft.com/centos/7/prod/aspnetcore-runtime-3.0.0-x64.rpm

[root@0e81ea137e09 /]# yumdownloader --urls aspnetcore-runtime-3.0-3.0.1
...
https://packages.microsoft.com/centos/7/prod/aspnetcore-runtime-3.0.1-rh.rhel.7-x64.rpm

The difference is that RHEL installs to a different location:

[root@0e81ea137e09 /]# repoquery --list aspnetcore-runtime-3.0-3.0.0 | tail -n1
/usr/share/dotnet/shared/Microsoft.AspNetCore.App/3.0.0/THIRD-PARTY-NOTICES.txt
[root@0e81ea137e09 /]# repoquery --list aspnetcore-runtime-3.0-3.0.1 | tail -n1
/opt/rh/rh-dotnet20/root/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App/3.0.1/THIRD-PARTY-NOTICES.txt

dotnet finds the runtime by a directory convention, so the different base dir of this particular RPM makes dotnet unable to find ASP.NET Core.

My guess would be that something went wrong in the release process, we publish both RHEL and non-RHEL-style RPMs, and one wins--the wrong one in this case.

• If it's last-wins, the solution may be to reupload the non-RHEL-style ones to make sure it's the one indexed for install.
• If it's first-wins, we may need to remove both, then add only the correct one.

@leecow @vivmishra @jamshedd

[omajid]

@dagood The "RHEL" style packages are for supplying the missing Microsoft.AspNetCore.App metapackage/framework missing in source-build, right? That's not necessary for 3.0 or later since source-build now includes ASP.NET Core. Maybe you can even stop producing those packages?

Aside:

/opt/rh/rh-dotnet20/root/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App/3.0.1/THIRD-PARTY-NOTICES.txt

The RHEL package convention would be to use rh-dotnet30 in that path for .NET Core 3.0. rh-dotnet20 is for 2.0 packages (only). For 2.1, that path would be /opt/rh/rh-dotnet21/..... For 2.2, that path would be /opt/rh/rh-dotnet22/... For 3.0, that path would be /opt/rh/rh-dotnet30/...

[leecow]

Looking now ...

[dagood]

It looks like this is the case for the Fedora 30 repo as well (and maybe more).

[dagood]

Filed https://github.com/aspnet/AspNetCore/issues/17265 to dig deeper into the RHEL-style package itself and maybe (probably) stop producing it. I'm not familiar with it but what you're saying makes sense, @omajid.

[leecow]

@dagood - the rh.rhel file is in all of the yum repos. Working to remove them now.

[leecow]

This issue is resolved for most distros now (verified centos). Will hopefully have the remaining three done this afternoon.

[markashton]

I can confirm that this is now working for me 👍

[b19901018]

I can confirm that this is now working for me 👍

aspnetcore-runtime-3.0.1 still have this problem.

[gabicrisan]

still having issues with 2.2 as well. I don't see any changes here https://packages.microsoft.com/rhel/7/prod/ and when trying to install dotnet-sdk-2.2.402-x64.rpm it's not able to find aspnetcore-runtime-2.2.8-rh.rh which seems to be right since the package in the repo has the name aspnetcore-runtime-2.2.8-x64. Also tried with dotnet-sdk-2.2.207-x64.rpm, which oddly seems to be the latest package, but I have the same issue. I apologize if this is not the right stream for my issue but seemed to be the most related one.

[jgerry]

As a workaround for my automation I've removed yum -y install aspnetcore-runtime-2.2 and added the following:

wget --quiet https://packages.microsoft.com/rhel/7/prod/aspnetcore-runtime-2.2.8-x64.rpm
rpm -i aspnetcore-runtime-2.2.8-x64.rpm

[dagood]

I've (re?)opened https://github.com/dotnet/core/issues/3886 to track the remaining repros of this issue.