Aspnetcore: [Discussion] Migrating dotnet/extensions content to dotnet/runtime and dotnet/aspnetcore

Created on 12 Mar 2020  路  16Comments  路  Source: dotnet/aspnetcore

This is the discussion thread for https://github.com/aspnet/Announcements/issues/411

As part of the ongoing repository consolidation effort in .NET 5, we are working to move most of the content from dotnet/extensions into other repositories. Most of these packages were developed by and are currently maintained by the ASP.NET team. However, moving forward we want to enable more scenarios with these packages, outside of ASP.NET.

The packages which we believe are usable in various application models (Configuration, DI, Hosting, etc.) are being moved to the dotnet/runtime repository. Packages which are primarily used in ASP.NET Core applications (Web Encoders, etc.) are moving to the dotnet/aspnetcore repository. Finally, in order to avoid introducing third-party dependencies into dotnet/runtime and dotnet/aspnetcore, some packages are remaining in the dotnet/extensions repository as they have dependencies on third-party packages (such as integrations with StackExchange.Redis and other components).

The full list of packages that are moving can be found at the end of this announcement. In general, tests and samples will be moving to the relevant repo based on where the main package moved. Issue tracking for the relevant packages will also move to that repository.

PR/Issue migrations

There are 400+ open issues and ~50 open PRs (at time of writing) in dotnet/extensions.

Since we are actively working on migrating the packages now, we recommend waiting to open new PRs until we update this announcement indicating the migration is complete. Some packages are building in two repositories and we want to make sure your changes land in the right place. Over the migration process, we'll be closing most of the active PRs that target packages which have been moved and asking the author to migrate the PR to the relevant repository.

For issues, feel free to continue opening them and we'll respond to them as best we can during the migration. If the package your issue relates to has been moved, we'll handle transferring the issue. We'll be doing some triaging of the incoming issues and the backlog and will attempt to migrate as many issues as possible. As part of this process, we'll also close issues that are no longer relevant, duplicates of other issues, or enhancements we don't plan to support.

Q&A

If your question isn't answered here, comment below to ask it!

Are these packages moving in to the Base Class Library ("BCL"), or Microsoft.NETCore.App?**

We do not plan to change how these components ship as part of this migration. Packages that currently ship on NuGet.org will continue to do so. Libraries that are currently included in ASP.NET Core will continue to be included. We are not adding any of these components to the BCL at this time.

If package Microsoft.Extensions.X is moving to the runtime repo, why not rename it to System.X?

While it is possible to rename assemblies and redirect type references, it is not possible to change package identities, namespaces or type names in a non-breaking way. Renaming these types would be extremely disruptive and provide very little benefit. All of these components are already covered by the .NET Core Support Policy and will continue to be in the future, just like the System.* assemblies.

Will these packages still support .NET Standard 2.0?

We do not plan to reduce the set of frameworks these packages can be used on as part of this migration. If they currently support .NET Standard, we have no plans to change that.

Why are some packages staying in dotnet/extension?

As a general rule, we want to avoid third-party dependencies in the dotnet/runtime and dotnet/aspnetcore repositories. The packages that depend on third-party dependencies are remaining in dotnet/extensions to help preserve this.

Where should I file issues related to these packages?

In general, file issues in the repo that contains the package. Some feature areas are split between multiple repos. If you're not sure, feel free to file issues in any of the three repos (dotnet/runtime, dotnet/extensions or dotnet/aspnetcore) and the team can transfer the issue as needed.

Why did you move package X to repo Y?

Feel free to comment below to ask questions about specific packages.

Package List

The following list identifies all the packages we currently ship from dotnet/extensions and which repo they are moving to.

picture anurse
🎉1

Most helpful comment

I have to say that I鈥檓 really not a fan of this plan. Yes, I do understand the desire to further consolidate the repos although that already doesn鈥檛 work if the extensions packages are now basically being split to three different repositories. I also get that this is done because the packages ship with everything else, so getting them in the same repo makes it easier for the release process.

But there is still a huge impact on contribution. Just taking the repo size alone, a fresh clone of dotnet/runtime is 1.1 GiB at time of writing; compared to dotnet/extensions being at 29 MiB. For those that are not primarily involved with the runtime, this means that they will have to get involved with a lot of unrelated stuff if they want to contribute to the extensions packages.

That applies to the build process too: Building the extensions repo was never easy but from what I heard from others, building the runtime repo is at a whole different level of being troublesome for outsiders. And it would be really painful having to build the full runtime if you just want to contribute to some rather high-level extensions library. So if you really have to do this, then at least provide a way that contributors do not need to build everything, and that they _especially_ do not require all the dependencies that the runtime needs (e.g. cmake, Python, the Windows SDK and all those other C-related dependencies).

The biggest issue I have however is with the issues. dotnet/runtime is a very active repository with over 5000 open issues, and every day there is a large amount of new stuff. That makes it impossible to monitor the issues, even if you are just interested in a small subset like the extensions. This is also mostly due to GitHub鈥檚 restrictions for useful notifications.

As someone who is interested in the extension packages, this migration will affect me both for contributing and for getting information about changes. You probably won鈥檛 reconsider this decision but I would like you to at least take the pain points into account when organizing this in the future.

picture poke on 15 Mar 2020
👍15

All 16 comments

Will these packages still support .NET Standard 2.0?
We do not plan to reduce the set of frameworks these packages can be used on as part of this migration. If they currently support .NET Standard, we have no plans to change that.

Excellent news, thanks for keeping the .NET Standard/.NET Framework support intact! 馃憦

kevinchalet picture kevinchalet on 12 Mar 2020
👍2

While I understand the motivation and the rationale behind the package split, this is going to make it harder for people to find the source code of a particular package or create an issue. Particularly for packages related to Configuration where some go to the runtime repo, others go to the aspnetcore repo and others stay in the extensions repo.

May I suggest adding to the readme of the extensions repo the Package List section of this issue?

epignosisx picture epignosisx on 13 Mar 2020
👍7

Agree with @epignosisx , I suggest add more specific git repository path to nuget.org , so we can find thire source code by just click the "Source repository" link

John0King picture John0King on 13 Mar 2020
👍3

Why not move Microsoft.Extensions.ObjectPool to runtime?Seem reusable by all kind of apps.

Microsoft.Extensions.Hosting.Systemd
Microsoft.Extensions.Hosting.WindowsServices

They have third-party dependencies but should be OS dependencies and runtime has got support for OS dependencies, I mean maybe some pinvoke could be shared, for instance https://github.com/dotnet/extensions/blob/master/src/Hosting/WindowsServices/src/Internal/Win32.cs.

MarcoRossignoli picture MarcoRossignoli on 13 Mar 2020

May I suggest adding to the readme of the extensions repo the Package List section of this issue?

Agree with @epignosisx , I suggest add more specific git repository path to nuget.org , so we can find thire source code by just click the "Source repository" link

Great ideas, I'll file some issues to track/verify that, but for the most part I think the "Source Repository" link will be updated automatically since that value is auto-injected by our build system.

anurse picture anurse on 13 Mar 2020

Why not move Microsoft.Extensions.ObjectPool to runtime?Seem reusable by all kind of apps.

It is a broadly useful concept but there are already some pooling concepts in the runtime and we think that right now we don't want to bring this one in. It's always possible to move things around though, and nothing about this move changes where you can use these packages.

They have third-party dependencies but should be OS dependencies and runtime has got support for

Yep, it's certainly possible to move these to dotnet/runtime and we're still looking in to that. The biggest priority right now has been taking components we believe will be usable in almost all app models in to the dotnet/runtime repo. Systemd and Windows Services are primarily for server and background-worker scenarios (there are always counter-examples though ;), so we're not quite ready to move those into the runtime repo.

anurse picture anurse on 13 Mar 2020
👍2

I have to say that I鈥檓 really not a fan of this plan. Yes, I do understand the desire to further consolidate the repos although that already doesn鈥檛 work if the extensions packages are now basically being split to three different repositories. I also get that this is done because the packages ship with everything else, so getting them in the same repo makes it easier for the release process.

But there is still a huge impact on contribution. Just taking the repo size alone, a fresh clone of dotnet/runtime is 1.1 GiB at time of writing; compared to dotnet/extensions being at 29 MiB. For those that are not primarily involved with the runtime, this means that they will have to get involved with a lot of unrelated stuff if they want to contribute to the extensions packages.

That applies to the build process too: Building the extensions repo was never easy but from what I heard from others, building the runtime repo is at a whole different level of being troublesome for outsiders. And it would be really painful having to build the full runtime if you just want to contribute to some rather high-level extensions library. So if you really have to do this, then at least provide a way that contributors do not need to build everything, and that they _especially_ do not require all the dependencies that the runtime needs (e.g. cmake, Python, the Windows SDK and all those other C-related dependencies).

The biggest issue I have however is with the issues. dotnet/runtime is a very active repository with over 5000 open issues, and every day there is a large amount of new stuff. That makes it impossible to monitor the issues, even if you are just interested in a small subset like the extensions. This is also mostly due to GitHub鈥檚 restrictions for useful notifications.

As someone who is interested in the extension packages, this migration will affect me both for contributing and for getting information about changes. You probably won鈥檛 reconsider this decision but I would like you to at least take the pain points into account when organizing this in the future.

poke picture poke on 15 Mar 2020
👍15

That applies to the build process too: Building the extensions repo was never easy but from what I heard from others, building the runtime repo is at a whole different level of being troublesome for outsiders. And it would be really painful having to build the full runtime if you just want to contribute to some rather high-level extensions library. So if you really have to do this, then at least provide a way that contributors do not need to build everything, and that they especially do not require all the dependencies that the runtime needs (e.g. cmake, Python, the Windows SDK and all those other C-related dependencies).

Thanks for pointing that out. At the moment the runtime repository requires a full clr+libraries build to build these projects which target netstandard2.0. The good news is that I'm currently working on lifting that restriction so that these projects can be built directly without the full runtime needed to be built. The tracking issue for that is: https://github.com/dotnet/runtime/issues/31844.

In addition to that I want to point out that have recently enabled scenarios like dotnet test, F5 debugging support and VS Test Explorer support and we will continue investing in developer productivity so that there are no compromises when creating a project in dotnet/runtime in comparison to any other repository.

cc @ericstj

ViktorHofer picture ViktorHofer on 8 May 2020
👍1

I would like to suggest that the nuget packages are all updated to reflect the current location of the source.

It's very hard to find up to date source when you keep moving it. Even search engines can't seem to keep up.

It took me a while to even find this announcement, because of https://github.com/aspnet/Announcements vs https://github.com/dotnet/announcements

For example, https://www.nuget.org/packages/Microsoft.Extensions.Hosting/3.1.4 lists https://asp.net/ as the projectUrl, and https://github.com/aspnet/Extensions as the source repository (5.x has dotnet/runtime as the source repository). Both of these links are useless. It would be more useful if the commit hash, which is part of the nuspec, was included in the link on nuget.org e.g. https://github.com/dotnet/extensions/tree/7cc2a109d98493cf617de56709ec188fa5cd3fc1 . The vast majority of results on Google point to https://github.com/aspnet/Hosting, which is archived but not deleted, causing even more confusion because it's all old source, and it doesn't mention that it's archived unless you go to the root of the repo.

ghelyar picture ghelyar on 20 May 2020

For example, https://www.nuget.org/packages/Microsoft.Extensions.Hosting/3.1.4 lists https://asp.net/ as the projectUrl, and https://github.com/aspnet/Extensions as the source repository

The 3.1 package鈥檚 source repository is aspnet/Extensions (well, dotnet/extensions but GitHub has a redirect in place) and both the 2.1 and 3.1 packages (which are in long-term support) will be supported from that repo.

The 5.0 preview builds should get the correct metadata as they are built out of the dotnet/runtime repo.

anurse picture anurse on 20 May 2020

Sorry, I had updated my comment before you posted, to make myself clearer.

the nuspec in the package contains
<repository type="git" url="https://github.com/aspnet/Extensions" commit="7cc2a109d98493cf617de56709ec188fa5cd3fc1" />

but nuget.org does not include the commit hash in the link, so if you actually go to https://github.com/aspnet/Extensions, you get redirected to https://github.com/dotnet/extensions, which now doesn't have the source in it, and it's difficult to find where it's gone. If you didn't know that it was previously there and has since been deleted, it is very confusing and just looks like the wrong link is on the package.

If it actually linked to the commit as it was when the package was built, instead of the repository as it is now, it would be less confusing.
https://github.com/dotnet/extensions/tree/7cc2a109d98493cf617de56709ec188fa5cd3fc1

This is similar to what @John0King suggested above. You replied saying for the most part it already does, but I think having the commit hash in the link would be very useful, for the reasons I've given here.

Also, if you wanted to have a link to the current project URL to raise issues against, perhaps you could use something like aka.ms instead of https://asp.net , if that would allow the redirect to be updated later.

ghelyar picture ghelyar on 20 May 2020

If Microsoft.Extensions.DiagnosticAdapter isn't being shipped anymore, what is the replacement option? Roll your own?

tillig picture tillig on 7 Jul 2020

If Microsoft.Extensions.DiagnosticAdapter isn't being shipped anymore, what is the replacement option

You don't need a replacement. It isn't being removed from nuget.org. Continue to use it if it works for you.

ericstj picture ericstj on 8 Jul 2020

While I appreciate the sentiment that I should keep using the existing library if it continues to work, "not being shipped" implies "no support." That has a ring of deprecation and doesn't sit too well for the long term.

tillig picture tillig on 8 Jul 2020

nuget.org does not include the commit hash in the link

https://github.com/NuGet/NuGetGallery/issues/7537 is a request for that feature.

KalleOlaviNiemitalo picture KalleOlaviNiemitalo on 8 Aug 2020
👍1

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog clean. If you believe there is a concern related to the ASP.NET Core framework, which hasn't been addressed yet, please file a new issue.

This issue will be locked after 30 more days of inactivity. If you still wish to discuss this subject after then, please create a new issue!

msftbot[bot] picture msftbot[bot] on 12 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

markrendle picture markrendle  路  3Comments
guardrex picture guardrex  路  3Comments
mj1856 picture mj1856  路  3Comments
rbanks54 picture rbanks54  路  3Comments
UweKeim picture UweKeim  路  3Comments