Aspnetcore: The type or namespace name 'Pipelines' does not exist in the namespace 'System.IO'

[TaiNguyen2406]

I got this error after upgrade to .NET Core SDK 3.1.102 and I do not see anything about this changing in doc

[mkArtakMSFT]

Thanks for contacting us. Can you please share a sample repro project, where you observe this issue.
@anurse I'm not certain about the area-label for this one. Can you please adjust? Thanks!

[TaiNguyen2406]

here are screenshots of 2 version that I installed in my PC

[stevejgordon]

I am hitting this issue also. I have an XUnit integration test project defined which is failing with...

error CS0234: The type or namespace name 'Pipelines' does not exist in the namespace 'System.IO' (are you missing an assembly reference?) 
error CS0246: The type or namespace name 'PipeWriter' could not be found (are you missing a using directive or an assembly reference?) 

Curiously, this only seems to be a problem when running the tests as part of my Docker (Debian) build. Running the same tests directly on my Windows PC via VS and CLI and they seem to pass!

I have code in my API project which accepts the PipeWriter (BodyWriter) from the HttpContext.

[stevejgordon]

@mkArtakMSFT Minimal reproduction

Build using DockerfileOld works fine, but DockerfileNew fails. The only difference is the SDK and runtime image versions.

[benaadams]

Looks like its busted for .NET Core as well as ASP.NET Core https://twitter.com/gcaughey/status/1230107136493072384

[Drawaes]

We are having the same issue. Worked before update, doesn't find pipelines after update.

[jzabroski]

Ceci n'est pas une pipe :(

Also see this.

[Drawaes]

If you add the System.IO.Pipelines package directly to your project it should work for now (until the framework ref is fixed).

[Pilchie]

Tagging @wtgodbe who just mentioned this elsewhere. It looks like the things that are packages built out of dotnet/corefx but part of the "Microsoft.AspNetCore.App" Shared Framework aren't in the ref pack in 3.1.2. The workaround above is the right one for now, and we'll look at fixing this in the next patch :(

[leecow]

@rbhanda so that we can capture this in the release known issues doc.

[wtgodbe]

Yes, this was an unfortunate mistake ☹️. Based on my current understanding of the problem, we'll have similar issues for all of the following packages:

• System.Diagnostics.EventLog
• System.IO.Pipelines
• System.Security.AccessControl
• System.Security.Cryptography.Cng
• System.Security.Cryptography.Xml
• System.Security.Permissions
• System.Security.Principal.Windows
• System.Windows.Extensions
• Microsoft.Win32.Registry

[benaadams]

and we'll look at fixing this in the next patch

Is that a soon patch or next month patch? (Since last month's was also broken https://github.com/dotnet/aspnetcore/issues/18334 and this was meant to be the fix for it); just seems a bad look to go for 2 months where creating a vanilla project (file new, or dotnet new) is broken.

[Pilchie]

We're still investigating what a fix looks like, but it's unlikely to be sooner than next month - we've already merged the rest of the fixes for next month's patch to begin validation, and our infrastructure is not well suited to releasing a one-off like this right now, though that is something that we are improving in .NET 5.

[benaadams]

Ok, good to know. There might be an extra step to include in post packaging validation 😉

[Pilchie]

Indeed! We're already looking at what we missed here. Unfortunately, it turns out that shipping previews of ref-packs didn't adequately prepare us for the realities of servicing them.

[dougbu]

Based on my current understanding of the problem, we'll have similar issues for all of the following packages:

Suspect only System.IO.Pipelines matters here. The rest of those packages are likely implementation details and not intended for us to expose.

[anurse]

The rest of those packages are likely implementation details and not intended for us to expose.

No, intended or not, they are exposed and must continue to be.

[Drawaes]

Agreed because people actively trim their dependency trees when they are picking them up from the framework anyway.

[johnkors]

Going from 3.1.1 to 3.1.2, our ASP.NET Core app failed to start:

crit: Microsoft.AspNetCore.Hosting.WebHost[6]
      Application startup exception
System.IO.FileNotFoundException: Could not load file or assembly 'System.Data.SqlClient, Version=4.6.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Data.SqlClient, Version=4.6.1.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

[timheuer]

@johnkors and you did not update any SqlClient package explicitly? Just the runtime to 3.1.2? I believe this is related to the same issue as SqlClient relies on AccessControl and Principal.Windows

[johnkors]

Only updating the runtime on the server and restarting the app, yes

[Pilchie]

@timheuer That should not be the same issue, for two reasons:

1. We have reports of SqlClient in non-ASP.NET Core projects.
2. The runtime pack doesn't seem to be broken here - it still contains those assemblies. This is purely a compile-time issue where the reference pack doesn't contain them.

[joperezr]

@johnkors I'm taking a look at the System.Data.SqlClient issue you pointed out. The problem seems to happen specially if your application depends on a .NET Framework library and use the System.Data shim, as that one has a new dependenciy to the System.Data.SqlClient nuget package version 4.8.1 for 3.1.2. Can you please try changing your ASP.NET Core app to reference the new version of System.Data.SqlClient and try to see if this issue still repros? Thanks in advance for your report and for checking.

[johnkors]

@joperezr Confirmed, updating to System.Data.SqlClient/4.8.1 works with 3.1.2. Thanks! Btw, where can I check what versions of the shims are compatible with what runtimes? If we get this matrix, it's easier to reason if we can safely apply a runtime patch or not.

Btw, we are not referencing the compat pack (https://www.nuget.org/packages/Microsoft.Windows.Compatibility ), but we do have a direct reference in the app to System.Data.SqlClient/4.8.0.

[joperezr]

All runtime patches are supposed to work with OOB packages, the problem is that for 3.1 we changed the way we build these shims and it looks like that change broke this scenario. We are now working on servicing a fix for this which will make it such that all new runtime patches will work with future/past packages, so that there wouldn't be a need for any matrix.

[johnkors]

Ok, that's even better.

if your application depends on a .NET Framework library and use the System.Data shim

Could you elaborate on this. I depend on a .NET Framework library, but how would I know if I'm using a shim? Where is this list of packages part of a shim? Would that be all the packages listed as a dependency for https://www.nuget.org/packages/Microsoft.Windows.Compatibility ?

[joperezr]

Not exactly. Here is kind of the background for shims. Most of the types that exist in .NET Framework are also present in .NET Core, but they don't necessarily live in the same assembly. For example, the type System.String lives in mscorlib.dll on .NET Framework, but it lives in System.Runtime.dll for .NET Core, similarly the type SqlNotificationRequest lives in System.Data.dll in .NET Framework, but in System.Data.SqlClient.dll in .NET Core. When you are running a .NET Core application, and try to use a library that was targeting .NET Framework, it will try to load the types from the assemblies that it thinks they should live in, so when using String, it will try to load it from mscorlib.dll, and when using SqlNotificationRequest it will look for it in System.Data.dll.

System.Data.dll and mscorlib.dll only really have types in .NET Framework, so what we did here in order for .NET Framework libraries to be able to use all of these types that live in a different place, is to create what we call shims (sometimes we also refer to them as facades, or forwarders) which basically act as re-mappers or re-routers so that when a .NET Framework library tries to load a type from them, they will re-route the search to the assembly where they actually live in .NET Core.

Could you elaborate on this. I depend on a .NET Framework library, but how would I know if I'm using a shim?

All that said, this means that if you depend on a .NET Framework library, and this library uses any types from System.Data.dll, then that most likely means that at runtime in .NET Core, we will have to load the System.Data.dll shim in order to re-route all of the types to the right place. You could inspect your .NET Framework dependency to check if it references System.Data.dll and inspect with ildasm to see if it actually uses one of System.Data types.

Where is this list of packages part of a shim?

We used to pack all of these shims in the Compatibility package, but now we pack them all as part of the runtime. The reason is that many people where trying to load .NET Framework assemblies from .NET Core, and didn't find it very intuitive to require adding an extra NuGet package reference, so all of the .NET Framework shims are now shipped as part of the shared framework everytime you target a new runtime or download a new SDK.

So why is this causing trouble now? Well, the problem is that we recently changed how we build these shims in order to use a much more robust infrastructure, and we missed one key difference that lead to this bug you are hitting. I merged yesterday a PR that will mitigate this for the System.Data.SqlClient case (PR https://github.com/dotnet/corefx/pull/42868 which will be shipped for 3.1.3 rutnime) and will work on making this infrastructure m ore robust to make sure these kind of problems never happen again.

I hope all of that made sense, but if not, feel free to ask me to clarify 😄

[jzabroski]

Great description, @joperezr

We used to pack all of these shims in the Compatibility package, but now we pack them all as part of the runtime. The reason is that many people where trying to load .NET Framework assemblies from .NET Core, and didn't find it very intuitive to require adding an extra NuGet package reference, so all of the .NET Framework shims are now shipped as part of the shared framework everytime you target a new runtime or download a new SDK.

When was this introduced? Thank you.

Also, as an anecdote, a JavaScript developer asked me, "Why C#" and "Why DotNet" in 2020. After thinking deeply about the problem, I enumerated the areas where choosing this stack slowed me down, and the total sunk cost incurred learning .NET Core in depth. My main pain points have been around understanding "Out-of-box assemblies" and Shims, but also: Since Out-of-box assemblies are shipped as Nuget package references, there is the following unexpected behavior possible: Nuget package reference Version 4.8 targeting net48 might have a different assembly version than the same Nuget package reference Version 4.8 targeting netstandard2.0. _This is the number one non-intuitive experience, and the one I saw the .NET team continuously punt on fixing. Should this have a separate issue for discussion? I've explained the issue elsewhere, and I just wonder if my medium for my message was poorly chosen._ Transclusion of shims doesn't solve this problem. It also doesn't answer the fundamental question: Where is this list of packages part of a shim? - because the real question isn't "Where is this list of packages part of a shim?" but rather "What packages ship which assemblies as part of the shim?" Hope that makes sense.

[johnkors]

First of all, thanks for the very thorough response 👏 Really appreciate the details!

I was unaware of the deprecation of the compat package moving it to the runtime, though. Is there a blog post or release notes somewhere talking about it? It's not mentioned in any of the migration docs..?

• https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0
• https://docs.microsoft.com/en-us/dotnet/core/compatibility/2.2-3.0
• https://docs.microsoft.com/en-us/aspnet/core/migration/22-to-30

[joperezr]

I think I explained myself incorrectly above 🤦‍♂. The package isn't deprecated, and it still contains more shims than the ones we ship with the runtime so it is still a viable alternative for people hitting .NET Framework vs .NET Core problems, but what I really meant is that we have gradually added more and more shims into the runtime as we keep seeing some shims are more often required than others.

[jzabroski]

~It was introduced for our 3.0 release with PR dotnet/corefx#37550~
EDIT I thought you were asking for the change in the generation of the shims.

I was indeed asking for the change. Thank you.

As far as

That is why whenever we want to do this, we just add the API to the places in the package where this is possible, and increase the assembly version of only those assets in the package. This of course over time, will mean that our OOB packages contain several different assemblies with different surface area, and different Assembly versions.

That all makes sense to me now, but:

1. there is no visibility into this from nuget.org perspective
   
   
   
   1. Many of the best programmers I know find it confusing when a package says it has "No dependencies".
   
   
2. Visual Studio 2019 doesn't surface there is a difference
3. even fuget.org doesn't quite surface quite this important idiosyncratic detail - you have to mouse over the assembly link in the left-hand nav to see the exact assembly version, and there is no "master grid" that shows you all the possible link-load combinations.
4. Enterprise package repositories similarly don't have a good way to package https://www.nuget.org/packages/Microsoft.Windows.Compatibility - it bloats our cloud subscription storage costs because it internalizes the dependency

Anyway, I can't tell if I'm rambling, but this is "the pain behind the pain".

[joperezr]

Those are totally valid points and I agree with all of them. I do have an issue on my backlog https://github.com/dotnet/runtime/issues/26002 which the title is a little confusing but basically what we wanted is to change the description of our packages, to instead also provide a table with all the assemblies contained in the package with their corresponding assembly versions, so that people might find this helpful when looking if a package contained a specifc API or not. I haven't made much progress on that issue because it got de-prioritized at the time, but if enough people think that this feature on the packages would at least help filling the gaps that nuget.org, fuget, or VS don't show, we can work on prioritizing that again.

[spottedmahn]

I do have an issue on my backlog dotnet/runtime#26002 which the title is a little confusing

Let's update it then! 😀

[Microshaoft]

mark, same build error in ubuntu 18.04
error CS0234: The type or namespace name 'Pipelines' does not exist in the namespace 'System.IO'

[AdamWillden]

I encountered this a few days ago and simply uninstalled 3.1.200 to revert to a working build.

I use VS preview which has just rolled out a new version packaged with 3.1.200

It seems I cannot uninstall this component meaning, unless someone has any ideas (@Pilchie ?), I'm now forced to uninstall VS preview(??)

Here's hoping a fix is found soon 🍻

[Pilchie]

@AdamWillden - you can work around the problem by adding a Package Reference to System.IO.Pipelines until the next runtime ships.

[AdamWillden]

@Pilchie I just came to the same conclusion - now I feel stupid 🤦‍♂

[jawn]

Is still still an issue with .NET Core 3.1.3?

[stevejgordon]

@jawn

Is still still an issue with .NET Core 3.1.3?

I've upgraded and it seems to be fixed.

[johnkors]

Is still still an issue with .NET Core 3.1.3?

Same here, 3.1.3 resolved the issues with referencing System.Data.SqlClient/4.8.0.

[Pilchie]

Yes, @wtgodbe fixed this for 3.1.3. I'll close it. Note that System.Data.SqlClient was actually a unique issue in the dotnet/corefx repo that was also fixed for 3.1.3.