Aspnetcore: Website publish assumes profile by default

[Petermarcu]

Repro steps:

Install .NET Core 2.0 SDK Preview 1

dotnet new web
dotnet publish

Look in the publish folder and note that only MyApp.dll is in the published output. I expected that without specifying a publish profile, everything above netcoreapp2.0 would have been in the publish folder. The way it is today, an app will not run on a target machine that has just installed the .NET Core 2.0 runtime and I don't see how to configure it such that it will.

[Petermarcu]

/cc @DamianEdwards @bleroy @richlander

[muratg]

@Petermarcu I believe this is by design when you use Microsoft.AspNetCore.All metapackage. The idea is that when you run the right installers, your server machine will have the runtime package store side by side with the shared runtime, so for most cases, this will be a non-issue.

BTW, you can prevent trimming via a csproj property (I forgot what it was called.)

[bleroy]

The same is true of any application published against any target manifest: if the bits are not there on the other side, it'll fail. In the case of ASP.NET, it's a little more annoying because the manifest comes with the template, but fundamentally it's the same thing. That's what the feature is.

[Petermarcu]

I didn't see it in the template. It wasn't in the project file.

I also don't see installers being provided for this across all the different distros. Where are those? How does a distro build these in?

If I use the project I just made and try to publish and run on a RHEL machine, what do I do?

[davidfowl]

This is by design.

I didn't see it in the template. It wasn't in the project file.

It comes with the meta package. You can either change the property or stop using the meta package.

I also don't see installers being provided for this across all the different distros. Where are those? How does a distro build these in?

@JunTaoLuo @DamianEdwards didn't we make something for OSX and linux?

If I use the project I just made and try to publish and run on a RHEL machine, what do I do?

I'm not familiar with how RHEL packages ASP.NET (maybe they do nothing today for us so it just works) but ideally we would provide the package store for all OSes.

[Petermarcu]

Can you point me to the installer links for the ASP.NET Runtime that I need to install on all the supported OS's in order for this app to run?

What is the property I have to set to get back to the behavior that enables me to run on netcoreapp2.0?

The implicitness of it in the meta package is unintuitive, it needs to be driven from a discoverable property in the project. This package now has side effects unlike any other package that I wouldn't expect and I have no way of discovering the setting that caused this behavior.

[muratg]

You can use https://www.microsoft.com/net/core/preview#linuxubuntu (and there are links for other OSes) to download the installers with the package store.

The csproj property to turn off trimming is PublishWithAspNetCoreTargetManifest which you should set to false. But beware, you'll end up a ton of stuff in your publish output. You may instead prefer to add direct references to the packages you use instead.

@livarcocc @JunTaoLuo are there better links for the installers/zips?

[davidfowl]

The implicitness of it in the meta package is unintuitive, it needs to be driven from a discoverable property in the project. This package now has side effects unlike any other package that I wouldn't expect and I have no way of discovering the setting that caused this behavior.

I don't agree. This is the default experience we want to have for all ASP.NET applications. It also means that when you get ASP.NET, you get the runtime store by default (so we need to make that happen). Anything else should require configuration in the project file.

I spoke to Damian about the discoverability of certain properties in the project file and I don't think we should have anything in there (that's how the csproj ended up being so bloated in the first place). We can have a URL or something that points to docs of common properties you can configure in the project file. I'd prefer that rather than bloating it.

[Petermarcu]

@muratg , does that mean I need to install the whole SDK on my target production machine to run ASP.NET apps?

[livarcocc]

I wonder if we should put the runtime store bundled with the shared runtime, instead of having it bundled only with the SDK.

[davidfowl]

I wonder if we should put the runtime store bundled with the shared runtime, instead of having it bundled only with the SDK.

👍

[DamianEdwards]

@livarcocc product-packaging-wise, I think that's the logical thing to do. It's part of our (.NET Core's) default experience in 2.0. The only tricky issue I'm aware of is the build-from-source requirement for certain distros.

[bleroy]

The only tricky issue I'm aware of is the build-from-source requirement for certain distros.

Or more generally that we can only issue recommendations to third-party distro maintainers for which we don't ship our own installers. That means I think that the recommendation should state this clearly. Currently, it doesn't mention ASP.NET.

[Petermarcu]

Nothing in the shared runtime can be packages and everything in there needs to be able to build from source.

[Petermarcu]

Ignoring which installer these are in, the real challenge is that we have picked a default experience that falls apart in the end to ends because the platform needs to build from source so that distros can build it in. Redhat needs to be able to build the same SDK we build and the templates in dotnet new need to be correct for developers on Redhat.

[DamianEdwards]

@Petermarcu let's talk about this in the runtime store sync-up in this week. I think this warrants an in-person discussion.

[ellismg]

/cc @omajid

[tmds]

The implicit manifest on publishing ASP.NET apps is a UX question.
If there are implicit manifests, a command option on dotnet publish to specify 'no manifest' would be nice :) (alternative: /p:TargetManifestFiles=)

I think the store should be separate from the sdk and separate from the runtime. They should meet as part of the installation.
The manifests describing 'official' stores should be versioned and shipped with the sdk.
Those manifests can be used to install a store using the dotnet store command.

As part of building an installer, the manifests can also be used to build & include a pre-zipped store.
For Linux, stores may be distributed as separate packages.
(Note: all the nuget packages in the store must have a license that permits this distribution)

This is not only useful for 3rd party linux distro maintainers:

• It's nice to separate .NET Core from ASP.NET Core. If you are doing pure .NET Core stuff, you shouldn't have to install an ASP.NET store.
• The dotnet docker sdk image contains a store which doesn't get used, it just bloats the image.

[omajid]

I spoke to Damian about the discoverability of certain properties in the project file and I don't think we should have anything in there (that's how the csproj ended up being so bloated in the first place). We can have a URL or something that points to docs of common properties you can configure in the project file. I'd prefer that rather than bloating it.

It might be worth looking at how maven does it. Maven has a pretty small pom.xml (analogous to a .csproj) file by default. maven also has a command to show the effective pom, which is the complete pom.xml file with all default values actually filled in. This keeps the default build configuration small but lets user expand it on demand to see all the default settings and lets them override it. If our msbuild configuration is too complex, that may be one solution we can adapt from.

[bleroy]

I like that idea. I've often had to dig deep to find where defaults were defined.

[ellismg]

For what it's worth (and I used MSBuild for a long while before I knew this was possible), you can pass /pp:<path-to-expanded-file> to resolve all the imports in an MSBuild project. This lets you get a complete sense of the file as MSBuild will see it. Conditionals are also evaluated so if an import would not happen do to a condition, you can see this.

This is super helpful when you want to understand exactly what is setting a property or where a target is coming from.

[bleroy]

oooh, now I need to make sure this is easy to find in our docs.

[tmds]

@Petermarcu @DamianEdwards do you have an update (especially regarding the build-from-source)?

[bleroy]

Yes, the current plan we came up with is the following. Please let us know if you have feedback.

1. The SDK installers that Microsoft ships will include everything, including the runtime package store.
2. Microsoft's runtime installers don't include the store, but there's a separate store installer with a dependency on the runtime installer.
3. We recommend package maintainers include the runtime package store in their dotnet-sdk packages, but we understand that is not always compatible with rules and usage for specific distros.
4. Microsoft builds DEBs and RPMs for the package store, that will be published on Microsoft feeds.
5. We'll provide a shell script that takes care of installing those same store bits for the cases where the Microsoft feed or native installers aren't available or can't be used.
6. Post 2.0, we'll work on how we can get ASP.NET building from source as well.

[omajid]

Please correct me if I am mistaken. Any .NET Core built by anyone not-Microsoft will result in a missing runtime store. Will the script to download the runtime store be part of .NET Core? When will it be available?

Does this also mean that if I build .NET Core from source, use that to build my ASP.NET Core application, the resulting application will fail to run unless I built it with /p:TargetManifestFiles=?

We recommend package maintainers include the runtime package store in their dotnet-sdk packages, but we understand that is not always compatible with rules and usage for specific distros.

TBH, I think everyone distributing binaries will be blocked. IANAL, but the .NET Library License (https://www.microsoft.com/net/dotnet_library_license.html) seems to suggest that distributing just the library (distributable code) by itself is not permitted. And that means no one can distribute it under this license?

Post 2.0, we'll work on how we can get ASP.NET building from source as well.

:+1:

That said, I agree with @tmds about how .NET Core should not be tied to ASP.NET Core.

[tmds]

I don't know why the store is bundled with the sdk. The store is a runtime feature. When you build an app, the store is not used (the nuget cache is). When you publish an app, the store is not used either (only the manifest is used to trim). This is not a concern, I simply don't understand the rationale.

From your list, I deduce it will be possible to build the repositories so the final result does not include a store. Good!

Point 4 and 5 gives the users the ability to install a store without the package maintainers having to violate the build-from-source guideline. Good!
One concern here is that both these approaches depend on external feeds and storage. A nicer way would be to put the manifest under version control win the cli repository. This puts them close to the dotnet store command which can be used to reconstruct a store. Also the only external dependency then is NuGet, which is a common dependency for .NET and is easy to cache on prem (which is useful to avoid external dependencies).

[tmds]

Q: Where does the ASP.NET Store manifest reside currently? Is it part of the dotnet installation? Is it in a NuGet package?

[omajid]

Is it https://dotnet.myget.org/feed/aspnetcore-dev/package/nuget/Build.RuntimeStore?

[JunTaoLuo]

@tmds The manifest currently resides in the Microsoft.AspNetCore.All metapackage. @omajid The Build.RS (Build.RuntimeStore is deprecated) package contains the runtime store itself but does not contain the manifest.

[ellismg]

Will the script to download the runtime store be part of .NET Core? When will it be available?

I don't think we intend to ship it in the product. We would treat it like the dotnet-install.sh script, it would be checked in somewhere on GitHub and we'd have a nicer URL to it as well (and it's very possible that we just add it as an option to dotnet-install.sh similar to how today you can use dotnet-install.sh to install either the SDK or the Runtime).

[bleroy]

I don't know why the store is bundled with the sdk. The store is a runtime feature. When you build an app, the store is not used (the nuget cache is). When you publish an app, the store is not used either (only the manifest is used to trim). This is not a concern, I simply don't understand the rationale.

The SDK, at least as Microsoft ships it, contains everything you need to write, build, package, and run .NET Core applications and libraries. The runtime package store is part of this experience. The runtime package should be in principle everything you need to run a .NET Core application, with the caveats that this issue exposes. The compromise we're proposing is that it should always be possible and relatively simple to install the store on top of it, if you need to. Does this clarify?

[omajid]

Not sure if this was addressed, so I want to bring this up again, slightly reworded:

Does this issue mean that if I build .NET Core from source, use that to build my ASP.NET Core application, the resulting application will fail to run by default on my .NET Core build (unless I build the application it with /p:TargetManifestFiles=)?

[bleroy]

@omajid it depends if your target environment has the ASP.NET store installed. If only the .NET Core runtime was installed, then yes it will fail.

[Petermarcu]

The answer is yes. Because when you build from source, the runtime store is not present. You would need to build from source, then acquire the runtime store. Otherwise, you need to set the property in your project to opt our of ASP.NET's runtime store.

[omajid]

Ouch. This will be a painful ongoing issue for us :(

[tmds]

Perhaps it makes sense to add the property to the ASP.NET Core project templates?

<PublishWithAspNetCoreTargetManifest>true</PublishWithAspNetCoreTargetManifest>

This makes it explicit this is happening and the property can be easily found and modified by the user.
With this change, the Microsoft.AspNetCore.All package can also be changed to not implicitly inject a manifest.

Your thoughts?

[davidfowl]

I'm not a fan of adding default like this to improve discoverability of properties, it's the reason csproj got as big as it has been in the past. I'd much rather prefer an "effective POM" approach.

[tmds]

I'm also suggesting to change the default so it doesn't implicitly inject a manifest.

@bleroy @omajid made a comment about the dotnet library license, who can answer that?

[omajid]

Any updates? We believe this issue - and its consequence in terms of users not being able to run ASP.NET Core applications out of the box - is a blocker for us.

[Petermarcu]

@DamianEdwards is the owner for this.

[davidfowl]

Can't we just change the instructions for redhat until we get a more complete solution?

[omajid]

FYI, I am from Red Hat.

We, in partnership with Microsoft, distribute versions of .NET Core that are supported jointly by Microsoft and Red Hat. These .NET Core packages are installable via yum install ....

We (Red Hat):

1. Can not ship anything we do not build from source

2. Do not want to ship that appears broken to users until they install software from another location. What this really tells the user is that our version of .NET Core is incomplete.

3. Prefer to avoid making things any more complicated than the normal yum install for users. Especially if that extra step is unique to our distribution of .NET Core and from the rest of the operating system. Asking users to download binaries would basically make us unable to ship .NET Core.

4. Avoid shipping something that is different from upstream (unless we absolutely have to, for example, it breaks compatibility).

[davidfowl]

I understand that, I'm just saying that until we are building from source, the deployment instructions for redhat would include an extra command line flag to disable the manifest. That IMO doesn't seem like a stretch.

[bleroy]

@tmds @omajid On the licensing issue, the ASP.NET Core bits should not be under a different license than the source code. @DamianEdwards can confirm that.

[omajid]

the deployment instructions for redhat would include an extra command line flag to disable the manifest.

That will resolve the issue, yeah.

But it wont be good enough. This will be a user experience that's specific to RHEL. To anyone who skips that part of the doc, or only uses information found elsewhere, .NET Core on RHEL will be broken out of the box, especially compared to .NET Core elsewhere.

Worse, if a user downloads .NET Core from Microsoft, it will suddenly appear to work on RHEL. I would certainly see this as the .NET Core version we ship being broken. Which is why we prefer to avoid this solution completely, if possible.

[omajid]

@bleroy https://dotnet.myget.org/feed/aspnetcore-dev/package/nuget/Microsoft.AspNetCore.All says it's under "MS-EULA".

[bleroy]

@omajid I know :) Since the issue was raised, we talked about it and reached a consensus that 2.0 should change that to the MIT Apache that's used for the source code. @DamianEdwards can you confirm?

[omajid]

@bleroy Oops. Please carry on, then :)

[davidfowl]

Worse, if a user downloads .NET Core from Microsoft, it will suddenly appear to work on RHEL. I would certainly see this as the .NET Core version we ship being broken. Which is why we prefer to avoid this solution completely, if possible.

This can't not be the default though. Everything is pretty much designed around this feature. It's the only reason we're able to use the .All package by default.

Worse, if a user downloads .NET Core from Microsoft, it will suddenly appear to work on RHEL. I would certainly see this as the .NET Core version we ship being broken. Which is why we prefer to avoid this solution completely, if possible.

Help me understand why that's not acceptable. How do people deploy to rhel today?

[tmds]

@davidfowl I think the bullet list @omajid made earlier sums it up nicely.

• Microsoft builds DEBs and RPMs for the package store, that will be published on Microsoft feeds.
• We'll provide a shell script that takes care of installing those same store bits for the cases where the Microsoft feed or native installers aren't available or can't be used.

Is this in progress? Can we try it out?

[omajid]

How do people deploy to rhel today?

Based on what I have heard, there are two common scenarios. When I say RHEL, it can be a bare metal OS, a virtual machine or a container.

1. Build on another OS, deploy on RHEL. In this case a developer builds their .NET Core application on another OS, publishes it and deploys it on RHEL.

2. Build on RHEL, deploy on a different RHEL instance. In this case a developer builds their application on RHEL and publishes it and runs it on another RHEL instance.

We generally suggest framework-dependent deployments since the RHEL package for .NET Core can work around various deficiencies in system-wide version of native libraries. For example, you get better http client support in your .NET Core application via a newer libcurl linked against OpenSSL. There are other benefits too, such as not having to rebuild your application if coreclr or another runtime component get a security update.

With the assumption that ASP.NET Core is available in the runtime store, this application fails to deploy. A user can use a Microsoft provided build of .NET Core on RHEL and this application will work. Or they can take their application to another operating system (and implicitly use the Microsoft provided build) and it will also work. The result: user thinks our builds of .NET Core is broken.

[bleroy]

On the licensing front, here's confirmation: https://github.com/aspnet/Coherence-Signed/issues/564 This is in the 2.0 milestone. Thanks @Eilon for this.

[omajid]

I can't access https://github.com/aspnet/Coherence-Signed/issues/564; I wont be able to provide any comments/feedback on that.

[bleroy]

I swear I was going to check that this was a public repo before I posted, and then I derped and didn't. Sorry about that. OK, let me copy the gist of it here, and let's comment here for the moment.

Change ASP.NET/EF Core NuGet package licenses to Apache

In other words, have the NuGet packages have the same license as the source code.
Eilon added this to the 2.0.0 milestone 44 minutes ago

[tmds]

@bleroy dotnet is installed to a non-default location on rhel (i.e. not /opt/dotnet), it would be good to verify that the rpm&script for the asp.net core store actually works. Who is working on this?

[omajid]

@tmds The RPM work is here: https://github.com/dotnet/cli/pull/6830

[muratg]

@tmds @JunTaoLuo is working on the runtime package store linux installers/scripts. Would you be interested in giving it a spin once we have the installer ready-ish?

[tmds]

@muratg yes, definitely. We'll want to ensure this works on rhel.

[aspnet-hello]

This issue is being closed because it has not been updated in 3 months.

We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.