Sdk: Unify names of argument --source-feed vs -s,--source

[ghost]

Steps to reproduce

Try:

dotnet nuget push -h

Usage: dotnet nuget push [arguments] [options]

Arguments:
  [root]  Specify the path to the package and your API key to push the package to the server.

Options:
  ...
  -s|--source <source>           Specifies the server URL
  ...

then:

dotnet tool install -h

Usage: dotnet tool install [options] <PACKAGE_ID>

Arguments:
  <PACKAGE_ID>   NuGet Package Id of the tool to install.

Options:
  ...
  --source-feed <SOURCE_FEED>   Adds an additional NuGet package source to use during installation.
  ...

Expected behavior

dotnet tool install should have -s, --source switch instead of --source-feed without (unreserved) -s alias

Actual behavior

dotnet nuget push has -s, --source, and dotnet tool install has --source-feed.

Environment data

dotnet --info output:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.300-preview3-008443
 Commit:    1e2e8138b9

[wli3]

@KathleenDollard

just add more context: --source-feed is additional while --source is exclusive. We do like source to be additional, however we don't want to introduce breaking change on existing restore.

Another factor is, we realize people frequently get confused by --source opinion in dotnet tool install context. (almost everytime, people ask for what is "--source" in early meetings)

[KathleenDollard]

@wli3 We are currently additive, correct?

Didn't we find another CLI command that used --source-feed that pushed us over on this issue?

[ghost]

@KathleenDollard, dotnet tool install and dotnet tool update are the two commands using this option. If it is possible to change the behavior of dotnet nuget push --source to be additive as well, and then align the names of the options (with -s alias), please consider it. :)

[wli3]

@KathleenDollard confirm, it is pretty much the same as @kasper3 said

use source-feed
dotnet tool install
dotnet tool update

use source
dotnet add package
dotnet nuget
dotnet restore

[KathleenDollard]

OK, I'm still concerned

• source-feed is additive, source on the other features is not
• almost everyone was confused by just "source" and assumed it was a "run from source" feature that would be sensible in this context.

So, don't change it yet, but I'd like more opinions.

[richlander]

I think we need to adopt a standard dialect across the CLI. If something is confusing, we have to accept that. The standardization across the whole nomenclature is super important and trumps most everything else. I also don't find --source the least bit confusing. The first thing I tried using this command today was -s to specify a feed and that didn't work. Why break muscle memory for advanced users. See: dotnet/cli#9131.

The additive, not additive thing is also confusing. Why do we need a different behavior for tools? Way too subtle.

[livarcocc]

I disagree, I don't think we should try to keep all the same just for the sake of being standard. I believe that if something is following the standard and being confusing to our users, we should break that standard in that particular case. That was the feedback we got from tools install regarding --source, that people did not understand what it meant in the context of tools, and so we changed.

As for the additive, we need to be able to resolve the Microsoft.NetCore.Platforms package (or we did at some point) and therefore needed the standard feeds to still be available when restoring the tool, so we needed to make it additive. Maybe we don't need it anymore (and know we won't need it in the future) and could change it. But while this was one encouraging point to change --source to --source-feed, it wasn't the only one. To me, the biggest point was the feedback we got about it not being clear.

[ghost]

@livarcocc, if --source is confusing for folks discussing dotnet-tool-install, then isn't it equally confusing for dotnet-nuget-push, dotnet-restore, dotnet-add-package? We can always have a breaking change to align these, for example when revving the major version so existing usages will not be affected.

I agree with you that they don't have to be same, but if the other commands's source-like options are discussed with same arguments, it would yield either similar or better/sane names.

[livarcocc]

I would argue that the commands listed above are more obviously related to restore (and NuGet) than dotnet tool install.

[ghost]

Sure but does the relation with restore/nuget makes it obvious that names cannot be improved or --source is the most suitable name?

[tmds]

It would be nice if dotnet tool install provided a --source parameter that is consistent with the other commands: i.e. not additive.

I like this consistency since we are not going to remove --source parameter from the other commands and we are used to how it behaves.

I see this when changing some scripts. I want to pass the same feeds to dotnet tool install as I am passing to dotnet restore and suddenly I need to deal with different semantics :confused:

Whether it makes sense to have an additive source parameter is a different question. A --source-feed (I personally like --add-source) can be added to dotnet tool install and other commands if it useful.

[richlander]

I like your proposal, @tmds.

[clairernovotny]

One question regarding additive vs exclusive feeds. For tool installs, do you think things may be more likely to fail if they're exclusive?

Scenario is putting up a CI build of a tool on a MyGet feed. Only the tool is there, any dependencies it might need are not (or do tools never have dependencies)? How likely will it fail if people use --source and the MyGet isn't configured to pass through to nuget.org?

[KathleenDollard]

We need additive feeds for tools to support Enable tools RID filtering availability dotnet/sdk#9088

I do not want the same switch name for an additive and an exclusive feed.

That does not mean --source-feed is the best name.

We want to think this through carefully because if we make a change it will be sufficiently close to release that we will not have an opportunity for feedback from the broader community.

Thoughts?

[richlander]

Can you elaborate on dotnet/sdk#9088? I cannot get enough context from that issue.

[wli3]

@richlander @KathleenDollard
To support RID filtering (which is a prerequisite of self contained app) we need to reference the package Microsoft.NETCore.Platforms (it will bring the RID graph, it can help to find if "ubuntu" can use "unix" asset). If the user use exclusive --source. The package cache(nugetfallbackfolder) will be excluded and the package cannot be found.

It is the same reason in preview 1 we suggest producer to use nuget.config, instead of --source to point to their result nupkg folder.

Global tools can only support "any" as RID. So it is not a problem today

[richlander]

Why would you need MS.NetCore.Platforms as part of install? I would have expected that as part of pack.

[wli3]

@richlander this package need to be up to date(as in the same level of SDK). If the rid graph is packed in. A new rid comes out later will not be able to install it. It happened to osx.10.13. https://github.com/dotnet/cli/issues/8436

[tmds]

@wli3 I'm trying to follow the discussion. Is this correct: currently tools are restored on install using the 'any' rid. There is a desire to use the system rid, to trim the tools on install. This requires a 'Microsoft.NETCore.Platforms' package to figure out the rid graph.

We need additive feeds for tools to support Enable tools RID filtering availability dotnet/sdk#9088

So when @KathleenDollard is saying 'we need additive feeds', this is not a user requirement; but a technical requirement to have a way to add 'Microsoft.NETCore.Platforms' as part of the tool restore?

Scenario is putting up a CI build of a tool on a MyGet feed. Only the tool is there, any dependencies it might need are not (or do tools never have dependencies)? How likely will it fail if people use --source and the MyGet isn't configured to pass through to nuget.org?

@onovotny afaik the tools are supposed to pack all their dependencies. So a single source should suffice (except for obtaining the 'Microsoft.NETCore.Platforms' package).

[tmds]

@KathleenDollard @livarcocc @richlander can you please update milestone to 2.1.3xx?

[KathleenDollard]

@tmds Yes, you explained the technical scenario: the need for an updated RID graph. And you are also correct on packing dependencies. That original design was to avoid the problem.

The user scenario for system RIDs is somewhat broader: We want to support tools that are platform specific and it supports signing.

Re confusion (which I thought too much about last night): I think there are two types of confusion. IIRC @richlander was one of the people that said "What's that?" about --source and I think hoped it was source build. So there is initial "What's that?" confusion and at the other end of the spectrum confusion when things you use every day waste cycles. If we were to have --source be additive in one case and exclusive in the other, that would be on that bad second end, and I think we're all against that.

So:

Are there other ways to get the RID graph and avoid this technical implementation detail?

Should we change to --add-source-feed or --add-source and consider including this as an option on restore, etc in the future? source-feed is only different, and not clarifying.

-a is not currently in use for restore. Is there any meaning to -a in the broader package installation space that we would block by using this abbreviation?

I agree with @richlander that we want consistency and a standard dialect. This involves thinking in the broad space about naming. Since source has a few different meanings across the CLI space, I favor source-feed and offering add-source-feed and source-feedon restore in the future. It's longer, but there's a good abbreviation and I hope we will have a good tab completion story in the nearish future.

[tmds]

@KathleenDollard @wli3 another technical consideration is .NET Core source-build currently doesn't have a sdk/NuGetFallbackFolder. So additive isn't adding 'Microsoft.NETCore.Platforms'.

[KathleenDollard]

@tmds me:face palm

[tmds]

@KathleenDollard me:face palms too... these differences are a pain.

[livarcocc]

Actually, the additive works here because it will then fallback to your NuGet.Config, that by default includes a nuget.org, so, it won't pick that from the fallback folder, but it will find that from internet.

[livarcocc]

I moved to release/2.1.3xx, but mostly so that we can track the conversation in the milestone, I am not convinced of the change yet! 😃

[richlander]

I think that downloading a new RID graph is a bad idea to start with. If the existing RID graph in user's SDK doesn't have the right info, tell them to download a new SDK. That's amazingly simple UX. Many customers don't want us downloading arbitrary files over the internet. This will become particularly important as we support local/repo tools that people install in their builds.

Let's not do magic. It never ends well.

[wli3]

@richlander

1. it will not download extra if it is in nuget fall back folder.
2. download new SDK will not solve this issue. You suggest add this step in pack, so the RID graph will be the locked version. As long as new RID is not in the locked rid graph stored in nupkg, I will not install.
3. there is no different than creating a new project and restore it. If you think Microsoft.NETCore.App is arbitrary
4. this is the opposite of magic. We will use the standard way of delivering RID specific content of nuget.

[KathleenDollard]

I want to be sure I've got this.

2) Is this the problem we are avoiding: _The user specifies a valid new RID, which is not in the graph in the SDK on the user's machine. With the right RID graph, this new RID would fall back to a another RID, so the only problem is that the local graph doesn't know what to do with this new RID. Thus, there is no other reason to download a new SDK. _

3-4) In the scenario of the new RID, what does dotnet restore do when --source (exclusive) is used? Why is this different?

[richlander]

I talked to @wli3 ... Here is what I learned:

• dotnet restore -s is exclusive with -s but does not treat the NugetFallbackFolder as a feed, so consults it, if needed.
• dotnet tool install --source-feed needs to be additive because it treats the NuGetFallbackFolder as a feed and needs to consult it for the platform package (the one that contains the RID graph).
• This difference in behavior was added to enable us to ship with tools in the NuGetFallbackFolder to enable offline installation. We don't rely on this / have any offline tools today (in 2.1.300).

That leaves us with a few options:

1. Status-quo.
2. Adopt the proposal from @tmds, which is change --source-feed to --add-source or some variation. The benefit here is that the behavior and syntax is clearer. It continues to break muscle memory between commands but a clearer reason and we may opt to add add-source to other commands like restore since (again) it is a more differentiated behavior/syntax.
3. Adopt the same behavior and syntax as dotnet restore. We'd have to cut the offline install of tools that I just mentioned above.
4. Adopt the same behavior and syntax as dotnet restore with a couple tweaks so that everything can work: special case the platform package (it can be found in the NuGetFallbackFolder) and disallow people to use -s for installing the tools that are made available for offline install in the NuGetFallbackFolder. -s doesn't make sense in that case anyway.

My take is that we're past the time in this release to address feedback that comes with risk or significant development cost. I've ordered them from lowest to highest cost/risk. I think that the proposal from @tmds is a generally good one, so that feels it hits the sweet spot. It's not a one-off choice for tools and it is cheap. I think we need more time to think about the last two or to consider other options. I'm not yet fully sold on the offline tools story, so I don't want to commit to anything around that in this short timeframe. That brings me back to the proposal from @tmds again.

Thoughts?

[ghost]

--add-source sounds like a like a really good choice that will (probably) mostly be rename refactoring in terms of implementation.

My suggestion was to also consider a shorthand, -a sounds like a reasonable option. However, if this doesn't warrant reserving that shorthand based on the usage/pervasiveness scale, that would be a good reason not to introduce it and instead save this shorthand option for something more popular, yet to come, in future.

For reference, in some tools/package-management utilities I use, they refer "--source" as "--repository" with shorthand "-k" (strange choice of letter but good for compactness, less finger dance etc.).

[KathleenDollard]

I think we should change the wording only:--add-source as new wording for the current --source-feed option. No changes to the current design of the feature.

This is a very low risk change.

Assuming the docs are complete in the abbreviations, -a is currently used only for --test-adapter in the .NET Core CLI. It means "all" in at least the bash ls context. I am fine using this abbreviation if no one can think of a probable future use that would conflict/be confusing.

[tmds]

@richlander @wli3 very interesting to learn how behavior differs between dotnet restore and dotnet tool install.

@KathleenDollard I think we need to consider adding a '--source' to dotnet tool install for the next iteration. I understand this is not considered now due to the risk involved.
The motivation is that in some environments direct access to nuget.org is blocked. All packages go through a local caching nuget server. In this case you want to explicitly control your external sources.
If RID trimming requires a package from the SDK, we shouldn't have to specify it as a '--source'. This is internal implementation. We also need to make sure that package is part of the source-build SDK.

To have this type of behavior already I intend to generate an empty NuGet.Config file and pass it as the --configfile parameter. This should make --add-source behave like --source.

[tmds]

_The user specifies a valid new RID, which is not in the graph in the SDK on the user's machine.

@KathleenDollard It's interesting that the user doesn't specify an RID (dotnet tool install doesn't accept such an option). The SDK selects an RID it doesn't know... Probably this is due to the SDK deriving the RID directly from OS information. So my SDK may be working on Fedora 27 and when I upgrade to f28 this may cause dotnet tool install to fail (when f28 is not a known rid). This is also something we need to look at.

[KathleenDollard]

@tmds Yes, f27->f28 is exactly the scenario we want to support.

@livarcocc Do we need anything else to go forward this name change? No other changes just --source-feed changes to --add-source and has the -a abbreviation.

[richlander]

I think we should avoid adding the -a short-hand until we feel like we've fully landed this design. I think --add-source is good, but I'm not convinced we're done here (meaning, for next release). I'd suggest we expose the minimum amount of surface area to get the job done. We'll get more feedback, which should guide further design changes.

[tmds]

I think we should avoid adding the -a short-hand until we feel like we've fully landed this design.

+1

[KathleenDollard]

OK,

@livarcocc @wli3 Change name to --add-source and do not add an abbreviation at this time. I will handle docs and release notes.

[livarcocc]

Sounds good to me.