Sdk: Investigate requirements to support fish shell

[wli3]

_From @leecow on June 3, 2017 15:59_

Submitted by @egilh

I'm not able to install dotnet-dev-2.0.0-preview1-005977 on Ubuntu 16.04 via instructions found here https://www.microsoft.com/net/core/preview#linuxubuntu when I have fish as my default shell.

Unsupported use of '||'. In fish, please use 'COMMAND; or COMMAND'. fish: dotnet new > /dev/null 2>&1 || true ^ dpkg: error processing package dotnet-dev-2.0.0-preview1-005977 (--configure): subprocess installed post-installation script returned error exit status 127 Errors were encountered while processing: dotnet-dev-2.0.0-preview1-005977 E: Sub-process /usr/bin/dpkg returned an error code (1)
Doesn't matter if I switch to bash or not, the error is the same. To be able to complete installation, bash needs to be set as default shell
chsh -s /bin/bash
Installation works as expected after this.
Using fish as my default shell is a big advantage for me (and others). Please make sure Debian/Ubuntu packages allows for the use of fish as default shell.

_Copied from original issue: dotnet/core-setup#2613_

[wli3]

_From @ellismg on June 6, 2017 0:49_

The problem is likely due to this file:

https://github.com/dotnet/cli/blob/master/packaging/deb/postinst

I think the problem is that when we SU we launch that command using the user's shell instead of sh.

[wli3]

_From @ellismg on June 6, 2017 1:21_

FWIW, that whole postinst script seems suspect. I don't think we should be doing first run stuff during install. Also consider the fact that when installing many packages a user may not see the first run text regarding telemetry.

Also, it seems to assume that $SUDO_USER is always set which would not be the case if someone logged in as root or just used su to obtain root.

[wli3]

_From @omajid on June 6, 2017 13:25_

Repating my comments from the PR (https://github.com/dotnet/cli/pull/6480):

Sorry for jumping in here very late. I suppose this same patch (or a derivative) will apply to RPMs too. I have a few questions:

This is important for Visual Studio.

Can you elaborate? Visual Studio doesn't work on Linux so this can't possibly impact Visual Studio, can it?

The same time the telemetry message should be shown in the installer

If this is a non-interactive installation, messages from the package install may not be shown to the user. And since the first run cache exists on subsequent runs, the message is never shown again.

The dotnet new invocation is a surprising implementation step that works differently under different scenarios:

• If there are 2 (normal) users on machine, only one gets the cache when .NET Core is installed.
• If root installed packages (no sudo used in installation, possibly via some GUI software center) root gets nuget cache, no one else does.

Packages touching a user's files (in this case under /home/$SUDO_USER) is generally a no-no: https://stackoverflow.com/questions/3140886/debian-packaging-deploying-files-to-the-user-home-directory

[wli3]

_From @omajid on June 6, 2017 13:27_

Also, it seems to assume that $SUDO_USER is always set which would not be the case if someone logged in as root or just used su to obtain root.

I think this is actually handled correctly. It is not quoted, so if it is undefined, the shell sees:

su - -c "dotnet new > /dev/null 2>&1 || true"

Which looks valid to me.

[wli3]

_From @omajid on June 6, 2017 13:29_

A quick and dirty fix would be adding --shell /bin/bash, but I agree that the entire script seems suspect.

[wli3]

_From @ellismg on June 6, 2017 23:52_

/cc @wli3 @livarcocc

Can you help @omajid and I understand why we do this? Is it so important to do this at install time? Note that is's highly possible the users would not see the telemetry message. I get that we'd prefer not to have the expansion on first run, but this thing seems fragile.

[wli3]

_From @ellismg on June 6, 2017 23:54_

A quick and dirty fix would be adding --shell /bin/bash, but I agree that the entire script seems suspect.

If we do this, we should at least try to use /bin/sh, I guess.

I'd rather we just delete this, personally.

[wli3]

The pull request is here
https://github.com/dotnet/cli/pull/6480

As stated in the pull request, the telemetry is added as part of the installer, just for the missing telemetry message.

The detailed reason and related work is here
https://github.com/dotnet/cli/issues/6370

We are invoking "first run experience" as part of the installer to prime cache, which is an old feature that considered multi user scenario. It will be triggered if the other user don't have the cache. The user will need to wait, but it won't throw

[wli3]

_From @ellismg on June 7, 2017 0:13_

@richlander The experience seems pretty busted to me (at least on *nix). For Windows and OSX, maybe this makes sense, but it's pretty orthogonal to how things are usually done, as @omajid points out.

If you wanted to keep the "when I install .NET I don't need to do the first run thing" I would expect that the way to structure this would be to have the package carry along a fallback source that's just laid out on disk at install time as a bunch of flat files (similar to how we handle the runtime store...). I think this is similar to what we do for VS on windows.

[wli3]

Hi @ellismg could you illustrate what is the broken scenario left? So far I covered telemetry notice, multi user experience, and run as root is also covered from previous discussion.

Again, the old experience is, the user install the package, and the first time the user run dotnet X, the user need to wait for the "first time experience" to run and prime the cache for about 1 minutes. This process is per unix user. People will get the same thing if install via tarball.

This is no different than the install experience of vim.spf13. As part of the install process, the payload, vim, will start and run _vundle install_ to install plugins for about 5 minutes. If it is not the part of the install process, the first time user want to type vim will have to wait for 5 minutes -- which is very frustrating

[wli3]

_From @ellismg on June 7, 2017 18:26_

Hi @ellismg could you illustrate what is the broken scenario left? So far I covered telemetry notice, multi user experience, and run as root is also covered from previous discussion.

The thing that's broken in my mind is that we assume messages printed to standard out while installing the package make their way to end users. I don't think that's reasonable. For the case where install just the single package, on the command line, then I agree you have a high chance of seeing message. However, if you install the RPMs through a graphical interface, or as part of a larger process (e.g. apt-get installing many packages) then it's likely that that folks will miss the message and not understand that telemetry is enabled.

If we think there's value in priming the cache at install time, I'd recommend that we at least decouple the telemetry message from priming the cache and not print the telemetry message until the first time someone runs dotnet interactively. Long term, I would like us to figure out how we can move this cache priming to RPM build time (via teaching the tooling about some system wide fallback cache that's owned by the RPM itself and laid down during install time as a bunch of flat files). I don't think we need to do that for 2.0, but I really do think we need to both address both the bug where if your login shell is not an sh-like shell the install fails and we need to ensure that folks are likely to to see the telemetry message.

[wli3]

_From @omajid on June 7, 2017 22:49_

telemetry notice

Echoing Matt here. The assumption that the user sees the telemetry notice is not necessarily true. For example here is a related discussion about another package that shows another message at package install time which points out that users may or may not this this message at all. Mind you, these "rules" do not apply to the .NET packages at all, but they do point out reasons for why this is not a good idea.

multi user experience

Two things here bother me:

• The inconsistent multi-user experience. Installing dotnet runs the first-run experience for only some users. Depending on how it is installed (su - $package_manager install vs sudo $package_manager install) the first run experience may or may not get triggered for my non-root user. If you use separate user accounts, one gets the first run experience at install time, the other at time they first run dotnet. If I install as root, I now have a ~/.dotnet (and/or other hidden directories) that I never asked for.

• The package writes to a location and does not tell the package manager about it. Generally the package manager knows every location an "install" touches so it can undo it. In this case the package manager has no idea that the installation created more files that it doesn't know about. If I query the package manger it will not tell me about files that the installation created.

People will get the same thing if install via tarball

I don't think so. Either the tarball was installed systemwide or for that user only. Either way, extracting that tarball did not trigger first-run experience. Every user will find out about the user-specific dotnet cache when they run it for the first time. If root extracted the tarball, it did not create hidden files in root's home dir.

This is no different than the install experience of vim.spf13.

I think there's significant difference. That install is per-user. And it creates a per-user archive. The linux distro package manager install is a global install. It should create a global cache. And tell the package manager about it.

It definitely should not assume the user has seen the telemetry prompt.

[wli3]

_From @rakeshsinghranchi on August 2, 2017 20:9_

@wli3 can you please move this issue to CLI repo ?

[wli3]

_From @ellismg on August 2, 2017 20:50_

Just a note, based on something @omajid mentioned to me in the past, RPM and DEB should support xz compression (which should match the LZMA we used for the cache). So for the installers, it may make sense to just lay stuff down when building the directory we are going to scaffold the installer from and not ship the LZMA archive. That means when you yum install dotnet the package manager lays down (and owns) these files. If you remove it, they are cleaned up. They are shared system wide. Since we use xz compression, it shouldn't cause gigantic expansion of the installers.

[wli3]

_From @omajid on August 2, 2017 21:39_

You know, it always seems strange to me that we care about compressing one part (the lzma archive) but not the rest of the install... Seems to me like we should lay out everything on disk and then compress it using the best compression we can.

[livarcocc]

It seems like the whole discussion here is about running first run here and the user missing the telemetry message. If this is correct, this has now been fixed, since we won't create the telemetry message sentinel through the installer anymore.

if that's the case, we can go ahead and close this.

[wli3]

This thread does have several things going on. From my understanding, the original question is to invest if we can install dotnet CLI on a machine with default shall as fish. I tried on a clean Azure Debian 8. It succeed. So I am closing it

commands before installation

chsh -s `which fish`
exit

[ibqn]

The installation of
sudo apt install dotnet-sdk-2.0.0-preview2-006497

failed with on ubuntu 16.04

Setting up dotnet-sdk-2.0.0-preview2-006497 (2.0.0-preview2-006497-1) ...
This software may collect information about you and your use of the software, and send that to Microsoft.
Please visit http://aka.ms/dotnet-cli-eula for more information.
Welcome to .NET Core!
---------------------
Learn more about .NET Core @ https://aka.ms/dotnet-docs. Use dotnet --help to see available commands or go to https://aka.ms/dotnet-cli-docs.

.NET Core Tools Telemetry
--------------
The .NET Core Tools include a telemetry feature that collects usage information. It is important that the .NET Team understands how the tools are being used so that we can improve them.

The data collected is anonymous and will be published in an aggregated form for use by both Microsoft and community engineers under the Creative Commons Attribution License.

The .NET Core Tools telemetry feature is enabled by default. You can opt-out of the telemetry feature by setting an environment variable DOTNET_CLI_TELEMETRY_OPTOUT (for example, 'export' on macOS/Linux, 's
et' on Windows) to true (for example, 'true', 1). You can read more about .NET Core tools telemetry at https://aka.ms/dotnet-cli-telemetry.
Unsupported use of '||'. In fish, please use 'COMMAND; or COMMAND'.
fish: dotnet new > /dev/null 2>&1 || true
                                   ^

dpkg: error processing package dotnet-sdk-2.0.0-preview2-006497 (--configure):
 subprocess installed post-installation script returned error exit status 127
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Errors were encountered while processing:
 dotnet-sdk-2.0.0-preview2-006497
E: Sub-process /usr/bin/dpkg returned an error code (1)

Therefore this package is definitely broken, if you use shell scripting you should not guess what shell users have! This is poor and dangerous...

[ibqn]

~> grep ibqn /etc/passwd | cut -d ':' -f 7
/usr/bin/fish

[ibqn]

This thread does have several things going on. From my understanding, the original question is to invest if we can install dotnet CLI on a machine with default shall as fish. I tried on a clean Azure Debian 8. It succeed. So I am closing it

C'mon, are you sure that clean Azure Debian 8 has fish shell package installed?

commands before installation

chsh -s `which fish`
exit

Are you sure that you command succeeded without error message?

To be honest, it looks like your results are grounded on false statements!

[ibqn]

@wli3 ^^