Sdk: CLI Usage of stdout v. stderr

[TheRealPiotrP]

We need to identify how CLI takes advantage of the stdout and stderr streams to set good expectations and help folks consume the CLI effectively, particularly in pipelining scenarios. I'd love to first get the community's thoughts on what stdout and stderr mean to them. Once we come up with guidance we want to follow we can use the issue to track cleaning up the CLI and its scripts.

/cc a bunch of folks who will have opinions
@nguerrera @juergenpf @blackdwarf @glennc @jaredpar @rainersigwald @MattGertz @davidfowl @eerhardt

[blackdwarf]

I would say, from my perspective, that we should use these two streams as they've been designed, that is:

• Operational output goes to STDOUT
• Errors and similar go to STDERR

Another thing we should consider doing, to be good *nix citizens, is to identify dotnet in the error messages, that is, prefix each error message with dotnet:.

[TheRealPiotrP]

@blackdwarf I've failed to find a good explanation of STDOUT vs. STDERR

@nguerrera brought up an interesting point that STDOUT may benefit from being pipelined to other apps/scripts. STDERR, in this scenario, would get things like diagnostic messages.

I think this topic merits some discussion and I'd love to get some community feedback before we write down what we intend.

[blackdwarf]

@piotrpMSFT http://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout is a pretty nice explanation.

TL;DR: STDERR is for diagnostic messages, while STDOUT is for normal app output. Since they are separated, they can be redirected independently, which means that if I redirect STDOUT into a file, STDERR will still show error messages on the console. Actually, this use case was the primary reason the stream was introduced. 🙂

Of course, I'm not saying we should do as I specified, I was just sharing my opinion on the topic.

[vcsjones]

I would say, from my perspective, that we should use these two streams as they've been designed, that is:

• Operational output goes to STDOUT
• Errors and similar go to STDERR

Yes I agree. Certain build tools and scripts treat output from each distinctly. For example, if we receive output on stderr, the build server may assume that the build has failed (in addition to checking the exit code).

Consider if we had a very noisy build on STDOUT, but we still wanted to receive STDERR to a log file. Perhaps we would do something like dotnet myapp.dll > /dev/null 2> error.log. This would throw away the "noise" but log all of the STDERR to the error.log file. Another option is dotnet myapp.dll > /dev/null which doesn't redirect STDERR, so it prints to the console normally, but STDOUT is thrown out.

[jaredpar]

When making the decision for CLI I think we should take a look at our existing tooling and see if it sets up a pattern here. Looking at the tools involved in the CLI toolchain I know for instance the compilers never use StdErr. Even error messages go to stdout.

I believe MSBuild is the same but wouldn't swear to it.

[rainersigwald]

MSBuild does put everything on stdout. But I don't think that's the right design, and I'd rather go with the industry standard (primary output to stdout, warnings and errors to stderr) for any new tool.

[jaredpar]

@rainersigwald Every MS tool I can find for dotnet uses stdout for all output. Why have this tool be an outlier in our dotnet pipeline?

[bricelam]

Instead of asking what to log where using broad categories like "warnings" and "primary output" (no idea what that means), shouldn't we start at the consuming end: What text would I actually want to pipe to another command, or what text would get in the way of that?

[bricelam]

My opinion: Differentiating between stdout and stderr only makes sense when all of stdout is expected to be in a well-defined, consumable format, but you also need to report errors that don't match that format. (Think of tools like diff or ls.) Otherwise, I don't really see the point of using stderr at all. As stated, it just creates a big hassle with things that treat anything on stderr as an error (even though it may just be a warning or diagnostic information).

[WernerMairl]

i like the discussions about this - i see the same question in a lot of other (console) projects and i hope that we come to a design decision that can be documented in one of the design repositories!

One aspect that comes in mind:
i see a lot of console applications (including msbuild) that are using colorized output
(red for errors, yellow for warnings etc.)

In my understanding (not 100% verified) , ConsoleColor can only be used with Console.WriteLine
and Console.WriteLine writes always and only to stdout.

Sample: ConsoleLogger from ASP.Net LoggerFactory (Microsoft.Extensions) is using colorized Output for Trace and Debug Messages.
Trace and Debug should (Unix-Convention) go to stderr, currently i see no way to write to stderr in a colorized way => feature loss.....

I can imagine, that one reason for non using stderr in the past was the easy usage of colorized (red) outpout on stderr.....

my personal long term vision for this:

• consuming app should use (int) exitcode as first and primary detector about success/not success
• verbose outpot goes to stdout
• trace and debug output goes to stderr
• if stderr is printed out into the default console window then we need a way to use colorization

[ghost]

If compilation fails, at least the error code should indicate that, so downstream scripts can make decisions?

# in my script
msbuild non-existing.sln

if (!$?) {
  echo "this will never hit"
}

Comparing it with a popular tool:

# powershell
git blah

if (!$?) {
  echo  "blah is not a command dude!"
}

git status

if (!$?) {
  echo  "if you are reading this, that means you ran this outside of git repository"
}

[rainersigwald]

@kasper3 MSBuild sets errorlevel, and I don't repro that:

s:\msbuild>msbuild non-existing.sln
Microsoft (R) Build Engine version 15.7.66.2115 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1009: Project file does not exist.
Switch: non-existing.sln

s:\msbuild>echo %ERRORLEVEL%
1

s:\msbuild>powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Loading personal and system profiles took 2137ms.
S:\msbuild [core-node-reuse +0 ~2 -0 !]> msbuild non-existing.sln
Microsoft (R) Build Engine version 15.7.66.2115 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1009: Project file does not exist.
Switch: non-existing.sln
S:\msbuild [core-node-reuse +0 ~2 -0 !]> echo $?
False

[jasonkarns]

Sharing my experience (which brought me to the repo to search for precisely this issue)

My attempt to iterate over migrations so that I could progressively apply-then-rollback failed because the build output is first sent to stdout (Build started... Build succeeded.)

for m in $(dotnet ef migrations list 2>/dev/null); do

Second scenario: naively intuitive attempt to generate the migration script fails for the same reason (diagnostic build information also goes to stdout).

dotnet ef migrations script 2>/dev/null > migrate.sql

Both of these scenarios were the _first_ thing I reached to as a long-time *nix/CLI user. That is what is conventional in the ecosystem and is what most (nix/cli) users would reach to intuitively. That these norms are broken makes the tool feel broken or at least unpolished and out of place. It gives one a feeling that the cli is a second-class experience (the first-class experience assumed as "use Visual-Studio-on-Windows").

[bricelam]

@jasonkarns Can you submit a new issue on dotnet/efcore? I've had similar thoughts recently on making these commands more scriptable. Currently, the best way to consume these commands is to use the --prefix-output and/or --json options combined with tools like grep or awk. Definitely not ideal.

[jasonkarns]

@bricelam sure can! I had thought about doing that, but assumed that since most of the undesirable output was coming from msbuild, I assumed this would need to be handled by the cli. I'll get an issue written up soon, though, thanks!

Adding another "broken" scenario so I don't lose it:

dotnet ef database --project MyProj --context ContextA update --no-build

This command executes _successfully_ by returning an exit code of zero; but the command fails to do anything. So the exit status is misleading in a catastrophic way: shell scripts using this command continue as if the command were successful!

[bricelam]

IIRC, we already intercept all that output using ProcessStartInfo.RedirectStandardOutput. We can very easily write it to stderr instead.