Docs: Poll: Should runnable samples include output as comments?

[BillWagner]

this comment highlights a question we've been wrestling with: should runnable samples include the output as comments or not?

The advantage of including comments is that you can see what code should do without running it. The advantage of not including the comments is that the samples take less space, and after running them, you can see the output and the code without scrolling.

Anytime we've asked, the responses have been almost evenly split. Try these two pages and vote:

Example where comments are included Vote 👎

Example without comments for output Vote 👍

[bryanjhogan]

I like seeing the comments because it lets me quickly see the output of some method and there might be many examples on a page. Sometimes I just want to glance at the docs to help choose between a few alternatives, e.g. the date formatting.

Suggestions - would it be possible to leave the comments in, but as soon as a user edits the code or runs the sample, then remove the block of comments from the sample?

[svick]

I think being able to see the output without running the code makes sense when the specific string matters, like with Console.WriteLine(dat1.ToString(CultureInfo.InvariantCulture));. But in general, I think it's mostly unnecessary, like with Console.WriteLine(dat1.Equals(DateTime.MinValue)); or the foreach loop.

[ChrisMaddock]

Option 3: it would be really cool if the page could load in this state in the first place! Presumably sample outputs could be cached?

This option requires some development, of course. 😄

[pkulikov]

@ChrisMaddock that would be perfect! Then one can edit code immediately, without the necessity to run it first as it is now.

[mairaw]

Adding @LadyNaggaga and @yunyunyunyun here for some suggestions on the interactive sample experience.

[LadyNaggaga]

@ChrisMaddock I like the idea of preloading the output. However I am worried that people might not run the code.
@BillWagner I am okay with the comments in the output because you also let the user know they can run the code. Have user not responded well to it ?

[BillWagner]

@LadyNaggaga The small sample in the votes above mirrors the responses I've gotten every time I show this in a conference: it's almost exactly split evenly for "I'd rather run the code" and "i'd like to know what it does when the page loads". No consensus is emerging. 😕

I like @ChrisMaddock 's idea if we can implement it. It would give users both in a way.

If that proves really difficult to implement, @svick 's guidance can help us do better in the content.

[sputier]

TL;DR; I think we should keep comments

As a developer with a bit of experience, I think the comments in 101 samples are "distracting", and I appreciate them in the advanced code samples.

... But 101 samples are not designed for those who already know the language : beginners probably need comments in basic code samples as much as I appreciate comments in advanced samples.

[SpaceShot]

At first I was going to come in here and say obviously Yes. Then I tried both interactive samples and I'm definitely more undecided. Removing the comments tidies the overall length of the document, and running the code is "cool". However, after that wears off, the seasoned dev might want to just see the output and get back to work. Sure it's just a few seconds but I see that angle too.

[jkaster]

I'm a huge fan of documentation, code comments, and reference implementations for users. However, this site provides the Run button in situ, so the user can look at the sample, mentally challenge themselves to see if they understand it, then hit Run. Or, just hit Run to see the output anyway. So I think omitting the comments when the output can be shown directly below the sample is the way to go.

[scottaddie]

My thoughts mirror those of @sputier. ☝️ I do, however, think there's a point at which the comments become unnecessarily noisy. We should always try to limit the number of them.

[nemec]

I vote for, at the very least, caching the response 😐

Until Blazor launches and we can run the samples client-side!

[rkeithhill]

One thing I don't like about the commented output is that the copy button copies the code and the commented output. That is just more lines of code I have to delete.

BTW why does the copy button go away after you run the code snippet? That's unfortunate.

[IEvangelist]

I prefer it without the comments. They seem a bit redundant making them less useful. Also, if the user is to change the variables - are the comments with the expected output going to change too? I think it is clean and simple enough to have it where upon executing the Run the Output is displayed.

Comments just muddy the waters for me, especially when they're simply stating the expectations. The beauty of this tool is that the user can interact with it, learn from it, edit and re-run and see it for themselves. I do want to take a moment to thank everyone on the team for this, it is utterly amazing and I love where you're going with it. Thank you!

[SeanKilleen]

It's a tough call for sure. My instinct is to recommend removing comments. I know that they can be helpful, especially for beginners, but I think losing them is the right way to go because:

• The concept of clicking a green "run" button doesn't seem to be a far-removed concept that would confuse people
• Those who are beginners may not be in such a hurry that they would need to see comments at a glance. Running doesn't seem to be a heavy lift from a time perspective.
• By keeping comments in a code sample, especially one that can be edited where the comments would then be incorrect, we perpetuate the idea to beginners that trivial comments like that might be appropriate to document their production code, when oftentimes testing the result of code would be preferred.

My last thought there has been gnawing on me so I'll just say: why not present the comments in the form of xUnit tests instead? That might be closer to the behavior that we want the community to model. At first I thought I'd be joking about this, but now that I've said it...

[cce32]

👍prefer the compact look without the comments.

[fhoo]

I vote for @ChrisMaddock option 3. If that’s not an option I‘d go for 👎 at least until the run button works reliably in all browsers.

[bryanjhogan]

@BillWagner this post http://www.felienne.com/archives/5947, by Felienne Hermans is interesting, it discusses how children learn to program and the particular difficulties they have with the meaning of symbols.

It leads me to a few questions -

• who is the primary audience for runnable samples?
• do their needs outweigh the needs of the other audiences?
• would it be possible to have two versions of the docs - one for experienced devs and one for novices?

[Felienne]

Thanks @bryanjhogan for sharing our work!

Please do include the example outputs. I understand it will make the code longer, but for novices this is of very high importance. One of the activities that makes learners better at programming is 'tracing' which basically entails executing the code in your brain and predicting the outcome. The example output helps with that a lot!

And yes there is the "run" button, but that might not be all that easy to find for novices and also it takes quite some time to run.

I could also see how you would make two versions.

[guntbert]

Being able to check my results against the "correct" results is well worth some effort (IMO).
Thats why I'd prefer comments even to @ChrisMaddock's excellent idea.
One thing comes to my mind: make displaying the expected results quasi optional by

• providing them as a "tooltip", shown on hover or
• having a section of the page fold/unfold (UX hint: down arrow or plus sign)

[crowcoder]

Agree with @guntbert , I'd like to see how the output in a folding section looks.

[mika76]

I vote keep the comments. Quicker to see and react. Lots of times I just need a refresher on exact output, and comments would be much quicker to see.

[rkeithhill]

How about always showing the output pane but prepopulated with the expected output until the user presses the Run button? When the Run button is pressed the new output overwrites the comment output.

[BillWagner]

First of all, thank you to everyone that responded. The votes, ideas, and comments have given us a lot to think about.

One clear message is that this question isn's so simple as always display comments, or never display comments.
The right choice depends on several factors:

1. The intended audience
2. The goal of the article
3. The nature of the output

There are likely other factors we haven't discovered yet.

Let's start by creating guidance for when to include comments, or output when using the current version of the REPL.

When to display the expected output without running the sample

• Articles intended for beginners should provide output so that readers can compare the output of their work with the expected answer.
• Samples where the output is integral to the topic should display that output. For example, articles on text formatting should show the text format without running the sample.
• When both the sample and the expected output is short, consider showing the output. It saves a bit of time.

Notice that I didn't use the phrase 'comments' above. We haven't experimented with showing the expected output in the article text. That is worth testing.

When to exclude expected output from the sample

• Articles where the sample generates a larger output should not include that in comments. It obscures the code once the sample has been run.
• Articles where the sample demonstrates a topic, but the output isn't integral to understanding. For example, code that runs a LINQ query to explain query syntax and then display every item in the output collection.

Other notes from comments

@SeanKilleen mentioned the concept of using xUnit tests:

My last thought there has been gnawing on me so I'll just say: why not present the comments in the form of xUnit tests instead? That might be closer to the behavior that we want the community to model. At first I thought I'd be joking about this, but now that I've said it...

We've thought about this. The concern for shorter topics and beginner topics is that it introduces more requirements to understand the sample. "I came here to learn how to format strings. Now, I have to learn xUnit first." For samples that represent full programs, we agree that explaining test strategies is an important part!

New site ideas

This has also generated great ideas for updates to the REPL. I've created new issues at https://github.com/dotnet/try for these features

• Pre-load the REPL window with the output from running the code. dotnet/try#63
• Display expected output in an optional window MicrosoftDocs/feedback#251

Thanks again. I'm creating a PR that updates our contributor guide with this information. We'll gather data and continue to improve.

[rpetrusha]

There's also one other issue that's emerged recently as a result of your PR on string comparison, @BillWagner. For culture-sensitive comparison, .NET relies on information supplied by the operating system, which can be very different across particularly .NET Core versions. In that case, it would seem, output either shouldn't be included, or should be included only with the caveat that it reflects output for a specified platform using a specified culture.