Nugetgallery: Support formatting in description

[thomaslevesque]

It would be nice to be able to add some basic formatting (perhaps using markdown) in the free-text fields of the nuspec file (summary, description and releaseNotes). Currently it's hard to display the release notes properly.

See also this issue on CodePlex: https://nuget.codeplex.com/workitem/937

[jozefizso]

+1 :+1:

[jedugas]

+1

[laktak]

:+1:

[304NotModified]

+1

[sbrickey]

+1

[304NotModified]

I think we need some specs for this. Questions that would be nice to answer:

Functional

• F1) Which markup language will we be supported (markdown?, GFM?)
• F2) Which nuspec parts are supported (summary, description, releaseNotes?)
• F3) Which markup elements are supported
• F4) Which markup elements aren't supported
• F5) What to do with HTML input? (remove, escape, don't allow?)
• F6) What about backwards-compatibility? (markup in current nuspec files)
• F7) What about displaying outside the nugetgallery? (e.g. Visual Studio plugin)

Technical

• T1) Which library for the markup language?
• T2) Which library for the html input? (remove/escape html)

[maartenba]

From the functionals list, I think 7. is the most important one to answer before building this as a feature. @yishaigalatzer can answer if we want the VS plugins to support formatting here or not.

[304NotModified]

I'm curious if the specs of the nuspec file should be changed for this.

Is this the only spec or is there a more official one?

[maartenba]

That's the most official one.

[304NotModified]

Thanks @maartenba. Then no spec change is needed (dependent of functional 2 of course), as there is nothing specified about the content of the elements. (no statements like plain text or non-allowed content). :+1:

[yishaigalatzer]

I agree 7 is the most important one. Security is another one.

The issue for nuspec changes should go to the home repo, and so do the vsix changes. This will mean releasing all the way back to older versions of vs, so this could become quite a large item.

Lets open one, and unfortunately this will have to wait until we resolve more critical client issues, as we are making some massive UI changes and don't have the capacity to test and update older clients

[thomaslevesque]

Just an idea: perhaps a nuspec could contain 2 elements for the description (and other fields):

• description
• markdownDescription

Older clients that don't know how to handle Markdown would just ignore markdownDescription

The drawback is that package authors would have to maintain the fields separately.

[yishaigalatzer]

That's a nice idea.

[xavierdecoster]

What if there was a markdownDescriptionUrl (or readmeUrl, or whatever we call it) instead of embedding the entire markdown description in the nuspec, allowing people for instance to point to their GitHub readme file or any other hosted markdown file?

Benefits:

• flexibility: mistakes in the markdown can be fixed in the hosted markdown file without having to modify the package metadata and all orchestration related to that
• lightweight: nuspec size would be easily duplicated when embedding the markdown (I know, it's just some markdown, but still, only consume bandwidth when you need it)
• perhaps even benefiting from online edit/preview functionality for the markdown file
• people get the updated markdown without having to update the package

[maartenba]

What if we'd just read the project URL from the NuSpec and load the README from there by convention?

[xavierdecoster]

@maartenba that thought also crossed my mind, but I don't think we can assume all project URL's are pointing to a source repository

[yishaigalatzer]

Richard has built this in his sample nuget site.

Sent from my Windows Phone

---

From: Xavier Decostermailto:[email protected]
Sent: ‎10/‎7/‎2015 7:30 AM
To: NuGet/NuGetGallerymailto:[email protected]
Cc: Yishai Galatzermailto:[email protected]
Subject: Re: [NuGetGallery] Support formatting in description (#2280)

@maartenbahttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fmaartenba&data=01%7c01%7cyigalatz%40microsoft.com%7cba7290cd133040326b9d08d2cf23ddcb%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=C0bmrCdzfqJYk8mLchDAk2LOfxP3Wjbeai3YWbIwHCo%3d that thought also crossed my mind, but I don't think we can assume all project URL's are pointing to a source repository

—
Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fNuGet%2fNuGetGallery%2fissues%2f2280%23issuecomment-146211460&data=01%7c01%7cyigalatz%40microsoft.com%7cba7290cd133040326b9d08d2cf23ddcb%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=2%2bOlYoAPgV8PVqsa0oFC6UEMfEfd4ak4WXDlfLbnkuY%3d.

[maartenba]

We have it on MyGet too, can whip up a prototype in the next days?

[csharpfritz]

Is there a solution to this that would not require an extra network request
from the gallery or the clients? Can we allow the inclusion of a readme.md
in the nupkg and allow it to be parsed in a future version of the VS
Extension?

I would vote against full HTML support with the dangers of XSS that could
be introduced. We have already heard complaints of browsers being opened
by powershell scripts delivered in nupkg files.

Jeff

On Wed, Oct 7, 2015 at 10:41 AM, Maarten Balliauw [email protected]
wrote:

We have it on MyGet too, can whip up a prototype in the next days?

—
Reply to this email directly or view it on GitHub
https://github.com/NuGet/NuGetGallery/issues/2280#issuecomment-146215916
.

[304NotModified]

The issue for nuspec changes should go to the home repo, and so do the vsix changes. This will mean releasing all the way back to older versions of vs, so this could become quite a large item.

I don't really understand this. What has this to do with older versions? Is this functional 6 (backwards-comp)?

What if we'd just read the project URL from the NuSpec and load the README from there by convention?

Not every package is public, so maybe this works for nuget.org, but it's a bit strange on the intranet.

Just an idea: perhaps a nuspec could contain 2 elements for the description (and other fields):

This will fix functional F6 and F7 (Visual Studio).

Another option is to use a markdown attribute, e.g. <description use-markdown=true, or <description markup="markdown">. The latter will only fix 6. The pro is that only one element needs to be maintained.

What if there was a markdownDescriptionUrl (or readmeUrl, or whatever we call it) instead of embedding the entire markdown description in the nuspec, allowing people for instance to point to their GitHub readme file or any other hosted markdown file?

Which functional will fix this? F6? IMO this is a nice solution, but really another feature request.

I would vote against full HTML support with the dangers of XSS that could
be introduced. We have already heard complaints of browsers being opened
by powershell scripts delivered in nupkg files.

Yes, I think we should not allow any html. But what should be done with the HTML? (removal, escaping, etc)?

---

I think that maybe F7 isn't a high prio thing. Right now you already see markdown markup in nuget packages. So that will change for the nuget gallery and in a later stage for the visual studio extension?

[csharpfritz]

Among the tenets of the NuGet package clients and service is the ability to
be 100% backwards and forwards compatible. This means that we can't make
breaking syntax changes to nuspec documents. We can add optional features
that will not be activated in older clients, but we can't change syntax in
a way that would break older clients downloading newer packages.

We need to operate as though every package is in the public. Otherwise,
you can do whatever you want with nuspec and nupkg files in your own
environment and they will be rejected by the public NuGet.org service and
mishandled by the public NuGet clients.

Could we avoid adding any markup to the nuspec document and detect if a "
description.md" or some other markdown document is present at the root of
the nupkg and present that content, failing over to the description node?

Jeff

On Wed, Oct 7, 2015 at 11:17 AM, Julian Verdurmen [email protected]
wrote:

The issue for nuspec changes should go to the home repo, and so do the
vsix changes. This will mean releasing all the way back to older versions
of vs, so this could become quite a large item.

I don't really understand this. What has this to do with older versions?
Is this functional 6?

What if we'd just read the project URL from the NuSpec and load the README
from there by convention?

Not every package is public, so maybe this works for nuget.org, but it's
a bit strange on the intranet.

Just an idea: perhaps a nuspec could contain 2 elements for the
description (and other fields):

This will fix functional 6 and 7.

Another option is to use a markdown attribute, e.g. use-markdown=true, or `. The latter will only fix 6. The pro is that only
one element needs to be maintained.

What if there was a markdownDescriptionUrl (or readmeUrl, or whatever we
call it) instead of embedding the entire markdown description in the
nuspec, allowing people for instance to point to their GitHub readme file
or any other hosted markdown file?

Which functional will fix this? 6? IMO this is a nice solution, but really
another feature request.

I would vote against full HTML support with the dangers of XSS that could
be introduced. We have already heard complaints of browsers being opened
by powershell scripts delivered in nupkg files.

Yes, I think we should not allow any html. But what should be done with
the HTML? (removal, escaping, etc)?

—
Reply to this email directly or view it on GitHub
https://github.com/NuGet/NuGetGallery/issues/2280#issuecomment-146227112
.

[304NotModified]

Among the tenets of the NuGet package clients and service is the ability to
be 100% backwards and forwards compatible. This means that we can't make
breaking syntax changes to nuspec documents. We can add optional features
that will not be activated in older clients, but we can't change syntax in
a way that would break older clients downloading newer packages.

If the nuspec file should be backward- and forward-compatible, you can't change it ever? Even adding an XML attribute / element isn't possible?

Could we avoid adding any markup to the nuspec document and detect if a "
description.md"

I like that idea :) But now sure how backward/forward-compatible this solution is.

[csharpfritz]

Re: compatibility - The clients don't throw errors on content that they
don't understand, however you cannot change the meaning of any content item
that they currently use.

Jeff

On Wed, Oct 7, 2015 at 11:30 AM, Julian Verdurmen [email protected]
wrote:

Among the tenets of the NuGet package clients and service is the ability to
be 100% backwards and forwards compatible. This means that we can't make
breaking syntax changes to nuspec documents. We can add optional features
that will not be activated in older clients, but we can't change syntax in
a way that would break older clients downloading newer packages.

If the nuspec file should be backward- and forward-compatible, you can't
change it ever? Even adding an XML attribute / element isn't possible?

Could we avoid adding any markup to the nuspec document and detect if a "
description.md"

I like that idea :) But now sure how backward/forward-compatible this
solution is.

—
Reply to this email directly or view it on GitHub
https://github.com/NuGet/NuGetGallery/issues/2280#issuecomment-146231835
.

[304NotModified]

In summary, we have the following (proposed) solutions:

• S1) Just use markdown the nuspec
• S2) Create new elements in the nuspec
• S3) Add an attribute to the current elements in the nuspec (e.g. markdown=markup)
• S4) Add an URL to the nuspec and fetch a markdown file
• S5) Add a markdown file to the nuspec package.

[304NotModified]

Note:

MarkdownSharp is probably what we would prefer for now, as it's what MarkdownMailer used.

From: https://github.com/NuGet/NuGetGallery/pull/2243#issuecomment-48219265

For T1

[jedugas]

Well, it's ~6 months later so this can't be THAT hot a topic. B^) As usual, the simplest and most secure implementation is the non-implementation; package authors that want full control over the description formatting should provide a _{url to the package's readme.md}_ in the Description element's text that can be copied/pasted to a browser. Send the consumer to the github repo's readme.md, or a blog post, or wherever.

If a change _is_ made, the best and safest approach feels like adding an optional xml attribute (e.g. markdown=markup) to the current elements in the nuspec. Provided this won't break schema validation, the pros of allowing markdown in an element's text outweigh the cons - consumers will still get mostly-human-readable content regardless of the display.

cheers!

[yishaigalatzer]

Honestly, we would like to spend effort on revamping the front end completely, rather than do small features like this one by one. Which is why this feature is on the backlog and low priority.

[pierrejochem]

markdown support would be nice!

https://guides.github.com/features/mastering-markdown/

[304NotModified]

Indeed but the discussion has been killed IMO. And that's bad.

[skofman1]

@karann-msft , is this something planned for the new site?

[karann-msft]

for all nuget gallery related issues please tag @jonwchu

[jgauffin]

ping!

I don't follow the argument regarding backwards compatibility. If you add support for markdown in the description field why do it matter if old/other clients do not support it?

Isn't it up to the package owner to decide if that's something the owner can accept or not? If one can't, one will not use markdown.

This issue is now over two years old. Isn't it time to implement it? Shouldn't be that hard to run MarkdownSharp or similar library before displaying the description field.

[304NotModified]

Indeed, if someone just decide which solution is OK, then it should be really easy to implement.

[rjamesnw]

This would be nice to see put into place.

[anangaur]

Supporting markdown requires us to reformat our website a bit so that important information is not lost due to a big markdown description. We plan to do the minimal reformatting required so that we can start addressing features like this one to make the package description and related information richer. Stay tuned.

[rjamesnw]

So, another 2 years at least? I won't hold my breath, lol. ;p

[jgauffin]

@anangaur you have to elaborate.

The PackagesController contains an action called DisplayPackage right?

It's in that method that you can use markdown to process Package.Description:

//new field in the view model
model.HtmlDescription = myMarkdownEngine.Parse(package.Description);

and then in the view update the code:

@if (Model.Package.UseMarkdown)
{
    Html.Raw(model.HtmlDescription);
}
else
{
    foreach (var line in Model.Description.ToStringSafe().Split('\n'))
    {
        <p>@line</p>
    }
}

The UseMarkdown property is a new in the Package entity.
By putting the property in the Package entity it allows authors to move to markdown descriptions one package at a time.

Regarding the new column in the database: simply set it to false per default when updating the database schema.

I don't understand why that would affect everything else in the site?

[erick-thompson]

Is this item still active? Having some support for markdown (or other formatting) in description and release notes would make them much more useful. Markdown is generally human readable, so it seems like a good choice.

[skofman1]

@erick-thompson , we are currently working on this feature.
You can find the spec here: https://github.com/NuGet/home/wiki/Package-README.md-support
And track our progress here: https://github.com/NuGet/NuGetGallery/issues/4251 (if you use ZenHub you will be able to see all the sub-tasks)

[ederbond]

@skofman1 Allow me to tell you that the spec is getting on the wrong way:
What we developers need is nothing related with any code repository type (GitHub, Bitbucket, or TFVC), we need something repository agnostic in order to don't left anybody behind independent of the repository technology you are using.

For the good sake, WE JUST NEED to use markdown inside nuspec files ('description, releaseNotes an summary') and GET IT RENDERED properly on Visual Studio and Nuget Gallery. THIS IS THE MVP;
It will guarantee that no one will be left behind because of his code repository of choice.

Specific repository features (GitHub, Bitbucket, or TFVC) must be added later.

[anangaur]

@ederbond We need to support older clients that do not understand markdown and we cannot go back in time to fix this for those clients. An alternative solution that we think works is to support "Documentation" for packages that can take markdown text. The solution has following parts:

1. Ability to associate a markdown documentation to packages (and existing packages)
2. Understand markdown and render it properly on NuGet.org package details page
3. Ability to associate markdown documentation content while packing the package from clients
4. Ability on VS client to show this markdown content.

While we are done with 1 and 2 as part of 2017 Interns project, we are yet to get started on 3 and 4. We did not want to wait for all 4 parts to complete before rolling out the feature. We have released the work we did so far for people who want to try it out (the current experience may not be an ideal one) and provide us feedback to improve the experience further.

We would love to hear further comments and feedback on the feature and accordingly we will proceed for 3 and 4.

[infin8x]

Definitely excited to see this - we've wanted have READMEs for NuGet packages for a while. Excited to see the phase that brings a README.md directly inside the package.

[skofman1]

Update on this topic:
We are starting work on allowing to upload markdown documentation as part of the package: https://github.com/NuGet/Home/wiki/Packaging-Documentation-within-the-nupkg

The discussion issue for this feature is here: https://github.com/NuGet/Home/issues/6873
Closing this one as duplicate.