Packages: [Markdown] A collection of issues

[aziz]

• [x] List punctuations #264
• [x] Underscore In the middle of words

The word `complicated` must be neither bold nor italic below:

perform_complicated_task
perform__complicated__task

But the first part below is italic and bold respectively:

_perform_complicated_task
__perform__complicated__task

• [x] Plain URLs

Normal plain link http://example.com
Complex url https://github.com/sublime/issues?page=1&state=closed

• [x] Links Refs with optional titles wrapped in single quotes

[3]: http://example.com/  'Optional Title Here'

• Escaped backticks inside inline code blocks (seems invalid @wbond)

`raw \` more`

• [x] Ordered lists and unordered lists inside a block quotes

> Here is a quote block
> - list items
> - inside block
> - quote
> 
> 1. numbered list 
> 2. numbered list another
>

• [x] Detecting empty lines inside a block quote with just > symbol

> Here is a quote block
This quote continues on.  Line breaking is OK in markdown
> Here it is again
>
> The `>` in line above should be marked as punctuation but it's not

• [x] Lists immediately after paragraph

this is a paragraph
- and this
- is a list

• [x] Blockquote and code blocks inside a list

* this is a list

   > This is a blockquote.

 This is a paragraph still part of the 
 list item

* Lorem ipsum

        This is a code block  
* list continues
* list continues

Extended _but common_ Markdown

• [x] Fenced Blocks

md ```js for (var i = 0; i < 10; i++) { console.log(i); } ```

Here is test file for fenced blocks

• [ ] Syntax highlighting inside fenced blocks
• [x] Tables
  Here is a test file for tables
• [ ] Strikethrough

~~This text should be parsed as _strikethroughed_.~~

~~There may be __bold__ or _italic_ text inside strikethroughed text.~~

~~There may be ![Mou icon](mo.com) a keyboard `complicated` shortcut like [GFM][GFM] inside strikethroughed text.~~

__There may be ~~strikethroughed text~~ inside bold text.__
_There may be ~~strikethroughed text~~ inside italic text._

~~ If there is a space in the beginning or end, it won't work as per the [GFM][GFM] docs ~~

[wbond]

Excellent, thanks for the examples!

[FichteFoll]

Can you point to a specification where backslashes inside backticks are supposed to escape the following backtick? Or is that meant the other way around?

Personally, I've been using the MarkdownEditing package with its GFM definition for the longest time.

[keith-hall]

@FichteFoll http://daringfireball.net/projects/markdown/syntax#backslash
although using multiple backticks is also supported: http://daringfireball.net/projects/markdown/syntax#code

[FichteFoll]

I'm rather sure this is supposed to escape the opening part (like this: ) and not to work _within_ code blocks, since those are literal. It doesn't work that way in any markdown implementation I tested (github GFM, discourse).
For including backticks in code blocks, more backticks are to be used.

[jonschlinkert]

Also, fenced code blocks are not highlighted when immediately following text without an extra newline:

With an extra newline:

[wbond]

I believe that technically lists right after paragraphs are not supported, and I'm not sure how we would handle that without incorrect marking up a paragraph containing a - or * at the beginning of a line incorrectly.

If there is info about this feature in terms of compatibility with various markdown engines, that could be useful.

[jonschlinkert]

I believe that technically lists right after paragraphs are not supported

I wasn't sure either, so I looked up the spec: http://spec.commonmark.org/0.24/#lists

Here is a quote a few paragraphs down:

In CommonMark, a list can interrupt a paragraph. That is, no blank line is needed to separate a paragraph from a following list:

[FichteFoll]

Markdown implementations are all different to this date, even after commonmark was published. Still, I believe that commonmark is the most specified flavor currently and basing on it sounds like a good idea, especially considering that more platforms are moving to it slowly.

[wbond]

@keith-hall just fixed a number of these in #803. I've checked off what I believe has been completed.

[keith-hall]

I believe "Underscore in the middle of words" can also be ticked off :)

[wbond]

And some more fixes by @keith-hall in #806!

The only thing I noticed was that:

* this is a list

   > This is a blockquote.

 This is a paragraph still part of the 
 list item

* Lorem ipsum

        This is a code block  
* list continues
* list continues

The last list item continues the paragraph scope from the second-to-last list item.

[keith-hall]

Plain URLs and tables have now been added to the Markdown syntax :)

[rwols]

Will syntax highlighting in code blocks ever happen? It seems very easy to add this feature at this point.

[FichteFoll]

There are two main issues with highlighting in code blocks.

1. It increases the size of the compiled syntax (and compilation time) by including many other syntaxes
2. It could only be done for syntaxes in default packages (which are still quite a few and most likely all that you'd ever need).

I'd be fine with 2 but I don't feel like being in a position to decide 1.

For reference, the latex syntax includes a lot of syntaxes and is by far the biggest compiled one, as far as I remember.

[wbond]

As @FichteFoll mentioned in point 1., the real issue is that the current implementation of including syntaxes in other syntaxes involves embedding them. LaTeX got to the point of the cached regex patterns taking up 9MB on disk (I believe there were around 8k contexts in the syntax), and users would see a lag the first time they opened a LaTeX file as the regex cache was loaded into memory to be used for highlighting.

We did some work on reducing the size of the C and C++ syntaxes, which cascaded to LaTeX, but I really need to see if I can get the syntax engine to no require embedding all contexts for things like a language including another. This applies to a lot of markup languages (HTML, Markdown, LaTeX) but also programming languages embedding other languages (such as SQL, HTML, JS, etc).