Markbind: Provide a way to escape nunjucks syntax

[damithc]

Current: any nunjucks-like syntax e.g., {{ x }} in MarkBind files will be processed by nunjucks, even if it is in a blockquote code block.

Problem: Others will not be able to use MarkBind for documenting projects that use such syntax.

Suggested: Provide a way to escape nunjucks processing for source segments so that nunjucks-like syntax can be shown in the generated pages.

[acjh]

even if it is in a blockquote.

code block?

[damithc]

_code block_?

Yup. Corrected :-p

[Xenonym]

The jinja2 docs (of which Nunjucks is a port) already has methods for escaping syntax, by wrapping braces in a variable ({{ '{{' }}/{{ '}}' }}) or using {% raw %}/{% endraw %} to mark a block to not be interpreted as Nunjucks.

However, these methods don't work in MarkBind because we run source files through Nunjucks multiple times, and any escaping present in one round of Nunjucks rendering gets interpreted as syntax in the next round.

I wonder if it would be possible to retain this syntax (eg. by only rendering Nunjucks once) because it would be more user-friendly to have MarkBind use existing syntax for escaping Nunjucks instead of inventing new syntax.

[yamgent]

I wonder if it would be possible to retain this syntax (eg. by only rendering Nunjucks once)

The issue for just rendering once, is that some things need to be called multiple times before it can render correctly. For example, variables referencing other variables:

---

variable.md

<variable name="text">
Some text here.
</variable>

<variable name="example">
{{ '{{' }} example {{ '}}' }}: {{ text }}
</variable>

<variable name="howToUseVariables">
Specify the variable name with double curly-brace. For example: {{ example }}.
</variable>

index.md

{{ howToUseVariables }}

What index.md currently renders:

Specify the variable name with double curly-brace. For example: : Some text here.

What index.md may possibly render if we only do rendering once:

Specify the variable name with double curly-brace. For example: {{ '{{' }} example {{ '}}' }}: {{ text }}

The ideal output for index.md:

Specify the variable name with double curly-brace. For example: {{ example }}: Some text here.

[luyangkenneth]

by wrapping braces in a variable {{ '{{' }} / {{ '}}' }} or using {% raw %} / {% endraw %} to mark a block to not be interpreted as Nunjucks.

This feels like the ideal experience from an author's standpoint, so I would love for MarkBind to support this syntax. When I encountered this issue at the beginning of the sem, this syntax was also the first thing I tried to use, because it's consistent with my prior experience with other templating engines.

However, these methods don't work in MarkBind because we run source files through Nunjucks multiple times

The issue for just rendering once, is that some things need to be called multiple times before it can render correctly.

I wonder if we are running the entirety of source files through Nunjucks multiple times. If so, then it's not surprising that "any escaping present in one round of Nunjucks rendering gets interpreted as syntax in the next round".

If we resolved variables and includes before injecting them into the current file, then it seems reasonable that we can make this work properly. Is there anything I'm not aware of that might prevent us from doing this?

[luyangkenneth]

Actually, I think I would go as far as categorizing this as a bug -- as an author, there are certain strings that are impossible to display in my website.

[yamgent]

Is there anything I'm not aware of that might prevent us from doing this?

Not that I know of, I will most likely be worried about the includes, but if those worked out to be fine, then having a single render would be a great improvement.

[crphang]

While already a great improvement, a syntax like {% raw %} might not scale well if we start introducing new components that uses syntax which require the use of curly braces (for example latex).

I was wondering for this problem whether it would be better to have the user experience as such:

1. By default all nunjucks are escaped and displayed raw with the curly braces {{ x }}
2. If a user define a variable, then it will be rendered.

What do you think @damithc @yamgent?

[damithc]

I was wondering for this problem whether it would be better to have the user experience as such:

1. By default all nunjucks are escaped and displayed raw with the curly braces {{ x }}
2. If a user define a variable, then it will be rendered.

What do you think @damithc @yamgent?

I'm not sure about the implementation aspect. Note that it's not just {{ }} but also {% %}.

W.r.t. multiple rendering, may be use the approach of replacing raw segments with our own custom string before the rendering and put them back after all the renderings are done? But that adds two more passes :-p We can require such files to be listed in site.json so that the two passes can be done faster, or even skip completely if there are no such files.

[crphang]

I attempted to achieve escaping by each file to crawl for {{ content }} to produce as map of escapedVariables:

const escapedVariables = {
  escapedContent: `{{${escapedContent}}}`,
}

const realVariables = { ... }

const variables = {
  ...escapedVariables,
  ...realVariables, //realVariables will override
}

This set of escaped "variables" are added as render throughout all nunjuck calls:

nunjucks.renderString(pageData, variables);

This allowed me to generate the file, where the mustache is rendered:

index.html
<body>
  <div id="app">
    <div id="flex-body">
      <div id="content-wrapper">
        <p>{{escapedVariable}}</p>
      </div>
    </div>
    <footer>

      <div class="text-center">
        <small>[Generated by <a href="https://markbind.org/">MarkBind 2.10.0</a>]</small></div>
    </footer>
  </div>
</body>
<script src="markbind/js/vue.min.js"></script>

However, this will lead to an error from Vue because Vue is also attempting to perform its own text interpolation.:

Some of the explored solutions are discussed here for Vue:

https://github.com/vuejs/vue/issues/4223
https://github.com/vuejs/vue/issues/147

The escaping would work after disabling:

Vue.config({ interpolate: false });

Do you foresee disabling interpolation for Vue to be an issue @damithc @yamgent? I think we should be fine as we are relying mainly on Nunjucks for this behaviour throughout Markbind.

[damithc]

Do you foresee disabling interpolation for Vue to be an issue @damithc @yamgent? I think we should be fine as we are relying mainly on Nunjucks for this behaviour throughout Markbind.

Seems OK to me but others might be in a better position to comment. Guess we don't need to support vue templating as we already support nunjucks?

[yamgent]

The escaping would work after disabling:

Vue.config({ interpolate: false });

Does this affect our ability to use interpolation in vue-strap itself?

[crphang]

Yes. It does. Are we currently using it anywhere? Alternatively, we can change the delimiters of Vue to something else other than {{ }} to be more future proof as well.

[yamgent]

Are we currently using it anywhere? Alternatively, we can change the delimiters of Vue to something else other than {{ }} to be more future proof as well.

We do, here's an example:

https://github.com/MarkBind/vue-strap/blob/695ee3856d89e0e12213410f01f94f535306a3ce/src/TipBox.vue#L9-L11

I am not sure whether it is wise to change the behaviour of Vue since this is quite a fundamental syntax that developers will expect it to work. Could we introduce a special syntax for authors to escape the braces instead?

[crphang]

I see.

Even if we use a special syntax to escape braces, for example {% raw %}, this would still mean finally rendering {{ }}} in the output, which would clash with Vue. If we display it using something like: {<span></span>{ }} but this wouldnt work for codeblocks.

Essentially, we actually have been relying on behaviour of nunjucks to prune all curly braces before reaching Vue. I think in this case we could separate the problem of

(1) Escaping nunjucks syntax from within markbind and
(2) Displaying the curly braces.

For (1) there is still value as it enables us to escape nunjucks syntax from future components like #984, and latex potentially. These components will require the curly braces to stay pristine, but will parse them themselves.

This can be done separately from (2), which requires also configuration on Vue.

[yamgent]

For (1) there is still value as it enables us to escape nunjucks syntax from future components like #984, and latex potentially. These components will require the curly braces to stay pristine, but will parse them themselves.

This can be done separately from (2), which requires also configuration on Vue.

OK, I am open to allowing changes on the Vue side to use a different delimiter then, since Vue runs on client-side, which makes it pretty much unavoidable to make the displaying of curly brace done during site generation, but must be done live.

[damithc]

Raising priority as the problem is also affecting other areas #1196