Marked: Overriding tokenizer bug??

[kim-tom]

Thank you for your wonderful library.

I wanna override default codespan tokenizer to include LaTeX same as official documents.
Strictly, I wanna change codespan regexp from /^(`+)([^`]|[^`][\s\S]*?[^`])\1(?!`)/ to /^([`$]+)([^`$]|[^`$][\s\S]*\?[^`$])\1(?![`$])/. I did that like below.

// Override function
const tokenizer = {
  codespan(src) {
    const match = src.match(/^([`$]+)([^`$]|[^`$][\s\S]*?[^`$])\1(?![`$])/);
    if (match) {
      return {
        type: 'codespan',
        raw: match[0],
        text: match[2].trim()
      };
    }
    // return nothing not to use original codespan tokenizer
  }
};
marked.use({ tokenizer });

This works well for ` but doesn't work when $ is not at the beginning of the line.
Like this,

> marked('a`b`')
> <p>a<code>b</code></p> 
OK!

> marked('$ab$')
> <p><code>ab</code></p>
OK!

> marked('a$b$')
> <p>a$b$</p>
Not working!

Am I wrong somewhere? Or is this a bug??

For your information I checked the regexp works correctly in the match function.

> '$c*d$'.match(/^([`$]+)([^`$]|[^`$][\s\S]*?[^`$])\1(?![`$])/)
> ["$c*d$", "$", "c*d"] 

> '`c*d`'.match(/^([`$]+)([^`$]|[^`$][\s\S]*?[^`$])\1(?![`$])/)
> ["`c*d`", "`", "c*d"]

[UziTech]

You probably have to override the inlineText to stop when it sees a $.

Right now inlineText matches a$b$ but should only be matching a and leaving $b$ for the codespan to check

[kim-tom]

Thank you very much for your quick response.
I overrode inlineText token. This works fine.
Is this what you mean?? To tell you the truth, I don' t understand how this works...

inlineText(src, inRawBlock, smartypants) {
  const cap = src.match(/^([`$]+|[^`$])(?:[\s\S]*?(?:(?=[\\<!\[`$*]|\b_|$)|[^ ](?= {2,}\n))|(?= {2,}\n))/);
    if (cap) {
      var text;
      if (inRawBlock) {
        text = this.options.sanitize ? this.options.sanitizer ? this.options.sanitizer(cap[0]) : cap[0] : cap[0];
      } else {
        text = (this.options.smartypants ? smartypants(cap[0]) : cap[0]);
      }
      return {
        type: 'text',
        raw: cap[0],
        text: text
      };
   }
}

[UziTech]

Ya if it works for you it looks good 👍

[kim-tom]

I cannot say thank you enough.
THANK YOU!!