Tiptap: Markdown styles are applied on a Mention query

[andreasvirkus]

Describe the bug
If you use the Mention class and want to search for a value containing some Markdown-compatible symbols like _, then the Markdown styles are applied to the search query, therefore not matching the suggestions list

Steps to Reproduce / Codesandbox Example
Steps to reproduce the behavior:

1. Open up the Suggestions example - https://tiptap.scrumpy.io/suggestions
2. Type in @Phi_lip_
3. See that results in Phi _lip_
4. Suggestion Philipp Kühn is matched, even though the search query is phi_lip_

If the suggestion list would contain some tags (e.g. some_tag_value), then the user would not see some_tag_value in the matched list.

Expected behavior
Markdown styles wouldn't be applied in that scenario by default. Or some guidance on how to work around this 🙏

Screenshots

[andreasvirkus]

The way we are currently working around this is by overriding the inputRules and pasteRules for Italic, Bold, etc extensions, e.g.

import { markInputRule, markPasteRule } from 'tiptap-commands'
import { Editor, Extension } from 'tiptap'
import { Italic } from 'tiptap-extensions

const ModifiedItalic = class ModifiedItalic extends Italic {
  inputRules({ type }) {
    return [markInputRule(/(?:^| )[^@#\s_]*(_([^_]+)_)/g, type), markInputRule(/(?:^| )[^@#\s*]*(\*([^*]+)\*)/g, type)]
  }
  pasteRules({ type }) {
    return [markPasteRule(/(?:^| )[^@#\s_]*(_([^_]+)_)/g, type), markPasteRule(/(?:^| )[^@#\s*]*(\*([^*]+)\*)/g, type)]
  }
}

new Editor({
  extensions: [ModifiedItalics]
})

[mahentoo]

The same is true if you have a link like https://en.wikipedia.org/wiki/The_Machine_(computer_architecture)

[BrianHung]

@mahentoo If you're pasting a link, I believe you can fix that by re-ordering new Link() to be included before new Italic().

[BrianHung]

I haven't looked too deeply into Suggestions.js yet, but it may be possible to wrap the active mention text in a mark or an inline node that disallows marks and mark input rules. For example, codeblocks can disable having their schema include code: true, marks: "".

[mahentoo]

@mahentoo If you're pasting a link, I believe you can fix that by re-ordering new Link() to be included before new Italic().

@BrianHung
It works for other links, I think with this link: https://en.wikipedia.org/wiki/The_Machine_(computer_architecture) the problem may be because of the parentheses.

[BrianHung]

@mahentoo
What's your editor configuration, i.e. the order by which you instantiate marks?

@andreasvirkus
Playing around with the tiptap website, I found that bold and italic inputrules won't trigger in a code mark. So replacing the decoration with a mark in Suggestions.js is a possible, easy route.

[mahentoo]

@mahentoo
What's your editor configuration, i.e. the order by which you instantiate marks?

@BrianHung
```
extensions: [
new Blockquote(),
new BulletList(),
new HardBreak(),
new Heading({ levels: [1, 2, 3] }),
new Mention({ /* code */ }),
new ListItem(),
new OrderedList(),
new TodoItem(),
new TodoList(),
new Link(),
new Bold(),
new Code(),
new Italic(),
new Strike(),
new Underline(),
new History(),
new Placeholder({
emptyClass: 'is-empty',
emptyNodeText: 'O que foi feito e qual o próximo passo?',
}),
],

[BrianHung]

@mahentoo You're pasting the link right, and the issue is the Machine gets italicize? I'm trying to reproduce that using the tiptap website: https://tiptap.scrumpy.io/markdown-shortcuts/.

[andreasvirkus]

@BrianHung thanks for looking into it. Seems i'd have to keep a clone of the whole Suggestions.js plugin in my own repo, since there'd be no easy way to extend it, right? And would this line range be the only part I'd need to change? (#232 - #244)
https://github.com/scrumpy/tiptap/blob/7a56da631516f8e620893a63b4d34857942ea44d/packages/tiptap-extensions/src/plugins/Suggestions.js#L232-L244

[mahentoo]

@mahentoo You're pasting the link right, and the issue is the Machine gets italicize? I'm trying to reproduce that using the tiptap website: https://tiptap.scrumpy.io/markdown-shortcuts/.

@BrianHung Yes, just like on the tiptap website.

[andreasvirkus]

Back in April, we solved this with extending the italic and bold marks. The code for those custom marks has remained the same, but this issue has resurfaced - a mention query like #awe_so_me gets matched with a mention item #awesome

Here are my extended classes, just in case. Is there anything from the Marks or Mentions side on tiptap that could cause a regression, @hanspagel ?

const italicsAsterisk = /(?:^|\s)[^\s*\w]*(\*([^*]+)\*)/g
const italicsUnderscore = /(?:^|\s)[^\s_\w]*(_([^_]+)_)/g

export const CustomItalic = class CustomItalic extends Italic {
  inputRules({ type }) {
    return [markInputRule(italicsAsterisk, type), markInputRule(italicsUnderscore, type)]
  }
  pasteRules({ type }) {
    return [markPasteRule(italicsAsterisk, type), markPasteRule(italicsUnderscore, type)]
  }
}

const boldAsterisk = /(?:[*]{2})([^*]+)(?:[*]{2})/g
const boldUnderscore = /(?:[_]{2})([^_]+)(?:[_]{2})/g

export const CustomBold = class CustomBold extends Bold {
  inputRules({ type }) {
    return [markInputRule(boldAsterisk, type), markInputRule(boldUnderscore, type)]
  }
  pasteRules({ type }) {
    return [markPasteRule(boldAsterisk, type), markPasteRule(boldUnderscore, type)]
  }
}

---

I also couldn't quite figure out how to replace the Suggestions.js plugin's decoration with a mark, as @BrianHung suggested back then - https://github.com/ueberdosis/tiptap/issues/632#issuecomment-607442766

[BrianHung]

The way to implement marks within a plugin could be to use the appendTransaction prop. The idea is as long as a suggestion is active, we'll continually move the bounds of the mark. The oldState and newState refer to the state we keep in the suggestions plugin.

    appendTransaction(transactions, oldState, newState) {
      // Do nothing if document has not changed or no modified textblocks to apply mark to.
      const docChanged = transactions.reduce((docChanged, tr) => docChanged || tr.docChanged, false)
      if (docChanged === false) return

      // Check if oldState and newState differ and maybe return

      // Replace old marks at modified line with new marks.
      let tr = newState.tr
      tr = tr.removeMark(oldState.range.start, oldState.range.end, newState.schema.marks.suggestions))
      tr = tr.addMark(newState.range.start, newState.range.end, newState.schema.marks.suggestions.create()))
      return tr
    }

Edit:
@andreasvirkus

I was looking at remirror, a react library that uses prosemirror, and they have implemented a suggestion marks plugin for their social editor: https://remirror.io/docs/showcase/social. Specifically I would look at how the auto-linking is achieved: https://github.com/remirror/remirror/blob/next/packages/%40remirror/extension-auto-link/src/auto-link-extension.ts