Rocket.chat: Text conversion to Emoji broken?

[Dirk23]

Description:

Text like ; ) is not converted to a Emoji until there is a space in front and behind it. That was not the normal behavior for long time.

Server Setup Information:

• Version of Rocket.Chat Server: 0.60.4
• Operating System: Ubuntu 16.04
• Deployment Method(snap/docker/tar/etc): snap
• Number of Running Instances: 1
• DB Replicaset Oplog: ?
• Node Version: ?

[Otiel]

I'm also having this issue with Rocket.Chat version 0.61.0.

~I think this is the same issue as #9351.~

[Dirk23]

Emojis are not correctly replaced in ALL messages, even when the only thing in the line is ;). Like i mentioned above: the only way to have them replaced is to put a blank space in front and after ;)

[Otiel]

I read #9351 too fast, you're right it's not exactly the same case.

Did you try upgrading to 0.61.2 (I didn't yet)? It looks like it could be fixed by #9776.

[Dirk23]

my snaps didnt update, i'm on 0.61.0

[karlprieb]

Hey @Dirk23 I can't reproduce this behavior on 0.62, can you upgrade your RC version and try again?

Thank you!

[Dirk23]

I Updated my RC to 0.62.1 now and i still ":)" is not converted to a emoji. I need to put spaces before and after it like this " :) "

[Otiel]

I also still have the issue with 0.62.1.

[karlprieb]

Can you send me the exactly message that you're trying to send?

[Dirk23]

Yes it is just: :) wich is not converted, or any other doublepoint and bracket Emoji you know.
If you add spaces in front and after it, it converts to the Emoji. It doesnt matter if i type some text and the emoji with in it, or at the end or beginning. or if i only type :) without any other text.

[karlprieb]

Ok, but you really need to add the space after? I know that on the first line we have a bug that didn't render the emoji if you start with the emoji, so you need to add a space before, but after too?

[Dirk23]

yes, if i only add one space (in front or after) it is not converted

[tzar]

Same bug here - at least I know the space before/after workaround now! The behavior felt very random as to whether I'd get an emoji or not. I am using 0.62.2

[tzar]

It's probably this (old) issue in emojione - https://github.com/emojione/emojione/issues/54

Looking at the regex used it will not match <p>:)</p> or <p> :)</p> or <p>:) </p>, only <p> :) </p>, try it here: https://regex101.com/r/XHfSKj/1

[gdelavald]

From the issue @tzar posted above:

A solution for this has been published in the 3.0.2 release. The object property riskyMatchAscii can be set to true, allowing all ASCII chars to be replaced regardless of whether they're space-char adjacent. While this isn't a perfect solution since it'll still break strings like 'c://', it should be a good starting point. Hopefully with feedback we can continue to improve this.

I think the current way is less harmful than showing an emoji when I try to write, e.g.C:// or http://.

[Dirk23]

Why should https:// be converted to a emoji? There is no : at the end!

[gdelavald]

Almost all smiley faces are also translated into emojis (like :/ :) etc)

[karlprieb]

I'll close that issue. We are not "fix" that.

[Otiel]

I'm confused why this is closed and why you consider that this doesn't need to be fixed.

[gdelavald]

@Otiel Like we mentioned above, the big problem is trying to translate emojis like :/ and :) inside "words" like http://rocket.chat, that's why we enforce the spaces.

[Otiel]

Why not enforce the spaces unless:

• the emoticon like :/ or :) is followed by nothing, or,
• the emoticon like :/ or :) is followed by a line break, or,
• the emoticon like :/ or :) is preceded by nothing.

[gdelavald]

I think we could work with that, can you open a PR for this change? :)

[Otiel]

Sorry, I won't be able to create a PR for this.

(Just to clarify, I was not asking for a quick resolution on this, but simply challenging for the fact that this issue should not be closed)

[tzar]

The problem is it's not followed by nothing - by the time the emoji renderer is called, it looks like this: <p>:)</p>. You also can't just wrap it in spaces because there might be several elements generated by the markdown renderer all with the same problem, try this:

- :)
- :(

You probably also can't render emoji earlier, because it might mess up the markdown format. The only solution I can see would be to use an XML parser on the message and then run emoji conversion for every text node within the message, instead of just running the emoji conversion on the raw HTML of the message. I'm not familiar enough with the codebase to make this change, but here's a proof of concept which seems to work with this approach:

https://codepen.io/anon/pen/rvagvm

Hopefully it can be cleaned up and adapted to suit ☺ code:

const renderEachChild = (el) => {
  let nodes = el.childNodes, rendered = "";
  for(let idx in nodes) {
    node = nodes[idx];
    if(node.nodeType == 3) {
      // Text node, render
      rendered = rendered + emojione.toImage(node.nodeValue);
    } else if(node.nodeType == 1) {
      // Element, recur
      rendered = rendered + renderEachChild(node);
    }
  }
  el.innerHTML = rendered;
  return el.outerHTML;
}

const renderEmoji = (raw_html) => {
  let element = document.createElement("div");
  element.innerHTML = raw_html;
  return renderEachChild(element);
}