gutenberg remove non english word in slug (save and draft)

[ShahramRahbari]

You can see the bug in this video when the draft is saved.

https://youtu.be/vvllwawBljs

When we want save and draft a title include both english keyword next to non english keyword in gutenberg plugin, automatically the permalink register only english part and it will delete non english part of the title

Regards Shahram Rahbari

[aristath]

From what I can tell this seems to be an issue with the url package.
We use lodash's deburr function to convert URL strings. The issue with that is that as mentioned in the deburr docs

Deburrs string by converting Latin-1 Supplement and Latin Extended-A letters to basic Latin letters and removing combining diacritical marks.

Perhaps we could implement a transliteration function here, or some other cleanup method if non-latin script is used.
Languages like Greek, Russian etc would benefit greatly from transliteration, but I don't know about other languages. Definitely something that would need to be discussed before any fix gets implemented as there's lot of languages to consider and the implications of a change could be far from simple.

[aristath]

Doing a quick search I found a package that seems to be able to do everything we need (and it's MIT-licensed so we're OK): https://www.npmjs.com/package/transliteration
The only problem with that is that it appears to be slightly big...

[ShahramRahbari]

I tested on WordPress which has no plugins and there is this problem. This problem is solved with the plugin

https://wordpress.org/plugins/enable-jquery-migrate-helper/

but maybe some people will not find this plugin?

[nosolosw]

Hi, I've tested this is the same error reported in the forums here.

In WordPress 5.4, both the block editor & the classic editor admit unicode characters in the slug (I've tried with Chinese and emoji in the title). However, in WordPress 5.5, it only works in the classic editor.

Note that this only happens when a draft is saved, once the post is published the slug includes all characters.

[james58899]

PR https://github.com/WordPress/gutenberg/pull/21007 remove ALL non english characters.
I do not think should prevent user using non english characters.