Dokuwiki: Possibly a bug: Cyrillic links in the table break the parser

[kin63camapa]

attempt to create a table eg

|any|
|https://ссылка.рф/any|
|any|

leads to the fact that further content on the page is not displayed

2018-04-22b "Greebo"
PHP 7.0.33
Apache / 2.4.25 (Debian)

[phy25]

Reproducible on https://www.dokuwiki.org/playground:issue2869

[phy25]

Your link is an IDN, which may need to be encoded using Punycode:

https://ссылка.рф/any -> https://xn--80atcxa4d.xn--p1ai/any

I think DokuWiki's parser is thinking you have // as italic symbol instead of a URL, which causes the issue, so you need:

|any|
|[[https://ссылка.рф/any]]|
|any|

I believe all IDNs are not working in this case e.g. http://谷歌.cn/ - not sure if we need more robust URL matching (since GitHub is parsing these correctly and we have ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? in https://www.ietf.org/rfc/rfc3986.txt)

https://github.com/splitbrain/dokuwiki/blob/af3a352caac697add659c226ca7a67cc0c689ddd/inc/parser/parser.php#L947-L956

Reference: http://urlregex.com/

[kin63camapa]

I noticed that the status of "bug" was removed from this issue.

Do you really think that placing a text like "xn--80atcxa4d.xn--p1ai" in human-readable text is a good idea?
I think the task of the wiki is to reflect reality, and not to force the author to look for the technical equivalent of the information he wants to share.

Of course, I used double square brackets to fix my page before I created this issue, but the text on the wiki can be added not only from the editor.

this led me to look for a problem in my text generator, later I thought that not all the text was uploaded to the web server.

I think this thing cannot be left as is

Thank you for reading this to the end.
Sorry my bad english.

[phy25]

I just realized I didn't put comments here...

Do you really think that placing a text like "xn--80atcxa4d.xn--p1ai" in human-readable text is a good idea?

I think the task of the wiki is to reflect reality, and not to force the author to look for the technical equivalent of the information he wants to share.

What you are talking about is IRI (https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier) while DokuWiki only supports URL at this point (which requires ASCII letters). This is not a bug because it currently simply doesn't support it, rather than a regression.

As a non English-native speaker, I understand that from the user perspective the two look similar and the lack of support of the advanced one is frustrating, but the matching becomes more tricky there, so someone has to put efforts into the related code change and verification. I would say pull request is welcome for anyone who is willing to bring IRI to DokuWiki :-).

[splitbrain]

Sorry I don't understand the original issue. The given syntax works just fine: https://www.dokuwiki.org/playground:punycode2

Edit: never mind. I didn't scroll far enough

Still, the only problem we have with international URLs is when they are not encapsulated in the link syntax. Our auto-recognizer only supports ASCII URLs.

[splitbrain]

A simple fix might be to replace the first $any in https://github.com/splitbrain/dokuwiki/blob/656cb03223898bc17f215be41d3e697d2b9fbed1/inc/Parsing/ParserMode/Externallink.php#L23 with .[\x80-\xbf]* to allow any unicode char. But we'd need a whole bunch of URLs to run tests against.

[KamarajuKusumanchi]

But we'd need a whole bunch of URLs to run tests against.

Here is another one - https://te.wikipedia.org/wiki/పైడిమర్రి_వెంకటసుబ్బారావు . It has ASCII and Telugu language characters. Would be great if you could include this in your testing.