Wordpress-ios: Login: URL field does not recognize emoji domains

[sandymcfadden]

Expected behavior

• Adding a self hosted site which has an emoji in the domain should be a valid URL and allow you to login

Actual behavior

• When adding a domain with emoji the Add Site button will not highlight so it can be clicked.

Steps to reproduce the behavior

• Go to My Sites
• Add Self Hosted SIte
• Enter username and password
• Enter a domain with an emoji in it and try to add.
• Puny code for the domain does not validate either.

Reported in #147692-h
Ping: @astralbodies

Tested on iPhone 6, iOS 10.1.1, WPiOS 6.7

[astralbodies]

@koke It looks like the NSURL+IDN methods are returning nil when the emoji-based URL is being passed in. I can file an issue in that repo - looks like we should probably modernize that project and add some good unit testing.

[koke]

I'm tempted to just replace NSURL-IDN with a Swift wrapper for libidn but I'm really really not in the mood for C right now 😬

[koke]

The ideal solution would be to use ICU's uidna functions, which is what WebKit uses. Unfortunately the headers are missing from iOS, and this would mean bundling our own version of ICU. I've considered doing that in the past to work around some other i18n issues, so we might as well do it and start leveraging that wonderful library.

[kwonye]

Same with the Russian keyboard

[kwonye]

@aerych any ideas?

[aerych]

@kwonye I think what @koke suggests is the best thing for a permanent fix. For the mean time, a work around is to enter the punycode equivalent instead of the emoji. Maybe we can refer folks to https://www.punycoder.com/ or something similar.

[koke]

Radar created: http://www.openradar.me/29948279
This should really be baked into NSURL 😞

[aerych]

This should really be baked into NSURL

Seriously!

[astralbodies]

Maybe something simple like this?

let urlWithEmoji = "http://hello☢️.com"

var allowedCharacters = CharacterSet.urlPathAllowed.union(.urlHostAllowed)
let urlWithEmojiEscaped = urlWithEmoji.addingPercentEncoding(withAllowedCharacters: allowedCharacters)
let url = URL(string: urlWithEmojiEscaped!)

Results in: http://hello%E2%98%A2%EF%B8%8F.com

[astralbodies]

Ahha .. the percent-encoding problem with NSURLSession...

[kwonye]

@aerych this is fixed on our end, correct? Can we close this out?

[aerych]

this is fixed on our end, correct? Can we close this out?

Not really, and no. :) #6378 did not resolve the issue with (specifically) emoji, only with non-latin characters.

[koke]

I've spent some time trying to figure out how to implement IDNA properly and Punycode is not an easy algorithm to digest. The problem is that NSURL-IDN uses some code copied directly from OmniFoundation and I don't really understand the algorithm enough to fix it. This would require quite some effort to fix for a small impact.

I wasn't sure how to approach this because even if it affected only a small set of users, it felt wrong that they couldn't use the app because we badly implemented a standard. Until I kept reading and arrived to this: Why can't you register emojis in gTLDs:

Emoticon domains only work in software that has not implemented the latest IDNA standard. Only the older, deprecated version of the IDNA standard allowed emoticons, more or less by accident. Over time, these domains will increasingly not work correctly as software vendors update their implementations.

Looking at the relevant RFC confirms it, and IDNA2008 does not allow emoji on labels. This means I feel much more comfortable with a _wontfix_.

I have to say however, that there's something odd with our Punycode implementation, as ❤️❤️❤️.ws should encode to xn--qeiaa.ws, but it's encoding to xn--qeiaa2390tbab.ws. It makes me really nervous that we have some algorithm that's not behaving correctly and we don't understand what it does nearly enough to change it.

[aerych]

This means I feel much more comfortable with a wontfix.

Agreed. Let's close this one for the reasons @koke stated. If we ever have a need to revisit how puny code handles emoji we can do that in a separate issue.