Wp-calypso: Domains: Able to reach checkout with Punycode version of IDN

[SiobhyB]

When attempting to register a domain, siobhánbamber.com, via https://wordpress.com/domains/add, I'm blocked by an error notice informing me that the domain isn't valid. This is expected as WordPress.com doesn't support the registration or mapping of IDNs (Internationalized Domain Names).

However, when attempting to register xn--siobhn-ttabamber.com (which is the IDNA encoded or Punycode version) of siobhánbamber.com then I was able to reach checkout.

The registration technically failed as xn--siobhn-ttabamber.com isn't supported at WordPress.com, however, no notice of the failure was given throughout the registration process.

[umurkontaci]

/cc: @aidvu since this better be done in the backend

[umurkontaci]

@SiobhyB Turns out there's already a check for this type for user accounts.

[mahangu]

@Just reopening this ticket because I am able to reproduce this error now from Tortuga:

1) Go to https://wordpress.com/domains/

2) Search for any punycode domain like xn--8zc3ctb0d.com

3) Select it and proceed to checkout:

4) Checkout, and observe how it throws this error:

Ideally, we should prevent the user from ever being able to add the punycode domain to their cart, like we do from /domains/add/.

Please let me know if I should file this on the Tortuga GH repo instead. Thank you!

[umurkontaci]

@mahangu can you confirm with a regular user account?

[mahangu]

@umurkontaci yes, I am able to reproduce it on a regular account, unproxied.

[klimeryk]

Seems like there are two aspects of this - if you search for a direct match, like xn--8zc3ctb.com, you'll be able to add it to cart. But the actual domain should be මහඟ.com, which you can't add to cart, because of the availability check that runs after you try to add it to cart.
We should also prevent Domainsbot from suggesting IDN domains, because right now it does :/

[klimeryk]

This is not yet fixed - the patch helped with hiding this issue for Tortuga users, but if someone searches for a domain in punycode format it will still allow it to register it.

[stale[bot]]

This issue has been marked as stale and will be closed in seven days. This happened because:

• It has been inactive in the past 9 months.
• It isn't a project or a milestone, and hasn’t been labeled `[Pri] Blocker`, `[Pri] High`, `[Status] Keep Open`, or `OSS Citizen`.

You can keep the issue open by adding a comment. If you do, please provide additional context and explain why you’d like it to remain open. You can also close the issue yourself — if you do, please add a brief explanation.

[klimeryk]

cc @Automattic/i18n - I was wondering what's your take on internationalized domain names (aka IDN)? My guess would be that they are popular in some countries (with a non-latin alphabet perhaps), but would love to hear your expertise, so that we can better prioritize this issue.

[naokomc]

Japanese domains are not super common but not too rare either.

A Japanese page on Wikipedia says 120K out of 900K .JP domains are Japanese IDN, but I don't feel I see them very often. Maybe people register them but use alphabet version as the primary one.

FYI - according to https://jprs.jp/faq/use/ , ドメイン名例.JP and XN--ECKWD4C7CU47R2WF.JP (Punycode version) can be used as test domains, similar to example.com.

[deBhal]

I don't think we've done any real investigation.

Tucows says they support IDNs in some languages for some tlds, which sounds "pretty fun", but we're in a good place to handle that sort of thing now (I believe that that "some languages" is enforcing iana specified character subsets).

The big question mark is the places where we've made broken assumptions. For example, I saw a thread where a publicize link was broken in facebook because of the encoding, and videopress has it's own is_valid_domain() that would reject IDNs whether or not they're encoded.

We don't even support mapping IDNs right now, but it looks like there are some tricks kicking around to make it work sometimes, so it's not all bad :)

There are also some security questions around using visually similar characters to nefarious ends.

We need some idea of the value here to compare the effort to. How valuable are IDNs to users?

[klimeryk]

Thanks for chiming in, @naokomc and @deBhal 🙇 Maybe let's start then with just supporting mapping such domains (it _is_ the prerequisite for support registering such domains anyway) and see how popular those are. Meaning, if someone searches for an IDN, let's direct them to map the domain (if it's taken) or show them the usual green nudge to buy it if a non-IDN version is available (if it's possible to easily convert to such - easy for stuff like żłóć, but Japanese alphabet not so much ;)). Sounds good? Any volunteers? 😁

[aidvu]

We would have to store them in the non-IDN representation in the DB, because collation/charset, so some mumbo-jumbo would be required encoding/decoding them every time.

Would be nice to have it, yeah, but needs a bit more research on P2. :)

[klimeryk]

Right - probably in the Punycode format. And that seems the biggest technical hurdle here - it will introduce a different "raw" domain and a different one for display purposes.

[stale[bot]]

This issue has been marked as stale and will be closed in seven days. This happened because:

• It has been inactive in the past 9 months.
• It isn't a project or a milestone, and hasn’t been labeled `[Pri] Blocker`, `[Pri] High`, `[Status] Keep Open`, or `OSS Citizen`.

You can keep the issue open by adding a comment. If you do, please provide additional context and explain why you’d like it to remain open. You can also close the issue yourself — if you do, please add a brief explanation.