Aspnetcore: There should be specialized normalizers for E-Mails

[MovGP0]

Is your feature request related to a problem? Please describe.

Given I have an E-Mail address that includes unicode characters. When its normalized in the database using the UpperInvariantLookupNormalizer, those unicode characters are not properly escaped into Punycode using the System.Globalization.IdnMapping class.

When registering a custom normalizer, it will apply to email addresses and user names, even in use cases when the user name is not an email address.

Describe the solution you'd like

I propose to have specialized versions of ILookupNormalizer for e-mail addresses and user names:

public interface IUserNameLookupNormalizer : ILookupNormalizer {}
public interface IEmailLookupNormalizer : ILookupNormalizer {}

as well as appropriate default implementations that remove diacritic symbols, or escape punycode respectively.

Describe alternatives you've considered

The current workaround is to implement custom IUserEmailStore<TUser>, UserManager<User>, and RoleManager<Role> derived classes and overwrite the methods which use the normalized username or the normalized email address. One can that implement the interfaces above and provide the normalizers using Dependency Injection.

[MovGP0]

btw: the class Microsoft.AspNetCore.Identity.UpperInvariantLookupNormalizer should probably be sealed, since the implementation is trivial.

[blowdart]

Can I assume you mean punycode in the domain name, rather than the account within the email address?

[MovGP0]

@blowdart yes, that's correct.

[HaoK]

@blowdart @ajcvickers

How about we make a breaking change and just add NormalizeName and NormalizeEmail (and getting rid of the old Normalize) in ILookupNormalizer instead of adding two new marker interfaces. Our default implementation can continue to just ToUpper both, but then its an easy drop in to tweak the normalization of either/both however people like.

[MovGP0]

@HaoK that would be a possible solution. I'm just unsure if it aheres to the SRP to do so. And to have four interfaces (INormalizerBase, IEmailNormalizer, IUserNameLookupNormalizer, AND INormalizer) seems like overkill.

[ajcvickers]

@HaoK Sounds okay to me.

[blowdart]

Works for me, SRP be damned. There's clean, and then there's pragmatic and usable :)

[Ponant]

@MovGP0 , I am facing a situation with unicode characters that I never had to deal with. Do you mind saying what is the common strategy when dealing with unicode, as it seems you are accustomed to that? For instance, do you store the emails as punycode ascii in the db, and then display them as unicode in the UI, or do you store them as unicode? As far as I can say some browsers will send you punycode while others send you content as is, so there must be some sanitation in place to homogenize the data.
Thanks
https://github.com/aspnet/AspNetCore/issues/9107

[MovGP0]

@ponant I store the email twice. once as unicode (nvarchar), exactly how the user typed it in. I use this address for presenting it in the UI.

Then I store it a second time normalized as punycode and uppercased (varchar). I use this one for authentication and searching.

[Ponant]

@MovGP0 , assuming you use type=email, I don't think this works because of the browsers specific behavior. In all my tests if I enter hello@günter.com then firefox will pass as unicode up to the server but chrome will send you the punycode. So I expect that many of your db entries have punycode in nvarchar. How did you deal with that then?

[Ponant]

@MovGP0 furthermore I find it tricky to find a solution that remains consistent. For instance if you allow user to enter an email or url and he enters say http://faß.de . This link in FF will display as is and route to it. But inm Chrome and Edge http://faß.de will redirect to https://www.bayern-fass.de/ .

[MovGP0]

if I enter hello@günter.com then firefox will pass as unicode up to the server but chrome will send you the punycode.

There are multiple methods to work around this.

1. First you can use <input type="text"> instead of <input type="email">. This will give you the correct formatting, but will mess up autofill.

2. Convert the email address to the unicode format before viewing it to the user.

3. Convert the email address to the unicode format before storing it in the database.

4. Don't do anything of this sort and only handle this case when the customer complains.

Note that there is no right/wrong here. It's more of an personal choice.

This link in FF will display as is and route to it. But inm Chrome and Edge http://faß.de will redirect to https://www.bayern-fass.de/ .

I would not resolve domain redirects, since they may change. Instead, you only need to do two things:

1. parse the mail address using the System.Uri class, in order to make sure it looks valid (remember to prefix the mailto: protocol when the user has not provided it). Don't use a hack like Regex for this.

2. use the address the user has entered and send an e-mail with an one-time token, in order to verify that the address exists and can be properly resolved.
   I usually just write an URL with the token as query parameter that the user can click; as well as preformatted instructions (where the user has to navigate to and copy & paste the token), in case the spam filter has filtered the URL.

[Ponant]

I agree but the main issue is that the user thinks he typed an email but you may receive its puny-coded value on your server because the browser decided to do so. Since chrome is the dominant in the market you will get [email protected] instead. In addition he might in some rare occurrence register to the website using chrome and the day after try to log on with FF. In that case he will not be able to login because FF will send to your server the umlaut ü.

So I think for your strategy 1 or 2 to be complete you would need to
-Receive input on the server and assume it is either unicode (Firefox) or punycode ascii (Chrome) or normal ascii
-Check if it has been punycoded by the browser or not, e.g. by using the IdnMapping class with try and catch (as its methods to convert back and forth to punycode will throw).
-Save to the db this unicode value (which again is not necessarily what the browser has sent you, but which is what the user wanted) and save the punycode value.

In short it would be the same as you proposed except that you cannot just take the value sent by the browser and assume this is what he meant.
If you agree with all of this then I understood you.

[LPLN-HZLD]

how can I remove any ILookupNormalizer injection in identity core3 preview4?
I use identity 2.2 and can't use ILookupNormalizer on ApplicationUserManager.
My contracture ApplicationUserManager :

public ApplicationUserManager(
            IApplicationUserStore store,
            IOptions<IdentityOptions> optionsAccessor,
            IPasswordHasher<TblUsers> passwordHasher,
            IEnumerable<IUserValidator<TblUsers>> userValidators,
            IEnumerable<IPasswordValidator<TblUsers>> passwordValidators,
            ILookupNormalizer keyNormalizer,
            IdentityErrorDescriber errors,
            IServiceProvider services,
            ILogger<ApplicationUserManager> logger,
            IHttpContextAccessor contextAccessor,
            IUsedPasswordsService usedPasswordsService)
            : base((UserStore<TblUsers, TblOrganizationChart, AbfaContext, int, TblUserClaim, TblUserOrganizationChart, TblUserLogin, TblUserToken, TblRoleClaim>)store, optionsAccessor, passwordHasher, userValidators, passwordValidators, keyNormalizer, errors, services, logger)

When I add ILookupNormalizer on AddCustomServices, I have this error:

Method 'NormalizeName' in type 'Project.Core.Identity.CS.CustomNormalizers' from assembly 'Project.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.

after remove "services.AddScoped();" I have this error:

'Method 'NormalizeKey' in type 'Project.Core.Identity.ApplicationUserManager' from assembly 'Project.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.'

I don't need to use Normalizer.
How to solve this?