Next-auth: Support for signing in without an email address

[iaincollins]

Overview

Currently v2 requires an email address on on oAuth profile to be able to sign in.

An issue with this is some providers, notably Twitter, are little different to most oAuth providers in that they don't return the email address unless you explicitly enable the option.

In the case of Twitter, this is the option "Request email address from users" permission:

Context

v1 copes oAuth providers that didn't return an email address by creating a dummy email address that looked something like {profileId}-{providerName}@localhost.localdomain (see this line and these lines from v1).

This ensured the accounts had a unique, well formed (but not real) email address associated with them.

It looks for email addresses like this and replace them with the users real email address if they ever link their account with with another service that did have an email address.

NB: Account linking is technically possible in v2 (you just sign in, then sign in again with a different service) but it's not clear yet, and you can't unlink accounts.

Users could also specify a valid email address if they were signed in; this is not something that is supported in v2 yet (when it is, it will work differently by requesting users confirm the email address before it is changed in the DB).

What should we do?

The easiest solution is to tell people to enable the option to return the email address in Twitter.

We could put a note about this in the docs.

Some other things we could do:

• We could improve the error messaging - eg return a specific error page that explains what is wrong (this might be the easiest thing; and we could even do this first, and change it later).

• We could try to do the same behaviour as v1 and support creating accounts without an email address. This is a great end user experience but requires some hackery like the above code I've linked to in v1 to implement and might be confusing for developers to have users with weird address in their user table if they forget to enable this option on sites like Twitter.

• Perhaps we could just put special logic in the Twitter profile() handler, to return a dummy email address (e.g. {profileId}:{providerName}@localhost.localdomain)?

[mihar-22]

Hey @iaincollins I've been following the library, cool stuff. Thanks for the hard work you've been putting into it.

You might already be aware but I just wanted to add that the email can return empty across providers for a bunch of other reasons, such as if someone has logged in via mobile, or in some cases when their email is not verified.

In my opinion the current solutions are a little too hacky and will come back to bite you. It'll definitely trip developers out storing strange emails for users, even worse they might not realize it's fake. I could be wrong though.

I propose we allow the email field to be optional/nullable because it can be and we should follow Auth0's strategy for account linking, in which all identities are treated separately (no automatic merging or linking because it's not safe). I won't go further because the link I provided goes into detail on how the process can be handled. It's best left for users to manually merge their accounts.

Just a side note I think we should also introduce an emailVerified field to the UserSchema.

[iaincollins]

Thanks for the feedback!

You might already be aware but I just wanted to add that the email can return empty across providers for a bunch of other reasons, such as if someone has logged in via mobile, or in some cases when their email is not verified.

So, the only time I've actually seen this come up is specifically with Twitter in the case the developer has not enabled returning the email address (this feature was supported in Twitter oAuth initially and they added it on later).

When the option is enabled in Twitter, I have not see this issue in scenarios or with any other providers. I'm conscious I've have not tested unverified email address flows with all providers (I know it varies, and some refuse to let you log in until you verify your account, and other pass a flag to indicate if an address has been verified by them or not).

However, I'm be interested in replicable examples of other scenarios or providers (other than with self-hosted oAuth services) where we see this, to frame the scope of how important this to address and help decide what the best course of action is.

In my opinion the current solutions are a little too hacky and will come back to bite you. It'll definitely trip developers out storing strange emails for users, even worse they might not realize it's fake. I could be wrong though.

For context, v1 used the same approach; I don't think we've ever had a bug report about it, but I get that is a bit weird.

This approach was chosen in v1 as using a convention for domain is prevalent on mail systems and on some unix systems (though is not actually specified in an RFC).

The email addresses use @localhost.localdomain as they are syntactically valid, but can otherwise be treated like a real address (eg. expired domain/deleted account) that won't deliver anywhere, which is a use case that also need to be handled by anyone doing anything with the user supplied email addresses.

It's been a product decision based on making a call over the best user experience trade off (with "users" in this context being next-auth developers, next-auth users and end users) and weighing up the happy paths and negative paths for all those people.

With that in mind, I think likely outcomes are either:

1. Email addresses will be required to be on a profile.
2. Automated address will be created as in v1.

I have also considered another option:

1. Instead of having email addresses on a User, we could create them in another table (e.g. Email), much like Accounts table is used to store data for oAuth accounts.

Technically this would work very well, as email accounts could easily then be optional and you could associate and verify multiple email address (e.g. in case your primary email address stops working).

While theoretically this is logical, the downside to the approach is introduces quite a bit more complexity for next-auth developers, next-auth users and that will almost certainly manifest in more complexity for end users too.

The product trade off includes working out if is it simpler to focus on a single email address per user as that is a much easier flow for developers to support, or is it worth the tradeoff to support multiple email addresses (or none) per account.

Relevant to this trade off is who are the users (including developers and end users) and what are the risks. If the software is too complicated people won't want to use it, and if they do use they might be more inclined to make mistakes, which could have implications for security.

It might also mean doing quite a bit more work in next-auth to support it, which could mean it is not feasible - but doesn't rule it out in future.

I propose we allow the email field to be optional/nullable because it can be and we should follow Auth0's strategy for account linking, in which all identities are treated separately (no automatic merging or linking because it's not safe). I won't go further because the link I provided goes into detail on how the process can be handled. It's best left for users to manually merge their accounts.

Not requiring email addresses at all is something we could do, but is problematic; accounts should have a unique constraint on the email address to enforce that there is no scenario where you can ever have two accounts with the same email address, however every database allows multiple instances of null, so the unique constraint would have to be removed, which is potentially dangerous.

Multiple fields being null when unique is defined is valid ANSI behaviour in SQL, but sadly not actually how all SQL databases work and of course not all supported databases are SQL.

To be clear, neither v1 or v2 do automatic account linking (and never have) because it is not safe and there are no plans to change that. Linking only occurs when a user is currently signed in already with valid account.

The exception to this is that if you do have an email address specified you can use that for account recovery, which I see as key feature and would not look to remove, but I would consider making it an option.

Just a side note I think we should also introduce an emailVerified field to the UserSchema.

This was present in v1 and is under consideration for v2 pending a decision on the above, I think a flag like this will definitely return (possibly as a date/time stamp rather than a bool so you can automatically check if someone hasn't confirmed an address is still valid in a while).

[mihar-22]

Awesome, thanks for the detailed response and enlightening me.

Here's the situation for Facebook and Google with regards to the email situation.

Facebook

The User's primary email address listed on their profile. This field will not be returned if no valid email address is available.

This could be due to (1) no email address was supplied because they signed-in via mobile, (2) they have not completed adding an email, (3) the email address is not verified.

Google

The user's email address. This value may not be unique to this user and is not suitable for use as a primary key. Provided only if your scope included the email scope value.

Note User.email is a scoped permission so it might not be granted, thus no email.

---

Instead of having email addresses on a User, we could create them in another table (e.g. Email), much like Accounts table is used to store data for oAuth accounts.

The solution does sound good on paper but as you said the implementation cost is too high for this sort of library. I still think the email field should be optional because it's clear when no email has been associated with the account. Developers can then decide how severe it is for their application whether users have supplied an email or not. If it's severe they could simply request that the user provide their email, potentially blocking certain actions in-app till they do so. I think the key point is it's much more natural and it's easier for the developer to decide what to do.

If we go ahead with option 1 (auto-gen email), the issue is it'll be hard to tell if a valid email is available or not without parsing the email field, I guess you could add a validEmail boolean but it just seems unnecessary. As a developer you'd simply check if user.email is defined or not and make a decision.

Multiple fields being null when unique is defined is valid ANSI behaviour in SQL, but sadly not actually how all SQL databases work and of course not all supported databases are SQL.

Which SQL database doesn't support this? That'd be a broken database wouldn't it? NULL is the absence of a value and not an actual value, so it simply can't violate the unique constraint. That is why you never do WHERE email = NULL but do WHERE email IS NULL instead.

Pretty sure you can achieve unique and null columns with NOSQL using partial indexes. This is true for MongoDB, not sure what specification all NOSQL databases conform to.

This was present in v1 and is under consideration for v2 pending a decision on the above, I think a flag like this will definitely return (possibly as a date/time stamp rather than a bool so you can automatically check if someone hasn't confirmed an address is still valid in a while).

Agreed on it being a timestamp, much more informative and serves the same goal as a boolean. We could name it emailVerifiedAt instead.

[iaincollins]

RE: Facebook. Thanks that's useful information.

RE: Google, we always get the email address as the scope is defined in the profile.

Users can always link/unlink oAuth accounts at will once signed in - and use email to sign in and unlink an old oAuth account if they no longer have access to it.

The email address on the oAuth account is not used as key for linking accounts; one email address can be securely linked to multiple oAuth accounts with the same (or different) email addresses; it's just used to populate the user object.

It's really up to the user journey for the specific site to handle what happens after that, based on what they need their users to agree to (e.g. provide a delivery address, enter a billing number, prompt them to confirm the email address was the one they meant to use with this account, etc); there are specific options to help with that such as forcing new users to a landing page URL.

Which SQL database doesn't support this? That'd be a broken database wouldn't it? NULL is the absence of a value and not an actual value, so it simply can't violate the unique constraint. That is why you never do WHERE email = NULL but do WHERE email IS NULL instead.

Microsoft SQL Server used to have that limitation (though thankfully is no longer the case); I've run into it on NoSQL databases too; it's also a fairly universal business constraint (relevant to writing additional / custom adapters).

Bear in mind too that the default adapter uses TypeORM for database abstraction and that has a significant impact on how queries are handled across different databases and the features I've used. For example, it's why the model is deliberately very simple, why relationships are not defined in the schema and why some of the keys are they way they are.

Making email address optional is one thing, but allowing email addresses to be nullable in the model is another and not a use case I have.

However, if someone is write the code to expose a runtime option (e.g. boolean) that allows email address to be optional - and to take on doing all the edge case and cross database testing for the flows in that scenario - then I would welcome a PR for it.

I'd also consider doing it if someone was willing to sponsor the work.

[mihar-22]

Ye I completely understand, I'm not trying to put more work on your plate just wanted to discuss ideas around handling of the email field. It's a miracle you've put in this much time for others, appreciate it. When I get a little more time on my hands I'll see how it goes with implementing a nullable email field.

Your option 3 of a separate Email table is a neat alternative as well, so you could still go down that route of optional email with low cost.

I just wanted to add are you sure the email is included for Google under the profile scope? It might be if the email is made publicly available or something but it seems to be listed under a seperate scope here -> https://developers.google.com/identity/protocols/oauth2/scopes.

userinfo.email -> View your email address

userinfo.profile -> See your personal info, including any personal info you've made publicly available

[iaincollins]

Hmm, maybe we could do something like require it short term, but shoot for adding in option 3 later (which feels doing it really well without technical downsides, but will take a bit longer).

RE: Email scope, you can check out providers/google.js.

[iaincollins]

Update: For now, more robust error handling has been put in place, so that users are requested to sign in with another account that has an email address associated with it.

This will be carried forward to a future release, for now have moved it to 2.1

[iaincollins]

For the record, this the sort of problem folks run into with badly behaving SQL servers that don't respect the ANSI SQL standard (which allows duplicate null values) https://github.com/typeorm/typeorm/issues/4859

The thing is MSSQL is the most prominent problem database here, and as I understand it is finally fixed in newer versions - but I would appreciate confirmation of that.

In addition to the changes to the flow (which now handle this better) I've been considering practical options to support this.

@LoriKarikari has been adding new OAuth providers and it looks we are going to have quite a few that never return email addresses, in addition to the ones that don't _always_ return an email address.

Options currently under consideration

Option 1) Add a flag to make email addresses optional

When enabled, we could just generate a random address (as per v1) like {uid}@localhost.

This would be very hacky but is also quick and simple to add and would be fine for lots of people and it's easy to check for in a conditional.

e.g. if (email.endsWith('@localhost') { ... }

If it was a flag, it would easy for people to use or not use, and when they go to turn it on the documentation could spell out what the behaviour will be so it won't catch people out.

Option 2) Alter the model and see what breaks

Alternatively, we could alter the user model, remove the check for missing email values and add in error handling to ensure that everything works without an email address.

If we do run into people reporting problems in future, we could go back and add in the option to make this conditional based on that option.

This is elegant but is likely to be more work, especially around edge cases (linking, unlinking) and may impact user experience, especially if multiple providers are configured.

Considerations

We probably want to do linking / unlinking (#230) and adding support for account deletion (#228) first or soon after, as allowing people to sign in without an email address increases the changes of people accidentally creating two accounts if the option to sign in with more than one provider is enabled.