Kuma: KeyError username on signup with GitHub

[peterbe]

https://sentry.prod.mozaws.net/operations/mdn-prod/issues/7399307/

Resolver404: {'tried': [[<RegexURLPattern None ^media/(?:redesign/)?css/(?P<doc>.*)-min.css$>], [<RegexURLPattern None ^media/(?:redesign/)?js/(?P<doc>.*)-min.js$>], [<RegexURLPattern None ^media/(?:redesign/)?img(?P<suffix>.*)$>], [<RegexURLPattern None ^media/(?:redesign/)?css(?P<suffix>.*)$>], [<RegexURLPattern None ^media/(?:redesign/)?js(?P<suffix>.*)$>], [<RegexURLPattern None ^media/(?:redesign/)?fonts(?P<suffix>.*)$>], [<RegexURLPattern None ^media/uploads/demos/(?:.*)$>], [<RegexURLPattern None (?i)^(?P<one>.*)//(?P<two>.*)//(?P<three>.*)$>], [<RegexURLPattern None (?i)^(?P<one>.*)//(?P<two>.*)$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_1_canvas_rect.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_2_canvas_moveto.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_3_canvas_lineto.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_4_canvas_arc.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_5_canvas_quadraticcurveto.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/2_6_canvas_beziercurveto.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/3_1_canvas_drawimage.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/3_2_canvas_drawimage.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/3_3_canvas_drawimage.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/3_4_canvas_gallery.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_1_canvas_fillstyle.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_2_canvas_strokestyle.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_3_canvas_globalalpha.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_4_canvas_rgba.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_5_canvas_linewidth.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_6_canvas_linecap.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_7_canvas_linejoin.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_8_canvas_miterlimit.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_9_canvas_lineargradient.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_10_canvas_radialgradient.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/4_11_canvas_createpattern.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/5_1_canvas_savestate.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/5_2_canvas_translate.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/5_3_canvas_rotate.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/5_4_canvas_scale.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/6_1_canvas_composite.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/6_2_canvas_clipping.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/globalCompositeOperation.html$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/backdrop.png$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/bg_gallery.png$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_1.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_2.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_3.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_4.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_5.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_6.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_7.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/gallery_8.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/picture_frame.png$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/rhino.jpg$>], [<RegexURLPattern None (?i)^samples/canvas-tutorial/images/wallpaper.png$>], [<RegexURLPattern None (?i)^samples/domref/mozGetAsFile.html$>], [<RegexURLPattern None (?i)^samples/raycaster/input.js$>], [<RegexURLPattern None (?i)^samples/raycaster/Level.js$>], [<RegexURL...
  File "redirect_urls/middleware.py", line 14, in __call__
    resolver_match = self.resolver.resolve(request.path_info)
  File "newrelic/hooks/framework_django.py", line 600, in wrapper
    return _wrapped(*args, **kwargs)
  File "newrelic/hooks/framework_django.py", line 588, in _wrapped
    result = wrapped(path)
  File "newrelic/hooks/framework_django.py", line 575, in wrapper
    return wrapped(*args, **kwargs)
  File "django/urls/resolvers.py", line 394, in resolve
    raise Resolver404({'tried': tried, 'path': new_path})

KeyError: 'username'
(10 additional frame(s) were not displayed)
...
  File "allauth/socialaccount/views.py", line 41, in dispatch
    return super(SignupView, self).dispatch(request, *args, **kwargs)
  File "allauth/account/views.py", line 77, in dispatch
    response = super(RedirectAuthenticatedUserMixin,
  File "allauth/account/views.py", line 190, in dispatch
    return super(CloseableSignupMixin, self).dispatch(request,
  File "allauth/account/views.py", line 103, in post
    response = self.form_valid(form)

KeyError: 'username'

[peterbe]

This user filled in the sign-up form after having authenticated with GitHub (I think).
There's a piece of code that tries to figure out if the user tried to change his username for the benefit of the auth instrumentation.

I'm not sure how it could have happened. It has only happened to 1 user so far but it could be serious if it blocks sign-up for many users. This is fresh new code does the GA auth instrumentation. Need to try to debug how it could happen.

[peterbe]

At the time of writing this, it has only happened to one person. I'm still a bit baffled how so I'm going to email him (he's got a male-sounding name on GitHub) to try to find out.

[atopal]

@peterbe can you clarify what happened and what the consequences are?

[peterbe]

It has happened to 3 distinct people now.
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/7435538/?environment=oregon%3Aprod

[peterbe]

Sentry issue: MDN-PROD-140

[tobinmori]

@atopal, I've told @peterbe he could go ahead and investigate/fix this one now, as it seems like it could be serious (e.g. blocking logins for some users).

Let us know if you don't think this is high priority.

[ghost]

Hi everyone,

I was one of the people affected by this. If there is anything I can do to help out getting this solved don't hesitate to ask.

[ghost]

To resume what I did:

1. Tried to get an account using GitHub authentication.
2. Was asked to select a new username and to confirm my email address.
3. I clicked Create Profile
4. Got the Internal Server Error Page with the message:

"An unhandled error occurred in the application. The error has been logged and an administrator was notified. We apologize for the inconvenience!"

and that's basically it.

[ghost]

Just now, I revoke the MDN OAuth application on GitHub and I tried again to get a new account. I performed the steps that I described on https://github.com/mdn/kuma/issues/6623#issuecomment-594809023 and got the same exact result.

[peterbe]

@marcocampos THANK YOU! It actually helps.
I haven't been able to reproduce it myself yet. But it's a clue to know that you didn't get a validation error or something. E.g. if you'd typed in your own in-valid preferred username.

I'm actively working on this. I'll let you know when a fix is in production (days).

[peterbe]

@marcocampos If I may ask, just in case it matters. Can you share a little about your browser? Which browser did you use and what version?

[ghost]

I'm using Google Chrome 80.0.3987.122 on macOS Catalina 10.15.3. I'm gonna test for the issue using Firefox and I'll report back soon.

[ghost]

Just tried the whole process using Firefox 73.0.1 and I still got the same result.

[peterbe]

I've figured it out. Yay! At least a strong clue. It's unrelated to the browser. It's something on the server side.

[escattone]

@marcocampos Thanks for your help! From looking at our account data, it looks like you created an MDN account using GitHub on Nov. 18, 2019, but your current GitHub account seems to have been created in February of 2020. Did you delete your previous GitHub account and create a new one?

[ghost]

Yes I did. Was that the problem? I totally forgot about that.

[escattone]

@marcocampos

Yes I did. Was that the problem? I totally forgot about that.

Thanks for the feedback! No, that's not the actual problem, but it helps partly explain why your current GitHub login didn't simply use your existing mcampos MDN account created via GitHub on Nov. 18, 2019. I think there are two issues here, one that caused your current error, and the other is related to us not being able to use/recover one or other of your prior MDN accounts (marcocampos created via Persona on April 15, 2014, and mcampos via GitHub on Nov. 18, 2019). I can see now the root cause of both issues, and we should have a fix within a day or two. Sorry for the inconvenience @marcocampos, and really appreciate your help.

[escattone]

@peterbe

Immediate Issue

As you probably already know, the immediate issue of the server error is caused by the combination of https://github.com/mdn/kuma/blob/b4b4a53e8652459abdc031fbd08fd66e92d56170/kuma/users/views.py#L624 and https://github.com/mdn/kuma/blob/b4b4a53e8652459abdc031fbd08fd66e92d56170/kuma/users/views.py#L701

There are multiple ways to fix that issue, but perhaps the simplest is just to change https://github.com/mdn/kuma/blob/b4b4a53e8652459abdc031fbd08fd66e92d56170/kuma/users/views.py#L701 to:

if saved_user.username != form.initial.get("username"):

Second Issue

The second issue is https://github.com/mdn/kuma/blob/b4b4a53e8652459abdc031fbd08fd66e92d56170/kuma/users/adapters.py#L319

I'm not currently handling the case when there are multiple existing user accounts that match one of the incoming verified email addresses, and then trying to recover/merge those accounts. This is an instance of that case, where two existing user accounts matched, one created via Persona and the other via a no-longer existing GitHub account. I think we'll have to think through the best approach to that case. It's not immediately obvious to me.

[escattone]

@peterbe I also created https://github.com/mdn/kuma/pull/6637 as another approach to the immediate issue of the server error.

[escattone]

@atopal @tobinmori We should definitely fix the immediate issue of the server error, but this case also brings up the special case of how to recover an existing MDN account when there are more than one to recover (which hopefully is not common).

[escattone]

@marcocampos Would you like me to delete your two prior existing MDN accounts (marcocampos created via Persona on April 15, 2014, and mcampos via GitHub on Nov. 18, 2019)? It looks like you've not edited any documents with either of the two accounts, so it would be easy to do that. That would give you a clean slate to sign-up again with your new GitHub account, using either the mcampos or marcocampos username, and avoiding this error altogether.

[ghost]

Absolutely! I'm embarrassed to say that I totally had forgotten about those previous accounts :sob:

Sorry for all the trouble!

[tobinmori]

@atopal @tobinmori We should definitely fix the immediate issue of the server error, but this case also brings up the special case of how to recover an existing MDN account when there are more than one to recover (which hopefully is not common).

Thanks for the update @escattone. I let @atopal know that we're taking this on as a p1 for now.

[peterbe]

@escattone I think https://github.com/mdn/kuma/pull/6637 is pretty much ready to go.

Regarding the second issue (the not immediate one), there's some serious thoughts needed there where we have to make a choice between one of the >=2 other matching users.
I do wonder if we can eliminate a large majority of this problem if we pause on that and instead invest in https://github.com/mdn/kuma/issues/6545 instead.
After all, both of his other accounts have 0 revisions. (I'd share the Admin search-by-email but that would reveal too much in his public issue)

[escattone]

@escattone I think #6637 is pretty much ready to go.

Thanks @peterbe!

Regarding the second issue (the _not_ immediate one), there's some serious thoughts needed there where we have to make a choice between one of the >=2 other matching users.
I do wonder if we can eliminate a large majority of this problem if we pause on that and instead invest in #6545 instead.

❤️ I wholeheartedly agree! I think #6545 is a fantastic idea with no downside.

[escattone]

@marcocampos

Absolutely! I'm embarrassed to say that I totally had forgotten about those previous accounts 😭

Sorry for all the trouble!

I deleted both of your prior MDN accounts (marcocampos created via Persona on April 15, 2014, and mcampos via GitHub on Nov. 18, 2019), so you should have no trouble any longer with signing-up with your new GitHub account and choosing one of your prior usernames if you wish. Also, just to let you know, we've merged a pull request (#6637) that fixes the error you were experiencing. Thanks again for your patience and help!

[ghost]

thank you all for solving this so quickly! Keep up the great work! Myself and a lot of other people really appreciate the great work you do on MDN Web Docs and Mozilla. Cheers!

[peterbe]

https://github.com/mdn/kuma/pull/6637 and manual fixing solved this.