Gitpod: Access Control settings resetting (seemingly at random)

[kevin-ci]

Describe the bug

The checked settings on the Access Control page occasionally seem to reset, resulting in this:

This causes a 403 unable to push status when trying to push. Note that in all cases, users were previously able to push from the same workspace. This seems to have just reset at random. Very strange.

Have noticed it happening with at least ten students over the past few months. One account which was recently affected: https://github.com/hschafer2017

Steps to reproduce

Unknown. Haven't been able to determine a pattern in the people affected.

Expected behavior

The settings stay checked until explicitly modified by a user.

Additional information

Reselecting the desired Access settings and clicking Update fixes the issue (i.e. the updated settings remain checked).

All affected users have been on the same Gitpod team/plan.

Cookies of an affected user:

And their browser settings:

Example repository

[Kreyren]

the Access Control page occasionally seem to reset,

First thing that comes in mind when i see this is cookie issue. Are you blocking any from gitpod.io domain?

EDIT: This also seems to be related to the Err403

[Kreyren]

Second thing that comes in mind is gitpod failing to store the required value in a database(?) or the database(?) being corrupted to restore in original state.

EDIT: Which assuming this being related to few students on i assume your network? Seems very unlikely, but possible?

[Kreyren]

Third thing that comes in mind: GitHub shortage that fails to store the relevant data somewhere? assuming this seemingly being used as a frontend to github API?

@kevin-ci You said this happening seemingly on random? If so that would point to that.

[Kreyren]

cc @nisarhassan12 assuming that he should have more info about how are these checks handled

[kevin-ci]

@Kreyren Yes, it appears to be happening at random. Am following up on cookie-blocking now. Will update the issue with info as soon as I get it.
All of the affected users/students are on the same team, as it happens. Will also add this info to the report.

[Kreyren]

cc @jankeromnes The fact that this is happening to users in the same team makes me think of some bug in gitpod?

[kevin-ci]

Have updated with cookie info. None seem to be getting blocked.

[Kreyren]

In which case my best guess is something related to GitHub shortage, but i don't have inside to that so better to wait for jankeromnes

[AlexTugarev]

Thanks for reporting @kevin-ci! I want to investigate this. A quick search in the logs showed no results for the user mentioned above. Do you have another one? Maybe a timeframe?

My best guess at the moment is that it's due to GitHub shortages we've seen quite often in the last week. On login into Gitpod you will be sent to GitHub twice, if you're an existing user and you have granted additional permissions in a previous session. If the second round fails due to network issues for instance you likely will end in this situation.

[kevin-ci]

Hey @AlexTugarev. Will put out a call to students asking anyone who is affected to reply here with their username and as best a timeframe as they can. Cheers :)

[jos-lab]

Hi @kevin-ci. Upon pushing to git also I get the 403 error also. My user name is jos-lab.
I experienced the error immediately I reset my github account password on 25th May.

[Kreyren]

I experienced the error immediately I reset my github account password on 25th May.

Why is changing your password relevant?

[jankeromnes]

I think this just happened to me this morning (although it could also be that I tested removing permissions and forgot about it).

Was surprised to see an error on git push:

$ git push upstream autofix-upgrade-pyenv-tools-63 
remote: Permission to gitpod-io/workspace-images.git denied to jankeromnes.
fatal: unable to access 'https://github.com/gitpod-io/workspace-images.git/': The requested URL returned error: 403

There was no permission alert in the IDE, but manually going to Access Control and granting back read/write repo permissions fixed the above error.

[Kreyren]

FWIW i don't have issues with git in theia (i am granting the permission per repository)

but i can't fork any repository through gitpod.

[RoyoftheRavers]

I had this issue for several days. Needed to contact CI support. My last successful push was yesterday June 5th. But I'm hoping to have more successful pushes today since I accepted the permission settings!

[kevin-ci]

Reporting another on behalf of https://github.com/Jammerref2015
They noticed the issue on June 3rd. Their last successful push before that was four weeks earlier though, so quite a large timeframe.

[kevin-ci]

Reporting another for http://github.com/Astig-1982
Seems to have happened twice to them over the last three days. The first was after having connection issues to Github (internet connection was fine, the site was having issues).

[Kreyren]

My best guess would be that this is caused by GitHub's uptime then and if so why is only a small user-base affected?

Maybe something relevant to GitHub mirror in the location?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.