Auth0-spa-js: FERRT errors from getTokenSilently intermittently during concurrent requests

[egadstar]

I shared this on the community forums already, but figured, this might be a better place for reporting if it a bug.

Describe the problem

Concurrent calls to getTokenSilently can allow multiple locks to be acquired causing FERRT errors if the access token in the cache has expired.

What was the expected behavior?

getTokenSilently prevents multiple concurrent from acquiring multiple locks.

Reproduction

Unfortunately, this is tough to reproduce as it is intermittent, but I was able to using the SPA SDK playground. I changed the getToken function in the playground to call the getTokenSilently method 15 times. I'm sure there is more aggressive configuration combination that might error quicker, but while debugging what we were experiencing, I was trying to stay as true to our application's situation as possible. Essentially, I tried to wait until the access token was going to expire and start consistently pressing the "Get Access Token" button.

Auth0 API
Audience: my.domain.com
Access Token Expiration: 70s

Application
Refresh Token Behavior - Rotating
Refresh Token Lifetime - 36000
Refresh Token Reuse Interval - 30

Playground
Audience - my.domain.com
Use local storage - true
Use refresh tokens - true
Use token cache when fetching new tokens - true
Use Auth0Client constructor - false

Environment

• Version of auth0-spa-js used: 1.11.0
• Which browsers have you tested in? Firefox 79, Safari 13.1.2

Screenshots

[adamjmcgrath]

Hi @egadstar - thanks for raising this

Would you be able to check it this can be reproduced with the following 2 versions:

• This branch: https://github.com/auth0/auth0-spa-js/compare/master...use-finally
• 1.10.0 (we made some changes around the timing of checking the cache)

Also, if possible, could your try and repro with a longer token expiration (120s)

Thanks!

[egadstar]

@adamjmcgrath

So, I was able to reproduce it on both use-finally and 1.10.0 with the token expiration at 120s. Looking at the browser-tabs-lock library, there appears to be a few delays coded to attempt to deal with race conditions; maybe it's being defeated?

here's one of them:

[adamjmcgrath]

Thanks for checking that @egadstar

Will try and reproduce it myself and get back to you

[adamjmcgrath]

Hey @egadstar - apologies for the radio silence. I'm planning to take a look at this this week

[adamjmcgrath]

Hey @egadstar - I've had a look at the issue and I'm able to reproduce it.

The problem occurs when acquiring the lock times out. If the async operation takes about 500ms, and you queue up 15 operations instantly. The 11th operation will have been waiting for ~5.5secs (500ms * 11).

Since the lock times out after 5 secs. The 11th and subsequent 3 in the queue will all be allowed to pass at the same time. If during this time your access token has expired, you will see one of those calls succeeding and the others failing with FERRT errors.

I've put a branch here that demonstrates the problem https://github.com/auth0/auth0-spa-js/compare/locks-test

Run for (let i = 0; i < 16; i++) { auth0.getTokenSilently() } and in the console you will see something like:

num locks:  1 , last token acquired:  true , ms waited:  586
num locks:  1 , last token acquired:  true , ms waited:  1179
num locks:  1 , last token acquired:  true , ms waited:  1774
num locks:  1 , last token acquired:  true , ms waited:  2359
num locks:  1 , last token acquired:  true , ms waited:  2949
num locks:  1 , last token acquired:  true , ms waited:  3540
num locks:  1 , last token acquired:  true , ms waited:  4130
num locks:  1 , last token acquired:  true , ms waited:  4721
num locks:  7 , last token acquired:  true , ms waited:  5314 <= after 5secs
num locks:  6 , last token acquired:  false , ms waited:  5507
num locks:  5 , last token acquired:  false , ms waited:  5509
num locks:  4 , last token acquired:  false , ms waited:  5508
num locks:  3 , last token acquired:  false , ms waited:  5508
num locks:  2 , last token acquired:  false , ms waited:  5508
num locks:  1 , last token acquired:  false , ms waited:  5508

Obviously if the server response times are slower, you will need to hit this less frequently to reproduce it - but you will always need the token to expire while you're making multiple requests, which makes this hard to reproduce (adding ignoreCache: true would make it easier to reproduce)

I'll have a chat with my team and look in our logs to see how common FERRT errors are with the SPA JS client.

[adamjmcgrath]

Hey @egadstar - I haven't had a look at this yet, but I'll update you when I have

Incidentally, I noticed that your Access Token Expiration is 70s. Since we invalidate the token cache if the token expires with 60s, it means that we'll fetch new tokens every 10 secs. Therefore if you increase token expiration to 80s, you'll halve the chance of this issue happening - this might be a reasonable workaround if you're seeing this issue only intermittently.

[egadstar]

@adamjmcgrath No worries, thanks for the update. Unfortunately, our production applications have set the access token to a 24 hour expiration to try and smooth out the end user experience before I posted here, but we still have tons of FERRT errors in the logs from when the users comes back to the application while still being authenticated the next day. For obvious security reasons, would like to reduce the access token expiration to a more reasonable expiration. I have been toying around with the idea of just building a worker that is in charge of keeping the cache's token valid, so calls to getTokenSilently are never refreshing the access token and pulling from the store.

[adamjmcgrath]

Hey @egadstar - I would look at just debouncing getTokenSilently first (this is what we'll probably end up doing) - the browser tabs lock stuff is only needed for concurrency issues across tabs.

[stevehobbsdev]

@egadstar 👋 just checking in to say that we are continuing to find a solution for this, I'll tag the PR here when we have something.

[rishabhpoddar]

This issue is due to the incorrect usage of SuperTokens' browser-tabs-lock library. I had opened another issue for this here.

Fix for this: https://github.com/auth0/auth0-spa-js/pull/658. Please note that while the library is being correctly used in the PR, it causes some other tests to fail. I'm not entirely sure why the other tests would fail.