Amplify-js: Pinpoint Analytics Exceeded maximum endpoint per user count 10

[Jun711]

Do you want to request a feature or report a bug?
seems like a misconfiguration issue?

What is the current behavior?

Amplify analytics configuration

Analytics: {
    appId: '***',
    region: 'us-east-1',
    // endpointId: '***'
  }

When I had endpointId as a constant, I could only see one endpoint at Pinpoint. Thus, I got advice in gitter that I should let Amplify set the endpointId. Thus, I did that and I see this Exceeded maximum endpoint per user count 10 error for some users:

400 Bad Request with pinpoint saying that
{"RequestID":"*","Message":"Exceeded maximum endpoint per user count 10"

What is the expected behavior?

Which versions of Amplify, and which browser / OS are affected by this issue? Did this work in previous versions?
0.4.3

You can turn on the debug mode to provide more info for us by setting window.LOG_LEVEL = 'DEBUG'; in your app.
yes

[powerful23]

The issue happens if your localStorage are cleaned too many times or you don't have localStorage so that the Amplify will generate a new endpointId every time.

[Jun711]

@powerful23
yea, I have it clean the localStorage specifically 2 cognito keys when I can't log in due to
Linked logins disabled on Federated Identities
https://github.com/aws-amplify/amplify-js/issues/1015

[powerful23]

@Jun711 Unfortunately for now this is an issue we can't provide a workaround. We will talk this with the Pinpoint service team to find a solution.

[Jun711]

Hi @powerful23, for a website, should we have a fix endpoint? or let Amplify assigns the endpoint? Thanks

[powerful23]

@Jun711 the endpoint stands for the browser in the device so I don't think the endpoint . When using amplify, it will automatically generate an Id and store it in your browser's localStorage. It will update the endpoint details like the browser version etc. If you don't want Amplify to assign the endpoint, you can configure it by yourself: https://aws-amplify.github.io/amplify-js/media/analytics_guide#manual-setup

[Jun711]

@powerful23 ok, I got it. Previously, I assigned only one fixed ID in the Amplify config and thus, I saw only one endpoint at PinPoint console. Because of the limit of 10, users can see a lot of PinPoint errors if they have the console open.

[TheVaporTrail]

@powerful23 I don't understand your previous comment: "...so I don't think the endpoint ." Is it missing a word?

I am receiving the same bad request error. Is there a recommended way to manually configure the endpoint so that I see one endpoint in the Pinpoint dashboard per user? I get the impression from @Jun711's comment that I will only see one endpoint in total if I use a fixed endpoint in my app.

As a possible feature request, would it be possible for Amplify to manage this better?

[powerful23]

@TheVaporTrail the endpoint is for per device so ideally each of the device(browser) has one unique endpointId. There is a userId property in the endpoint context which is for the user. Each userId can have several endpointId which means the user logged in through different devices.

There is a limit for the number of endpoints of each user so that's why the bad request happens. If your device can not persist the endpointId, then each time a new endpointId will be generated for that device which will also hit the limit.

Yes we are working on to find a solution for this case.

[hounded]

Am I correct in understanding that if my user logs in many different web browsers, one at home one at work, one while traveling, eventually he is going to hit this issue?

[powerful23]

@hounded yes

[leantide]

I would even be content to workaround this by disabling Analytics. Unfortunately, the issue persists even when I set aws_app_analytics to 'disable' in aws-exports (I am using Mobile Hub) and also persists even when I call Analytics.disable().

The only way I can shut it down is to remove aws_mobile_analytics_app_id and aws_mobile_analytics_app_region from aws-exports altogether.

[bbernays]

Any update on how to fix this issue?

[powerful23]

@bbernays hi we've escalated this issue to the service team. Waiting for their response.

[mohanrao]

Hi This is happening after i upgraded from aws amplify version

[mohanrao]

@powerful23 Any update on this?

[mohanrao]

Hi,

Any update on this ??

Can we trust using this library or AWS for production critical apps??

@powerful23

At least any update on this will really help.

[CiscoKidxx]

Any update @powerful23 ?

[mevert]

We are also facing this similar issue and it came out of nowhere for our users today. Our config is following:
Amplify.configure({ Analytics: { AWSPinpoint: { appId: config.pinpoint.APP_ID, region: config.pinpoint.REGION, mandatorySignIn: true }, ... }

For now we will take Pinpoint out from production and replace it with some other useful user analytics service.

[powerful23]

Sorry for the late response. I will work on a fix in the next few days. The root cause of the issue is that the endpoint id is cached in the local but might be expired/removed for some reason and thus a new one will be generated which will increase the number of endpoints per user. I am going to call this api: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Pinpoint.html#getUserEndpoints-property to get those endpoints from the specified user id and see if we can unlink some unused ones.

[mohanrao]

@powerful23 in which version this is going to be released?

[janpapenbrock]

@mohanrao This appears to have landed in 1.1.18 at least, but it throws errors for me.

[mohanrao]

@janpapenbrock I see this is still happening for 1.1.18

@powerful23 Please help

[powerful23]

@mohanrao did you add the mobiletargeting:GetUserEndpoints policy in your IAM role? This policy is required to get rid of this error. You can also open the debug mode by window.LOG_LEVEL='DEBUG'

[mwarger]

@powerful23 How and where do you add the mobiletargeting:GetUserEndpoints policy? Which IAM role does it need to be added to?

I'm getting this error on the most recent version of amplify. I think this issue should be re-opened. Thanks for any assistance.

[powerful23]

@mwarger if you are using Cognito Identity Pool Service to get the AWS Credentials, then you will have two roles, one for unauth identity and on for auth identity, you need to add this policy to these two roles.

[rromanchuk]

This needs to be added to documentation somewhere, anywhere. Also
mobiletargeting:GetUserEndpoints isn't an action recognized by IAM editor

https://docs.aws.amazon.com/mobileanalytics/latest/ug/migrate-sdk.html

https://console.aws.amazon.com/pinpoint/home/?region=us-east-1#/apps/[app-id]/settings/nativeAnalytics

Still getting Message=Exceeded maximum endpoint per user count 10, responseDataSize=57, NSLocalizedFailureReason=BadRequestException} on ios

[bneigher]

Why is this closed?

[kvasukib]

@rromanchuk Sorry for the delayed response. I have created an issue on GitHub to add links to the documentation: https://github.com/aws-amplify/docs/issues/726

[bernhardt1]

@powerful23

Perhaps there is a regression? I have GetUserEndpoints in my IAM policy but I am still seeing Exceeded maximum endpoint per user count: 10

I saw this error a few months ago and had fixed it by updating my IAM policy with mobiletargeting:GetUserEndpoints. Now the error has returned.

versions:
"@aws-amplify/pushnotification": "^1.0.28"
"aws-amplify": "^1.1.29"
"aws-amplify-react-native": "^2.1.13"

[bneigher]

@powerful23 I'm pretty sure I see the problem...

Looks like in the code it is checking for the err.message to be: Exceeded maximum endpoint per user count 10
But that is actually not the value, note the actual err.message is Exceeded maximum endpoint per user count: 10 with a colon.

Thus the routine this._removeUnusedEndpoints is never firing.

https://github.com/aws-amplify/amplify-js/blob/master/packages/analytics/src/Providers/AWSPinpointProvider.ts#L353

Perhaps make that check not care about anything after count, as that is technically a magic number and may change on aws side in the future... suggestion:

if (err.message.startsWith('Exceeded maximum endpoint per user count')) {
  ...
}

Do I get a cookie?

[bernhardt1]

@bneigher

Nice catch. That took care of the issue immediately.

I would email you a cookie if I could. <3

[bneigher]

@powerful23 ❗️
I'm realizing that there must have been some kind of service regression that this code didn't account for, and I'm seeing a bunch of users on my platform that have maxed out their endpoints... any idea when ya'll could push an update either to amplify-js or the pinpoint backend? I would imagine others are silently suffering from this too 😱

[powerful23]

Hey Guys, thanks for reporting. I have sent a pr as @bneigher suggested to fix this issue. Thanks!

[joebernard]

@powerful23 This still seems to be a problem as of "@aws-amplify/analytics": "^3.1.4". If I re-install my app enough times it hits the 10 endpoint limit and does not clear unused ones. I've only installed this app on a single test device so there should be plenty of unused endpoints to delete.

It seems this line isn't recognizing the error message correctly and not attempting to clear out the old endpoints:

https://github.com/aws-amplify/amplify-js/blob/master/packages/analytics/src/Providers/AWSPinpointProvider.ts#L460

My IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "mobiletargeting:GetUserEndpoints",
                "mobiletargeting:PutEvents",
                "mobiletargeting:UpdateEndpoint"
            ],
            "Resource": "arn:aws:mobiletargeting:*:*:apps/*"
        }
    ]
}

Packages:

    "@aws-amplify/analytics": "^3.1.4",
    "@aws-amplify/auth": "3.2.1",
    "@aws-amplify/cache": "3.1.4",
    "@aws-amplify/core": "3.2.1",
    "@aws-amplify/pushnotification": "3.0.5",
    "@aws-amplify/storage": "3.1.4"

[daundkarmangesh]

This issue is resolved in the latest stable versions of analytics and auth. Installing below versions resolved the issue for me.

"@aws-amplify/analytics": "^3.2.5",
"@aws-amplify/auth": "^3.3.3"

[njdullea]

I am getting this error with aws-amplify 3.3.10.

[rjuevesano]

same here with "aws-amplify": "^3.3.2"