Sp-dev-docs: 馃悶 SPO triggering throttling errors (429's) on OOTB telemetry logging

Created on 16 Nov 2018  路  7Comments  路  Source: SharePoint/sp-dev-docs

Category

  • [ ] Question
  • [ ] Typo
  • [x] Bug
  • [ ] Additional article idea

Expected or Desired Behavior

MSFT to not trigger 429's on OOTB pages.

Observed Behavior

Just confirmed this in 3 different SPO tenants... haven't noticed this in the past. There's some call to what appears to be a telemetry collector... it's spamming itself non-stop. All these failure calls are 429's.

image

Steps to Reproduce

Specifically seeing in SPO hosted workbench, but others seeing it off the workbench.

throttling bug-suspected

Most helpful comment

(sorry for the delay)
Thanks for bringing this up. Turns out that the telemetry to make sure that auto-save worked didn't work, causing more telemetry calls to say the telemetry didn't work. The telemetry system then sends information to make sure that the telemetry system is working (I can't make this stuff up), and that system didn't quite work (429s). The work-bench (which generally defaults to edit mode) runs through this codepath. We're going to remove that first telemetry call, which should clear everything else up. (writing this explanation reminds me of the m酶酶se credits in the Holy Grail)

All 7 comments

Hey AC, can you clarify how often those requests are getting sent? Are the 15 or so that you show over the span of a second or two, or a minute, or ?

Trying to figure out if there is an issue on the front end sending too much information, or on the back end where they are sending 429's.


Also, if you switch your filter to be 'collec', you should see a collection of requests like ?qsp. Are you seeing any of those get throttled? Are there a flood of them as well?

@patmill I see it every 2 seconds or so.

Do you also see calls to ?qsp=true... every 2 seconds as well? If you do, can you look at the payload, and let me know what you see? In the network tab, the RequestPayload will have a bunch of somewhat packed data. Feel free to strip out any PII information like URLs, ids, etc.

If you wish, you can send it to me on email. My user name here, at microsoft.com

Yup... seeing the ?qsp=true every 2s as well... I'll send you a screencast offline.

(sorry for the delay)
Thanks for bringing this up. Turns out that the telemetry to make sure that auto-save worked didn't work, causing more telemetry calls to say the telemetry didn't work. The telemetry system then sends information to make sure that the telemetry system is working (I can't make this stuff up), and that system didn't quite work (429s). The work-bench (which generally defaults to edit mode) runs through this codepath. We're going to remove that first telemetry call, which should clear everything else up. (writing this explanation reminds me of the m酶酶se credits in the Holy Grail)

@patmill EPIC issue closing comment!

image

Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues

Was this page helpful?
0 / 5 - 0 ratings

Related issues

bengtmoss picture bengtmoss  路  3Comments

Ralms picture Ralms  路  3Comments

mikeparkie picture mikeparkie  路  3Comments

waldekmastykarz picture waldekmastykarz  路  3Comments

patrick-rodgers picture patrick-rodgers  路  3Comments