Azure-docs: How long does it take for new telemetry to be stored in the blob storage with continuous export?

[vlflorian]

See title

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 860f6a72-85d8-6296-d9f0-638907f1382b
• Version Independent ID: 36512771-6c2a-a651-d8e0-fde95f563252
• Content: Continuous export of telemetry from Application Insights
• Content Source: articles/application-insights/app-insights-export-telemetry.md
• Service: application-insights
• GitHub Login: @mrbullwinkle
• Microsoft Alias: mbullwin

[mrbullwinkle]

@vlflorian thank you for your question. The current guidance we provide in the doc your referenced is _"There can be a delay of about an hour before data appears in the storage."_ after you setup continuous export.

[vlflorian]

@mrbullwinkle Hey, thanks for the reply! There can be a delay of up to an hour, but most of the time it doesn't take as long, right? What real delay is to be expected?

[mrbullwinkle]

@vlflorian that is my understanding, but let me ping the PM who owns continuous export to confirm.

[mrbullwinkle]

@vlflorian one quick follow-up that I am sure the PM will ask me. Do you have a specific use case where an up to an hour initial delay doesn't work for you and you need it to be consistently kicking off in a shorter time frame when you setup continuous export, or is it just curiosity as to what the potential expected min/max range for data being populated is?

[vlflorian]

@mrbullwinkle The first option, sir! Specific use case where we'd like a third party to view the state of our applications by extracting the AppInsights Data, dump it into an sql database and use Tableau to query that database. I know AppInsights has PowerBi integration, but we have to use Tableau for a reason that is difficult to explain this way and the default integration with PBI isn't anywhere near real time either.

I set up continuous export just recently and after only a couple minutes I could see data in the Blob container already, so it probably won't always be an hour until it gets in there?
Do you think we might be better off using the REST api to query AppInsights for data?

[mrbullwinkle]

@vlflorian that you so much for the awesome level of detail, that will help a lot.

[vlflorian]

@mrbullwinkle Hi, have you perhaps got any more information on this matter?

[mrbullwinkle]

@vlflorian yes sorry for the delay. There is the possibility that it can vary by region but the PM pulled in the engineering team and they ran numbers for the past 30 days for South Central US to get you a general idea. Once continuous export has gone through the initial setup and has already written data to blob storage the actual ongoing export is quite fast with an average latency of 1 minute, and a 99 percentile of around 2 minutes.

With that said you probably want to be cognizant of the entire ingestion pipeline which looks a little like:

SDK --> Internal System --> Export --> Customer Storage

The engineering team also pulled numbers for the Internal System portion of the ingestion pipeline for South Central US to give you a better sense of the portion of the latency picture which wouldn't necessarily be transparent to you.

Average latency for the Internal System is close to 5 min. (Though effort is ongoing to improve this performance). So once telemetry has gotten to that portion of the pipeline you can assume approximately 5 min + 1 min (for export) = roughly 6 minutes before you would see that data in your Azure Blob Storage.

The 99th percentile might hit approximately 10 min for Internal System + 2 min (for export) so up to 12 minutes.

Again this can vary by region, and does not account for the first time you ever write data to blob storage via continuous export. For the first time the engineering team felt there might be a few extra minutes of delay, but after that the approximate latency ranges above should apply.

[vlflorian]

Alright, thank you very much for the detailed answer and for the help.