Azure-docs: Does Time Series Insights maximum property limit apply to TSI Preview

Created on 23 Aug 2019  Â·  10Comments  Â·  Source: MicrosoftDocs/azure-docs

TSI GA has a maximum number of properties/ columns of 600-800

https://docs.microsoft.com/en-us/azure/time-series-insights/how-to-shape-query-json

There is no limit mentioned on this page - can I assume there is no limit on the Preview?


Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

Pri2 assigned-to-author product-question time-series-insightsvc triaged

Most helpful comment

Hi @xtellurian - I'm afraid that I am unaware of the absolute max cap. My apologies - I will research this further and try to identify what that is from the Engineering and PM team.

I can, however, provide you with some answers to your original question:

  1. Using two custom JSON files: https://gist.github.com/KingdomOfEnds/1272e22b29630dc8ec2ce07fc87187f0, I was able to pass in 800 and 900 entries or so into IoTHub and then into TSI Preview. I used the following JS:
/**
 *  Copyright (c) Microsoft. All rights reserved.
 *  Licensed under the MIT license. See LICENSE file in the project root for full license information.
 */
'use strict'
const c = require('../config')
const Mqtt = require('azure-iot-device-mqtt').Mqtt
const DeviceClient = require('azure-iot-device').Client
const Message = require('azure-iot-device').Message
const YOUR_CONNECTION_STRING = c.CONFIGURATIONS.IOTHUB.PREVIEW
const YOUR_INTERVAL = 1000
const client = DeviceClient.fromConnectionString(YOUR_CONNECTION_STRING, Mqtt)

let x = 0
while (x < 100) {
    //800 or 900-property.json
    let js = require(`${c.PROJECT_PATH}/tsi_column_tester/_out/800-property.json`)[0]
   //test or test2
    js = Object.assign({"deviceId": "test"}, js)
    const message = new Message(JSON.stringify(js))

    client.sendEvent(message, err => {
        if (err) console.error(`send error: ${err.toString()}`)
        else console.log('message sent')
    })
    x++
    setTimeout(() => {
    }, YOUR_INTERVAL)
}
  1. 1

  2. 2

So, it look likes the original cap has been raised, unfortunately, I do not yet have a maximum cap but I hope the information above answers the immediate question and will aid you in your testing. I will continue to sync with the PM and engineering team's for a specific maximum cap.

Thanks!

All 10 comments

@xtellurian Thanks for the feedback! We are currently investigating and will update you shortly.

@deepakpalled Can you please confirm on this? Couldn't find more details regarding the limits.

@deepakpalled @AshokPeddakotla-MSFT did you find the answer? It makes a big difference in resource planning and architecture decisions...

Hi @KingdomOfEnds , sorry to pull you into this, but there hasn't been any activity on this issue in a month. It seems like a simple documentation issue with a yes/ no answer (unless I'm missing something)

Could you please help?

Hi @xtellurian - I'm afraid that I am unaware of the absolute max cap. My apologies - I will research this further and try to identify what that is from the Engineering and PM team.

I can, however, provide you with some answers to your original question:

  1. Using two custom JSON files: https://gist.github.com/KingdomOfEnds/1272e22b29630dc8ec2ce07fc87187f0, I was able to pass in 800 and 900 entries or so into IoTHub and then into TSI Preview. I used the following JS:
/**
 *  Copyright (c) Microsoft. All rights reserved.
 *  Licensed under the MIT license. See LICENSE file in the project root for full license information.
 */
'use strict'
const c = require('../config')
const Mqtt = require('azure-iot-device-mqtt').Mqtt
const DeviceClient = require('azure-iot-device').Client
const Message = require('azure-iot-device').Message
const YOUR_CONNECTION_STRING = c.CONFIGURATIONS.IOTHUB.PREVIEW
const YOUR_INTERVAL = 1000
const client = DeviceClient.fromConnectionString(YOUR_CONNECTION_STRING, Mqtt)

let x = 0
while (x < 100) {
    //800 or 900-property.json
    let js = require(`${c.PROJECT_PATH}/tsi_column_tester/_out/800-property.json`)[0]
   //test or test2
    js = Object.assign({"deviceId": "test"}, js)
    const message = new Message(JSON.stringify(js))

    client.sendEvent(message, err => {
        if (err) console.error(`send error: ${err.toString()}`)
        else console.log('message sent')
    })
    x++
    setTimeout(() => {
    }, YOUR_INTERVAL)
}
  1. 1

  2. 2

So, it look likes the original cap has been raised, unfortunately, I do not yet have a maximum cap but I hope the information above answers the immediate question and will aid you in your testing. I will continue to sync with the PM and engineering team's for a specific maximum cap.

Thanks!

Thanks for getting back to me @KingdomOfEnds !

I'll operate under the assumption of a "large enough" cap, and if I run into a limit I'll comment here.

@xtellurian - I've run a couple more tests using 5000 rows. In all of those cases, the maximum number of rows that appears in TSI Preview is 900 rows (including the uuid). I've also reached out to engineering to confirm this as well. Once we have that confirmed we will update our documentation.

Thank you!

@xtellurian - Quick update for you - the new TSI Preview limit will be 1000 properties / rows. Thanks!

Can we close the thread? (We are currently in the process of updating the limits section, should be done very soon).

@deepakpalled Thanks for the heads up. Also, please notify here once completing the limits updates.
@xtellurian As mentioned, limits will be updated soon.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

spottedmahn picture spottedmahn  Â·  3Comments

monteledwards picture monteledwards  Â·  3Comments

spottedmahn picture spottedmahn  Â·  3Comments

AronT-TLV picture AronT-TLV  Â·  3Comments

JeffLoo-ong picture JeffLoo-ong  Â·  3Comments