Azure-devops-docs: Pipeline limitations for multi-stage scenarios are not documented

[matebarabas]

Pipeline limitations in general and more importantly, any limitations related to stages should be documented. It should be mentioned that having an extensive amount of stages and a complex flow, fan-in scenarios (when multiple stages are configured to be prerequisite of a consecutive stage) have limitations, i.e. the maximum number of lines in conditions field of the JSON structure backing the pipeline is limited to 2000. With about 20 stages, this can be easily hit.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 4266f72c-c774-0046-4593-d01eb775d3c3
• Version Independent ID: f20827aa-a6c5-96a8-5969-e576ffbc2e38
• Content: Stages in Azure Pipelines - Azure Pipelines
• Content Source: docs/pipelines/process/stages.md
• Product: devops
• Technology: devops-cicd
• GitHub Login: @vijayma
• Microsoft Alias: vijayma

[matebarabas]

@RoopeshNair , My customer – who faced these undocumented limitations of Azure pipelines – would like to know if there’s anything happening in this space. In the meantime, I think I could provide more details.

The customer has a Release pipeline, where they are using quite a lot of stages to run certain Pester tests in parallel (this is an automated testing framework to validate Azure Policies with Pester scripts).

After ALL the parallel stages finished running, there has to be an “integration” stage that waits for all the results (success or failure, doesn’t matter) of all the preceding stages. This is essentially a fan-in scenario.
When we tried to set this up, we received an error message that was basically stating that the “dependsOn” field of the “integration” stage can’t have more than 4000 characters. The exact error message is: “The property Conditions length cannot exceed 4000 characters.” I know that for the first sight, this seems to be a reasonably big number, but this particular solution operates with quite lengthy stage names, and having 40+ stages adds up quickly.

We managed to work around this limitation, by using “collector” stages that aren’t doing anything essentially (see below), but this solution is not too elegant.

The customer would like to know if this is a known/expected limitation and not a bug, and if so, why it is not documented. Additionally they would like to see all the pipeline limitations documented, that would help them avoid hitting any of these in the future.

Can you confirm that this limitation is known/expected and that efforts are going to be taken to document these in the near future?

Thanks a lot for your help in advance.

[RoopeshNair]

Thank you for sharing this. It does look like there is a cap on 'dependsOn'. will take a look at this. However, currently we are working on features/issues that affect broader set of customers. It will take a while before we can get to this. We know this is important for you. we will continue to track this.

[fleed]

In general, are the limitations documented? What if we have dozens, up to hundreds of different environments? Are there guidelines/best practices for scalability?

[aaroncaito]

We're likely going to run into a similar issue. We're going to have many stages ourselves. We'll be fanning out with ephemeral environments to perform some operations in parallel.

[PirateBread]

YAML pipeline performance in general is crap compared to the old build and deploy steps even when using the same build agents. Console information is much slower and cancelling jobs often takes forever.

[cxwtool]

This issue hasn't been updated in more than 180 days, so we've closed it. If you feel the issue is still relevant and needs fixed, please reopen it and we'll take another look. We appreciate your feedback and apologize for any inconvenience.