Azure-devops-docs: Deployment Groups in YAML

[principaltyke]

Though this article was written over a year ago, is there any update as to when the support of Deployment Groups in YAML will be available?

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 45bcaa19-4ee9-55c5-9b1c-92f8d0f2dfa7
• Version Independent ID: d9424c8a-571d-0f48-e8f7-4bb7a9544efb
• Content: Deployment group jobs in Azure Pipelines Build and Release - Azure Pipelines
• Content Source: docs/pipelines/process/deployment-group-phases.md
• Product: devops
• Technology: devops-cicd
• GitHub Login: @alewis
• Microsoft Alias: alewis

[joesnyder]

I too am interested in an update. It would be very helpful to use Deployment Groups in our YAML pipelines.

[WilliamAntonRohm]

@principaltyke Thank you for your feedback. The product teams maintain a roadmap of upcoming features here.

[brooknovak]

How is this closed? The link provided to the "roadmap of upcoming features" doesnt have anything about Deployment Groups in YAML.
We need to deploy on prem via Deployment Groups, but we really want to deploy as part of our yaml pipeline (multi-stage/multi-job). Currently we cannot do this and have to build all our on-prem deployments with the GUI (which creates a maintenance nightmare for us)

[joesnyder]

How is this closed? The link provided to the "roadmap of upcoming features" doesnt have anything about Deployment Groups in YAML.
We need to deploy on prem via Deployment Groups, but we really want to deploy as part of our yaml pipeline (multi-stage/multi-job). Currently we cannot do this and have to build all our on-prem deployments with the GUI (which creates a maintenance nightmare for us)

@WilliamAntonRohm I am curious as well as to why this was closed. There is nothing on the roadmap that mentions this. If this isn't on the roadmap then simply say that. Just don't point us to a document that literally has no mention of Deployment Groups in context of what we are referring to.

[stevenvolckaert]

@WilliamAntonRohm I second @joesnyder's point. Our team needs to deploy to on-premises VMs that are grouped in a deployment group.

[winstonhenke-work]

Another +1 for wanting a update here

[hoffmanntravis]

Bumping this as this is a critical feature from my perspective as well. Managing this through the UI is a big strike against Azure DevOps when it comes to moving towards declarative deploys, and using the tool for more than just build management.

[WilliamAntonRohm]

@principaltyke, @joesnyder, @brooknovak, @stevenvolckaert, @winstonhenke-work, @pirotrav -- thank you for your interest.

@steved0x -- please look into this issue.

[Baylife101]

Bumping this aswell.
We have put all our multipipeling innovation on standby until this feature is implemented.
Would be nice if we could get some feedback if this is even going to be solved and possibly an ETA.

[TroySchmidt]

I was shocked that once mulstiage, approvals, and environments that when I was deep into developing our multistage YAML that this was missing.

[TroySchmidt]

I have found a workaround for this. What I did was add the computers that I needed to deploy to as normal agents and _not_ part of a deployment group. The other secret is that deployment by default downloads all artifacts so you have to use the special syntax there for it to only download specific file(s). This deployed the IIS app to a provisioned instance in our AWS cloud. I only have one machine in the pool. So not sure specifically how it would respond to multiple machines hooked up to the same pool if it deploys to them all or just randomly one.

- stage: DeployDev
  displayName: 'Deploy DEV'
  dependsOn: 'Bundle_Default_Data'
  condition: succeeded()
  jobs:
  - deployment: Web_Api
    displayName: 'Web Api'
    pool: 'DEV-WEB'
    environment: 'dev-web'
    strategy:
      runOnce:
        deploy:
          steps:
          - download: current
            artifact: websites
          - task: IISWebAppDeploymentOnMachineGroup@0
            displayName: 'Deploy IIS Website/App: WebApi'
            inputs:
              WebSiteName: WebApi
              Package: '$(Agent.BuildDirectory)/websites/WebApi.zip'
              TakeAppOfflineFlag: true

[Baylife101]

I have found a workaround for this. What I did was add the computers that I needed to deploy to as normal agents and _not_ part of a deployment group. The other secret is that deployment by default downloads all artifacts so you have to use the special syntax there for it to only download specific file(s). This deployed the IIS app to a provisioned instance in our AWS cloud. I only have one machine in the pool. So not sure specifically how it would respond to multiple machines hooked up to the same pool if it deploys to them all or just randomly one.

- stage: DeployDev
  displayName: 'Deploy DEV'
  dependsOn: 'Bundle_Default_Data'
  condition: succeeded()
  jobs:
  - deployment: Web_Api
    displayName: 'Web Api'
    pool: 'DEV-WEB'
    environment: 'dev-web'
    strategy:
      runOnce:
        deploy:
          steps:
          - download: current
            artifact: websites
          - task: IISWebAppDeploymentOnMachineGroup@0
            displayName: 'Deploy IIS Website/App: WebApi'
            inputs:
              WebSiteName: WebApi
              Package: '$(Agent.BuildDirectory)/websites/WebApi.zip'
              TakeAppOfflineFlag: true

I was working on something similar, but as far as i can tell there's 3 major differences.

1. It only deploys to the first available target, thus not really a group.
2. People in your org might fuck up and use the new agent for building as well thus resulting in source code being available on the production servers.
3. Since it's a normal agent it will consume a "parallel job" if multiple project happens to be deployed overlapping each other. This can end up being a pricey solution, since you end up paying for consuming additional parallel jobs.

Honestly Microsoft not even responding to the solution, while almost pushing users to use yaml almost seems intentional to have companies pay more for parallel jobs.

[hoffmanntravis]

Agreed with @TroySchmidt and @Baylife101 . Those are the exact circumstances we found ourselves in. Honestly at first I thought I just didn't understand the tool well enough and hadn't found the appropriate YAML keyword / function. Eventually we looked at agent pools but the solution was not at all elegant and seemed about the equivalent of using a wrench to pound a screw into the wall; could be done but ultimately the wrong tool for the job and the outcome is subpar. The potential for source code leaking into prod is also an interesting potential side effect I hadn't considered.

I'm also not entirely clear how Environments fit together in relation to deployment groups, stages, users, and approvals. The approvals feature on environments seems really nice, but if I can't assign a set of machines to an environment it seems limited in it's utility. I guess you can protect the YAML file for pull requests limited to allowed users and consume the Environment tag that has the associated approvals in the YAML file. Seems a bit convoluted. But this is generally a separate point that is related but not the core issue with this thread, so take it for what it's worth. Getting deployment groups in place for YAML is for sure step number 1.

[TroySchmidt]

@hoffmanntravis For approvals and the environment you are just using an empty environment. Once the environment is created then it doesn't need resources to control approval workflow. What you do is add the 'checks' to the environment for manual approval. Once you include that environment in the deployment stage then it is there. So there doesn't have to be any resources assigned to the environment for it to work.

There is also a YAML option to ensure that no source code is downloaded. I didn't set that option in my YAML sample above, but no source code was downloaded to the deployment machine.

[Baylife101]

@hoffmanntravis For approvals and the environment you are just using an empty environment. Once the environment is created then it doesn't need resources to control approval workflow. What you do is add the 'checks' to the environment for manual approval. Once you include that environment in the deployment stage then it is there. So there doesn't have to be any resources assigned to the environment for it to work.

There is also a YAML option to ensure that no source code is downloaded. I didn't set that option in my YAML sample above, but no source code was downloaded to the deployment machine.

Regardless of how it works, it still does not make sense. I would expect a deployment group to contain x-environments.

My point regarding the source code is, that since it's a regular agent in a regular agent pool a mixup might happen and somebody might end up using the agent you created for deployment to build thus downloading source code to a production server.

[TroySchmidt]

Nope it sure doesn't make sense. This is only a workaround \ stopgap measure until Azure comes out with full fledged deployment groups. And the documentation exists, but it is spread out and not intuitive process at all.

[hoffmanntravis]

@TroySchmidt Yeah, we did set up a POC with that, and it does work. My problem with that is your protecting the environment tag essentially, not the resource / machine itself. In Octopus Deploy for example, there is the notion of an agent, and that agent belongs to an environment which is protected. Users that are not authorized to the environment cannot see or interact with that agent in any meaningful way offering a sensible layer of protection. Likewise, all API calls to the backend (which is what the UI also consumes) utilize those same roles and protections.

With AzD, if I a user has access to the YAML file they can just go remove the environment tag while still targeting the agent and suddenly have the ability to deploy to the theoretically locked environment and make changes etc. If the machines belonged to the environment this would not be possible, and limited privilege users could still make changes to the YAML file to update the deploy process as needed. From what I can tell deployment groups don't belong to an environment either, so I think this problem would persist even if they are added to YAML unless this is taken into consideration.

Does that make sense in the context of your reply? I know you're just trying to help out and this is not the principle problem on this issue anyway, so no worries if I'm missing the plot here or something. Basically a variation of what @Baylife101 is getting at as well I think.

[TroySchmidt]

Yeah multistage YAML showing everything in one succinct overview is amazing. It is easier to digest than project managers (non developers) following the release to the build to figure out which one did what and what work items were done. I couldn't even find an example of deploying to a machine really this was all cobbled together amongst threads and documentation.

The use case for deployments, approvals, and environments is extremely narrow currently. Expanding it by exploiting agents is terrible as a final solution. Hopefully they will be adding machines (currently deployment groups) as full fledged resources of the environment. So then to deploy to them you have to use the environment syntax environment.resource. That would eliminate them circumventing approval checks.

[zarochi]

Just chiming in to indicate that we need this too. I'm trying to deploy to both of my load balanced servers, and it seems deployment groups are the way to do it. Like Troy I didn't find this until my pipeline was basically done, and it is disappointing to see that we have to use agent pools for this.

I guess I'll have to make another server, make that my agent, then use Powershell to do all the goodness this is supposed to do for me.

[ljfraney]

The only mention I can find of it is in Gopinath Chigakkagari's blog post announcing Multi-stage YAML pipelines back in early May. In the section "A look at what's next" he says...

We’re adding support for additional resource types in environments, so you can get going quickly with virtual machines (through deployment groups) and Azure Web Apps.

In Gloridel Morales's blog post on July 31st, the only mention of enhancements to Deployment jobs is where she states that they are going to be adding more deployment strategies.

A deployment job is a special type of job that is used to deploy your app to an environment. We will enhance the strategies supported in deployment jobs to enable rolling, canary and blue-green deployments.

These strategies sound very interesting, but many of us can't even use the runOnce strategy until there is support for deployment groups, or something that behaves similarly. We are currently in the process of moving over 100 applications to Azure Pipelines with a goal of completing this by the end of the year. Although I'm confident Microsoft is working on this, it would be nice to know whether we should abandon multi-stage pipelines in favor of separate build and release pipelines, or if we can expect to see something soon. All of this being in preview, I'm not complaining, just hoping for some guidance.

[TroySchmidt]

We have since dialed back our adoption of multistage YAML for deployments. We are only going to use in a build validation workflow. Basically once the build validation is successful it will immediately deploy the code using the workaround.
Microsoft has pushed preview functionality for associating work items directly to releases for the normal Pipeline > Releases. That functionality combined with the ability to edit releases (for unforeseen issues to adjust parameters etc) multistage YAML deployments just don't make sense from a functionality standpoint now.

[ljfraney]

@TroySchmidt I set up a multi-stage pipeline to build and publish a library to our NuGet feed with stages that include build > alpha > beta > final. It works great for this since there are no deployment groups involved. I can probably think of other cases where multi-stage pipelines could work for us, but the bulk of our pipelines are building and deploying web applications. We will probably go forward with creating separate Release pipelines for these until more features are implemented.

[dotnetcanuck]

@steved0x Is there any update you can provide on this issue? Thanks!

[mniak]

Bumping this, as we also would appreciate deployment groups support soon!

[ggretz]

a statement from Microsoft would be good... we are wainting for this too ;)

[felipegsantos]

Please, implement this feature!

[stevenvolckaert]

@TroySchmidt I set up a multi-stage pipeline to build and publish a library to our NuGet feed with stages that include build > alpha > beta > final.

That's interesting, if you could share some more details of your setup that would be awesome!

For example, let's assume the version of a library (set by the Version property in the *.csproj file) is set to 1.2.0.

How do you make sure that the Alpha stage is adding a unique, sequential number to the end of the 1.2.0-alpha informational version string? If possible, I would prefer the releases to be numbered like this: 1.2.0-alpha01, 1.2.0-alpha02, and so on for the Alpha stage, 1.2.0-beta01, 1.2.0-beta02, etc. for the Beta stage.

Perhaps you achieve this by adding the formatted date ($(Date:yyyMMdd)$Rev:.r))? Or perhaps $(Build.BuildId)?

[TroySchmidt]

We use Git and GitVersion task for versioning.

On Wed, Sep 25, 2019, 8:14 AM Steven Volckaert [e-mail]
wrote:

@TroySchmidt https://github.com/TroySchmidt I set up a multi-stage
pipeline to build and publish a library to our NuGet feed with stages that
include build > alpha > beta > final.

That's interesting, if you could share some more details of your setup
that would be awesome!

For example, let's assume the version of a library (set by the Version
property in the *.csproj file) is set to 1.2.0.

How do you make sure that the Alpha stage is adding a unique, sequential
number to the end of the 1.2.0-alpha informational version string? If
possible, I would prefer the releases to be numbered like this:
1.2.0-alpha01, 1.2.0-alpha02, and so on for the Alpha stage, 1.2.0-beta01,
1.2.0-beta02, etc. for the Beta stage.

Perhaps you achieve this by adding the formatted date (
$(Date:yyyMMdd)$Rev:.r))? Or perhaps $(Build.BuildId)?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/vsts-docs/issues/4486?email_source=notifications&email_token=AALXG3OGVNOV5JGOFALHYN3QLNIZJA5CNFSM4HXGV7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7RVOVA#issuecomment-534992724,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALXG3I63HPMVAS3TMORSZTQLNIZJANCNFSM4HXGV7UA
.

[stevenvolckaert]

Thanks for the info @troyschmidt, I'll check it out.

[dotnetcanuck]

We did something similar...we set the first three "digits" (i.e. 1.0.0) in a variable. Then depending on the branch name, we add a prerelease tag (alpha for feature, beta for develop, RC for release or hotfix, or nothing for master).

Finally, the build ID is appended to the very end. The result would be something like 1.0.0-beta.6384, or 1.0.0.6424.

[ljfraney]

@stevenvolckaert I require the developer to modify the <VersionPrefix>1.2.3</VersionPrefix> in the .csproj file based on whether they are making breaking changes (major), adding new features (minor), or fixing bugs (patch). I document this in my readme files. In my mind, only the developer knows the answer to this. My stages look similar to this...

- stage: Alpha
  jobs:
  - deployment: PublishAlpha
    displayName: 'Publish Alpha NuGet Package'
    pool: default
    environment: NuGet-alpha
    strategy:
      runOnce:
        deploy:
          steps:
          - task: PowerShell@2
            displayName: 'Publish Alpha Package'
            inputs:
              targetType: 'filePath'
              filePath: '$(deployScriptsArtifactName)\nuget_alpha.ps1'
              workingDirectory: '$(Pipeline.Workspace)\$(artifactName)'

The nuget_alpha.ps1 file just looks like this...

$packageSuffix = "alpha"
Write-Output "Running $PSScriptRoot\nuget.ps1..."
$deployScript = "$PSScriptRoot\nuget.ps1"
&$deployScript -packageSuffix $packageSuffix

The nuget.ps1 file looks like this...

param([String]$packageSuffix)
$configuration = "Release"
$csprojLocation = ".\MyNugetProject.csproj"
$nugetSource = "MyNuGetFeedSource"

# Edit the .csproj file to make sure versioning can be done correctly
[xml]$csproj = [xml](Get-Content $csProjLocation)

# Ensure the <VersionPrefix> tag exists.
$versionPrefixNode = $csproj.SelectSingleNode("//Project/PropertyGroup/VersionPrefix")
if ($null -eq $versionPrefixNode) {
    throw "Edit the .csproj file for $csProjLocation and add a <VersionPrefix> tag."
}

# Remove the <Version> tag if it exists. It will keep the --version-suffix switch from working.
$versionNode = $csProj.SelectSingleNode("//Project/PropertyGroup/Version")
if ($null -eq $versionNode) {
    $versionNode = $csProj.SelectSingleNode("//Project/PropertyGroup/version")
}
if ($null -ne $versionNode) {
    $versionNode.ParentNode.RemoveChild($versionNode)
}

$csproj.Save($csProjLocation)

# Package the .nupkg file
Write-Output "Packaging NuGet package with package suffix ""$packageSuffix""..."
if ($packageSuffix) {
    dotnet pack --configuration $configuration --version-suffix $packageSuffix
}
else {
    dotnet pack --configuration $configuration
}

# Push the package to Azure DevOps Artifacts
$releaseFolder = "bin\Release"
Write-Output "Publishing NuGet packages in $releaseFolder to Azure DevOps."
$publishOutput = dotnet nuget push "$releaseFolder\*.nupkg" --api-key AzureDevOps --source $nugetSource | Out-String
if ($LASTEXITCODE -eq 1) {
    if ($publishOutput.Contains("Conflict - The feed already contains")) {
        Write-Output "The NuGet feed already contains this version of the package."
    }
    else {
        Write-Error "An unexpected error occurred pushing the NuGet package to the feed: $publishOutput"
        Exit 1
    }
}
else {
    Write-Output "The NuGet package was successfully pushed to the feed."
}

Exit 0

I don't have the ability this way to do -beta1, -beta2, etc. I could see how you would want to. But, I'm not too concerned with burning patch version numbers either since these packages aren't on a public feed.

[stevenvolckaert]

Thanks a million @ljfraney for sharing this!!

[ljfraney]

@stevenvolckaert No problem! I ran in to an issue this morning where I realized I need to be clearing out my bin folder. I'm publishing $releaseFolder\*.nupkg, and the alpha .nupkg was still hanging out in the folder with the beta. It was trying to re-publish the alpha and failing before it got to the beta stage. Should be a simple fix, though.

[guythomasdavis]

Why isn't this already available?

As with all the other commenters can you please make this a priority. While looking at the roadmap I don't understand why this isn't even listed. And when it does get listed should be moved to the top of the heap.

We are just getting deployment groups setup for a .net migration to AWS and without this feature it certainly puts a lot of stress on our teams to setup the classic deployment pipelines AND THEN have to redo the entire pipeline once this is released. In the interim I will be pushing the CIO to choose either Ansible or Terraform. Frankly if you could just put out an estimated release that MAY keep me in the Azure DevOps camp.

The silence is deafening!!!

[dotnetcanuck]

We worked around the issue by setting up an agent pool with our deployment targets.

In the deployment pipeline, we set up a stage containing a job for each target in the pool. And each job had a demand to ensure it would be run on the correct agent.

[zarochi]

@dotnetcanuck that's a clever solution. I didn't realize you could target a machine in the pool. I was going to just make different pools and have one machine per pool then do the same thing and have a stage for each of the pools. I like your way much better. Powershell remoting has been doing the trick for me so far, but it's going to be a bit of a pain to maintain.

[BlairMcc]

Multistage pipelines have been announced for GA in the Q4 Roadmap :: https://devblogs.microsoft.com/devops/azure-devops-roadmap-update-for-2019-q4/

Has there been any further news on deployment groups?

It almost feels like MS is pushing us towards installing agent pools on our deployment targets and having a convoluted workaround as per above

[cawoodm]

For anyone who wishes to build and deploy via YAML on multiple servers the following may be of use and save some hair-pulling:

# Deploy Apache Configuration on all DEV servers
#  This pipeline is triggered when code is committed to develop

trigger:
- develop

jobs:
- job: deploy_apache_dev
  displayName: Deploy Apache on all DEV Servers
  strategy:
    matrix:
      Dev1:
        AGENT_NAME: 'server1'
      Dev2:
        AGENT_NAME: 'server2'
  pool:
    name: MYAGENTPOOL
    demands:
      - agent.name -equals $(AGENT_NAME)
  steps:
  - powershell: .\build.ps1
    displayName: Build (build.ps1)
  - powershell: .\tests\mytests.ps1 -Verbose
    displayName: Tests (mytests.ps1)
  - task: PublishTestResults@2
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'junit.xml'
  - powershell: .\deploy.ps1
    displayName: Deployment (deploy.ps1)

The issue is that only agent pools support YAML and agent pools don't directly allow running a job on multiple agents. This can be overcome by using the matrix strategy above.

Also, I discovered you can't use deployment directive but have to use job for some strange reason...

[stevenvolckaert]

Thanks for sharing this, @cawoodm!

Would you know whether a machine can be registered in an agent pool and in a deployment group at the same time? I'd like to know if I can migrate pipeline by pipeline, or if I need to migrate all at once.

[dotnetcanuck]

@cawoodm Thanks for sharing that. Do you know if you can still use approval checks with the job directive?

Also, for everyone else in this thread, agent demands have stopped working for deployment jobs in the past couple of days. There is an open bug on Microsoft's developer community site: https://developercommunity.visualstudio.com/content/problem/793882/deployment-stage-is-ignoring-demands-on-pool.html

[winstonhenke]

@cawoodm, Thanks for sharing!

@dotnetcanuck, I think one of the downsides to this work around is you do loose those approval checks since you can't target an environment like you can in a deployment job

[dotnetcanuck]

@winstonhenke I figured...thanks for the reply!

[BlairMcc]

@dotnetcanuck, @winstonhenke, Assuming you can set up normal jobs to depend on deployment jobs could your stage start off with a more or less empty deployment job with an environment that is configured with checks?

To borrow from @cawoodm above

```jobs:

• deployment: deploymentJob
  displayName: Empty Deployment Job for Environment Checks
  environment: DEV.Apache
  strategy:
  runOnce:
  deploy:
  displayName: nothing
  steps:
  - powershell: .\helloworld.ps1
  displayName: Empty deploy step
• job: deploy_apache_dev
  displayName: Deploy Apache on all DEV Servers
  dependsOn: deploymentJob
  strategy:
  matrix:
  Dev1:
  AGENT_NAME: 'server1'
  Dev2:
  AGENT_NAME: 'server2'
  pool:
  name: MYAGENTPOOL
  demands:
  
  
  
  • agent.name -equals $(AGENT_NAME)
    
    steps:
    
    
    
    
    
    • powershell: .\build.ps1
      
      
      displayName: Build (build.ps1)
      
      
      [...]
      
      
      ```
    
    
    
  
  

The next issue is that there are no email or other notifications provided for environment checks, as highlighted in this issue - https://developercommunity.visualstudio.com/content/problem/682663/environment-checks-approval-doesnt-send-email-to-a.html

[winstonhenke]

@ArchSerpo clever solution. Still unfortunate we are forced into these wacky work arounds.

[dotnetcanuck]

@ArchSerpo @winstonhenke That is a clever workaround for that, thanks.

Speaking of wacky workarounds, here's one I just thought of for notifications - take the aforementioned idea of an "empty" job, except make the Powershell script make an SMTP request that sends an email to the approvers! 😝

Not that I would _actually_ do that, but I can totally see somebody out there doing that if they _really_ wanted approval notifications! 😂

[cawoodm]

Would you know whether a machine can be registered in an agent pool and in a deployment group at the same time? I'd like to know if I can migrate pipeline by pipeline, or if I need to migrate all at once.

We're doing all our CI/CD (build, test and deploy) with pipelines and agent pools. The situation with Azure DevOps at the moment is a bit of a mess and until we can do YAML releases we won't touch releases (and the associated deployment groups/pools).

So, you install build agents on all your hosts and put them in one or more agent pools. You then create one or more .yml pipeline definitions per repository/app. We've gone with one .yml file per environment so we have:

• deploy-dev.yml - triggered on develop branch: build, test and deploy on all DEV servers
• deploy-test.yml - triggered on master branch: build, test and deploy on all TEST servers
• deploy-prod.yml - no trigger - run manually: build, test and deploy on all PROD servers

[cawoodm]

@dotnetcanuck, I think one of the downsides to this work around is you do loose those approval checks since you can't target an environment like you can in a deployment job

By "approval checks" do you mean the approvers in a pull request? You don't loose those at all with the approach I posted. code => pull request => merge => trigger CI/CD pipeline works just fine. Approval rules are defined on the branch of your repo under "Branch Policy".

Can anyone explain the concept of "Environments" in DevOps without Kubernetes? I don't see how they apply to on-prem agents if we can only add Kubernetes resources.

[stevenvolckaert]

@cawoodm Thanks for sharing your workflow and insights! 🙏

[winstonhenke]

@cawoodm

My example below is using a AzureCLI@2 task to call a powershell script that creates various Azure resources (but could be any script really). This pipeline will both show up in the deployment history within environments & will go through the approval checks defined on the environment. So I do think environments can be useful without K8s. So when I responded about loosing approval checks from deployment jobs when switching to a regular job this was mainly what I meant.

- stage: test
  jobs:
  - deployment: my_test_job
    pool:
      vmImage: 'vs2017-win2016'
    environment: QA
    strategy:
        runOnce:
          deploy:
            steps:
            - task: AzureCLI@2
              inputs:
                azureSubscription: 'myAzureSub'
                scriptType: 'ps'
                scriptLocation: scriptPath
                scriptPath: '$(Pipeline.Workspace)/deployScript/deploy.ps1'
                scriptArguments: '-param1 value1'

[cawoodm]

So, if I want my deployments (pipeline jobs) to show up under "Environments" I have to make an empty deployment item in my YAML pipeline because I can't deploy with deployment because it doesn't support the matrix strategy and I therefore cannot target multiple agents so I have to use job which doesn't support environment.

@Microsoft, have your guys really thought this one through?

[nickfoster]

So I've spent the week building a custom Azure Pipelines task to manage the ServiceNow change ticket for a deployment only to find it I want to pass the ticket number between jobs in a stage I have to use YAML releases.
Now I'm discovering if I move from UI to YAML pipelines I can't use the deployment groups my applications are being deployed to? My apps are all on premises with multiple servers in deployment groups.

Can you please let us know of a rough timescale when the YAML pipelines will be able to use deployment groups, or when UI releases can pass output variables between jobs? (the ability to pass output variables between stages would be amazing, I'd like to raise one ticket and use the number on multiple deployment stages)

Cheers.

[chrisbuttacavoli]

We have everything registered in a deployment group, many with multiple servers. For us, YML is not production ready yet. Wasted a lot of time today trying to get this to work.

Does anyone have an ETA of when this will be done?

[Flaniga3]

Our automation platform depends on standardization and simplicity via pipelines as coded templates. Is this, clearly high-demand, feature going to get implemented anytime soon?

[WilliamAntonRohm]

Thanks for the suggestions! The product teams aren’t always watching this repository, so to make sure it gets in front of them, please submit your ideas here.

[jlaw84]

Looks like this idea was already submitted here. Might help if everyone upvotes it there to bring more attention to it.

[N-Usha]

Quick update: We now support adding “Virtual machines” as a resource within an Environment , which can be targeted for Rolling deployments in YAML pipelines. Feature is currently behind a feature flag and under Private Beta testing. We will roll this out to all the customers over the next few weeks.

Here is the release notes that we released for this feature this week, just in case you missed. Thanks again for all the patience and support :)

[NetTecture]

Yeah, except most of us will most likely NOT use VM deployment but file deployment. While I can se the benefit of this for certain scenarios, it is widely totally not usable in most enterprise environments.

In my case, i.e., I deploy to VM's - but those VM's are managed by a separate IT department with a month long turnaround time. And VM's are handling multiple applications that also have separate release cycles.

Unless "VM" for an environment_

• Allows tags to select subgroups
• Is a total misnomer and allows usage of physical computers "as" VM's

you successfully managed to totally NOT implement anything useful for most companies I seem to work with.

This is a large gap that seems to not be addressed or discussed by anyone having a month experience in deployments outside of cloud environments. Like many others it has successfully made sure that we can not tough yaml release pipelines and guess what - this is critical for us. We start evaluating alternative deployment stacks in 2020 because of this.

[Kadajski]

The link provided in the previous comment does mention you can install the deployment agent on the target servers. So I think you're getting caught up in the naming of "virtual machine". I suspect that naming was used as that is what azure provides and this is azure devops after all. The functionality does seem like it is the replacement for what this thread was originally created for. Instead of a deployment group you deploy to an environment which has machines/vms associated to it

[NetTecture]

I do not know because I only hve this link for now - not prone to seeing this in my features yet (but it should show up in a couple of weeks). if it is the case, it is a one of the most epic misnaming ever, and it would require me to use the same machine in multiple environments OR them adding tags to allow futher filtering - will see what the documentation says once it become available.

[gojanpaolo]

Quick update: We now support adding “Virtual machines” as a resource within an Environment , which can be targeted for Rolling deployments in YAML pipelines. Feature is currently behind a feature flag and under Private Beta testing. We will roll this out to all the customers over the next few weeks.

Here is the release notes that we released for this feature this week, just in case you missed. Thanks again for all the patience and support :)

@N-Usha can we use this feature if we have DevOps Server? We cannot use yaml because it doesn't support Deployment Groups and DevOps Server does not seem to support Environments (which is what most people point to as a workaround). :crying_cat_face:

[ryanbuening]

It appears when you add a Virtual machine resource to an environment, the Registration script is specifically for that project (--projectname 'Project Name').

We have dozens of projects in DevOps. Does this mean for each project, I need to get the server admin involved to run a script on all of our servers (dev/test/prod)? Or is there a way to have a generic/global environment that can be used on multiple projects?

[slawekww]

@cawoodm I could confirm that VM could be registered within Environment and Deployment group in the same time, only thing is to update registration script to use different destination path to install agent package (default folder is 'azagent'). It may be temporary solution once migrating from GUI release pipelines to YAML multi-stage pipelines.

[cawoodm]

The link provided in the previous comment does mention you can install the deployment agent on the target servers. So I think you're getting caught up in the naming of "virtual machine". I suspect that naming was used as that is what azure provides and this is azure devops after all. The functionality does seem like it is the replacement for what this thread was originally created for. Instead of a deployment group you deploy to an environment which has machines/vms associated to it

OK, so now I would have to have on each of my on-prem servers 3 identical agents:

• Build Agent (in an "Agent Pool")
• Deploy Agent (in a "Deployment Group")
• VM Agent (in an Environment)

2 was enough - I don't see why we even need 2. Pinging @Microsoft, 1 agent per machine please.

[slawekww]

@cawoodm I think each agent type is for different purpose: Build Agent for build CI pipeline, Deploy Agent for use in Release pipelines, VM agent to use for YAML pipelines.

I agree that it should be only 1 type for Deployment Agent which is supported by old Release pipeline and new multi-stage pipeline - that why it is feature request: https://developercommunity.visualstudio.com/idea/665340/use-deployment-groups-in-a-multi-stage-yaml-file.html

I've faced this issue once migrating from release pipelines to YAML pipeline - for temporary transition time, the same VMs were registered by 2 methods: in deployment group and VM in Environments.

[cawoodm]

What are these "YAML pipelines" of which you speak? I know of YAML Build Pipelines and non-YAML Release Pipelines. Are these YAML Release Pipelines? So our release now targets a VM instead of a Deployment Group?

[slawekww]

@cawoodm Yes, MS offers multi-stage YAML pipeline to replace old release pipeline: https://docs.microsoft.com/en-us/azure/devops/pipelines/release/define-multistage-release-process?view=azure-devops
And release would target VMs in Environment instead of Deployment Group however for transition process, the same VM must be registered in both to support both pipeline for temporary slot of time.

[arcotek-ltd]

@slawekww, can you confirm that link. It seems to be pointing to a document from 2018 that describes the old release pipeline. This "issue" is about deployment group support in yaml, so that link doesn't make sense. Maybe it's just me.

Thanks 🙄

Update:

I think this may be relevant:
https://devblogs.microsoft.com/premier-developer/azure-devops-pipelines-multi-stage-pipelines-and-yaml-for-continuous-delivery/

However, this comes down to Service Connections and AFAIK, unlike the old Deployment Group (and agent) traffic flows from AzDO to the service connection, meaning it has to have a public IP on it.

I think we now need to pressure @Microsoft to create a Service Connection to behave in the same way Deployment Groups work. i.e. agent > AzDO.

Getting MS to do this is going to be a challenge as they want people to deploy to Azure, however, for the small scale I'm working at, no amount of number crunching makes Azure remotely affordable for IaaS.

I'm thankful Octopus is on the list. They have multi-environment delivery well covered. Even IaC is possible. I might as well forget AzDO for CD. It's just not there yet.