Azure-docs: Deployment of ARM Template to existing Production Data Factory

[bdforbes]

When I deploy an ARM Template to an existing Production Data Factory (through "Import ARM Template") as part of a continuous improvement process, the resources in the prod DF that were removed or renamed in development are still there after deployment. Should the entire Data Factory or all pre-existing resources be deleted first before deploying? Otherwise it will eventually be littered with resources that are no longer present in the development environment.

Alternatively, would it make sense to link dev, test and prod data factories the the Git repository, reserving the master branch for deployment to test and prod, and a "develop" branch and sub-branches for development? I haven't seen that workflow mentioned anywhere, but it sounds simpler than dealing with ARM Templates.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 1e22e4d6-366f-fbf8-e141-8905ca7ab7ae
• Version Independent ID: 5be2acf2-66e7-cadf-6666-5a85a65b9771
• Content: Continuous integration and deployment in Azure Data Factory
• Content Source: articles/data-factory/continuous-integration-deployment.md
• Service: data-factory
• GitHub Login: @douglaslMS
• Microsoft Alias: douglasl

[vasivara-MSFT]

@bdforbes This question is more suited in our forum. In github, we handle only doc issues like doc-enahncement, doc-bugs or any issues related to docs. This is more like product question, Can you please try pasting this in our MSDN forum.

[vasivara-MSFT]

@bdforbes We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.

[bdforbes]

Hi @vasivara-MSFT, it is somewhat of a docs question as the docs mention deployment to prod but do not clearly explain or walk through how to handle iterative development in the way I described, even though the title of the page is Continuous Integration.

[douglaslMS]

@bdforbes FYI here is some information from a member of the Data Factory team:

Deleting all the resources in the data factory means the customer will lose all the monitoring information, since we have to delete and recreate the factory, so that is not a viable option.

Another viable option is to run a script during CI/CD to delete the renamed/deleted resource. The documentation already contains a sample script for this.

[bdforbes]

Thanks Douglas, I will review the page again and look into that option. In principle such a script could be generated from the Git repo automatically so that sounds like a plausible option.

[douglaslMS]

@bdforbes After you take another look at this, I would appreciate your suggestions about what specific points we need to add to the doc, or to clarify. Thank you!

[bdforbes]

Happy to do so Douglas.

On Fri., 31 Aug. 2018, 02:28 Douglas Laudenschlager, <
[email protected]> wrote:

@bdforbes https://github.com/bdforbes After you take another look at
this, I would appreciate your suggestions about what specific points we
need to add to the doc, or to clarify. Thank you!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/14190#issuecomment-417381811,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AO-Gg4WebRaR9X9djhN_kA1FkntDWnUHks5uWBLJgaJpZM4WRWS1
.

[douglaslMS]

@bdforbes FYI I'll be away for two weeks so it will be a while before I comment again.

[bdforbes]

No worries Douglas, I'll also not be able to try this out straight away.

On Sat., 1 Sep. 2018, 01:47 Douglas Laudenschlager, <
[email protected]> wrote:

@bdforbes https://github.com/bdforbes FYI I'll be away for two weeks so
it will be a while before I comment again.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/14190#issuecomment-417706169,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AO-GgxBI4UXMxNJ88qZOaCqbOO-Mb4w_ks5uWVqZgaJpZM4WRWS1
.

[mariojvargas]

I'm also interested in seeing documentation on implementing the CI/CD process that @bdforbes described, as I have a similar scenario. We target multiple environments (dev, QA and production), each having a separate branch (production = master). We want to be able to pull artifacts from the correct environment but the only option in the documentation that I've seen for setting up the CD pipeline is to pull from the adf_publish branch, which ADF handles automatically.

[douglaslMS]

@mariojvargas You can now configure the publishing branch in a limited manner: Configure publishing settings.

[douglaslMS]

@bdforbes We've provided a suggestion with regard to your first paragraph at the beginning of this issue, and entered a backlog item to work with the Data Factory team to flesh out the CI/CD guidelines over time. I'm going to go ahead and request that this issue be closed. You can still add comments later with your suggestions to improve the documentation. Thank you!

[douglaslMS]

please-close