Azure-docs: Azure Batch task scheduled with Data Factory to run a docker container

[ivana61]

I use a docker container image that already contains a python script which is in an Azure Container Registry and I want to schedule it to run through Azure Data Factory. I get this error: Task failed "Container-enabled compute node requires task container settings", but I have already specified the container type, image and registry when I created the pool. What else could this error refer to?

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: cb309d5a-11ab-d66e-41f5-55b446081df6
• Version Independent ID: aaff858d-796f-f0a5-2857-88ca6a1059e3
• Content: Create tasks to prepare jobs and complete jobs on compute nodes - Azure Batch
• Content Source: articles/batch/batch-job-prep-release.md
• Service: batch
• GitHub Login: @dlepow
• Microsoft Alias: danlep

[mimckitt]

Thanks for the question! We are investigating and will update you shortly.

[mimckitt]

@ivana61 where exactly are you getting this error? When creating the Job Schedule? Could you provide some screenshots for me?

[ivana61]

@MicahMcKittrick-MSFT I will explain the whole setup I have.
I have a python application in a docker container which is stored in Azure Container Registry and I use that one for the compute nodes in Azure Batch. This are the settings in my pool where I set up the container settings (image, registry):

The option Start Task is enabled and my command for the task is "cmd /c "docker run simple-flask-app" to run my application when I schedule a start of the node with Data factory.
Then in data factory I have a custom activity linked to the Azure Batch and in the command line I just have cmd /c dir.
I see the errror when i go to Jobs in Azure batch and I see the ADF job created there and in the tasks there I see this:

[mimckitt]

@ivana61 thanks for the extra details. Have you checked the following:

https://docs.microsoft.com/en-us/azure/batch/batch-docker-container-workloads#container-settings-for-the-task

It discusses setting up the container settings in Batch which might help fill in any gaps

[ivana61]

@MicahMcKittrick-MSFT When I schedule the task with data factory indeed the task container settings are missing. I can see that in the properties of the task created with ADF in Azure Batch:

How can I give the task container settings through ADF?

[ivana61]

@MicahMcKittrick-MSFT Can this be the problem actually? That ADF custom activities cannot be used to schedule a task on a container enabled pools?

[mimckitt]

@ivana61 Pradeep is one of our engineers. I reached out and confirmed with him on this.

So yes, the issues seems to be that this scenario is not currently possible/ supported :(

[mimckitt]

@ivana61 I will close this out for now. If you have additional questions just let me know and we can reopen and continue.

Also, if you want to provide feedback for Azure Products and features you think should be added you can add that feedback on
UserVoice

[BartVanDerVurst]

@micahtan Any update on this? This feature is still missing and creating a bottleneck on ADF?

[mimckitt]

@BartVanDerVurst no update on my end. Not sure if @ivana61 filed a feedback request on Uservoice for this as it was a feature request

[OscarGarciaPeinado]

@micahtan Any update on this? Exists any other solution to run a docker from DataFactory?
Sounds useless the fact that cannot launch a docker from the orquestation module...

[syedhassaanahmed]

+1
@micahtan Why is this issue closed? Customers should be able to use ADF to orchestrate Container Tasks in Azure Batch

[SidGrundfos]

I have run to the same problem, we need this feature. Any updates please?

[guderkar]

You can run Azure Container Instances from Data Factory.

• option one: Use WebHook Activity and Http triggered Logic App to orchestrate ACI and then POST to DataFactory callBackUri.
• option two: Use Azure Functions to run the ACI. I've made python based FunctionApp with tutorial on how to run it https://github.com/guderkar/ACIProxy