Azure-storage-azcopy: panic: mkdir : The system cannot find the path specified

[ramteju]

Which version of the AzCopy was used?

Azcopy version 10.6.0

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What command did you run?

azcopy cp "{{blob URL}}" "D:\home\site\wwwroot\DownloadFilesrequest-fastqfiles\RA3810AB\RA3810AB1DO1_1_S1_L001_R1_001.fastq.gz"

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

What problem was encountered?

while running above command we are getting below message when we are running in azure function app but when we are trying from local its working fine as expected

panic: mkdir : The system cannot find the path specified. goroutine 1 [running]: github.com/Azure/azure-storage-azcopy/common.PanicIfErr(...) /home/vsts/work/1/s/common/lifecyleMgr.go:553 main.main() /home/vsts/work/1/s/main.go:55 +0x562

How can we reproduce the problem in the simplest way?

To reproduce this issue we need to call the above command in azure functions or any app service in azure

Have you found a mitigation/solution?

[mohsha-msft]

Hey @ramteju ,

Thanks for reaching out!
I see you are using windows and if I understand correctly you want to download a blob from Azure storage account to local directory. The problem here is that you are specifying the destination incorrectly. When you specify a destination in double quotes (""), a backslash ("\") in the destination path works as an escape sequence. So you might have to add two backslashes ("\\") instead of single backslash ("\").

For instance,
azcopy cp "{{blob URL}}" "D:\\home\\site\\wwwroot\\DownloadFiles\\request-fastqfiles\\RA3810AB\\RA3810AB1DO1_1_S1_L001_R1_001.fastq.gz"

If above explanation doesn't resolve your issue or if you want more clarity, please feel free to ping here. I'll be more than happy to help you out 😄

[ramteju]

hi mohsha,

even after updating the command also same message we are getting
updated command:
azcopy cp "{{blob URL}}" "D:\home\site\wwwroot\DownloadFiles\request-fastqfiles\SS8379AB\SS8379AB1DO1_5_S1_L001_R1_001.fastq.gz"

Error We got:
panic: mkdir : The system cannot find the path specified.

goroutine 1 [running]:
github.com/Azure/azure-storage-azcopy/common.PanicIfErr(...)
/home/vsts/work/1/s/common/lifecyleMgr.go:482
main.main()
/home/vsts/work/1/s/main.go:55 +0x562

[mohsha-msft]

Hey @ramteju

It should not be
azcopy cp "{{blob URL}}" "D:\home\site\wwwroot\DownloadFiles\request-fastqfiles\SS8379AB\SS8379AB1DO1_5_S1_L001_R1_001.fastq.gz"

but

azcopy cp "{{blob URL}}" "D:\\home\\site\wwwroot\\DownloadFiles\\request-fastqfiles\\SS8379AB\\SS8379AB1DO1_5_S1_L001_R1_001.fastq.gz"

Notice that I am using double back slash (\\) instead of single one to specify the destination path.

Also before using AzCopy, please validate whether the path is correct. Simply go to the explorer and paste D:\home\site\wwwroot\DownloadFiles\request-fastqfiles\SS8379AB\ in it and see if the file SS8379AB1DO1_5_S1_L001_R1_001.fastq.gz exists or not.

You can forgo the filename as well. Simply specifying the folder will do too. azcopy cp "{{blob URL}}" "D:\\home\\site\wwwroot\\DownloadFiles\\request-fastqfiles\\SS8379ABz". This will download the blob in the specified folder.

Please reach out for more information.

[ramteju]

Hi @mohsha-msft,

Below is updated command i am calling to download, we are verifying folder also but folder is there but file failed to download.
Can you please help me to solve the issue

Running command: azcopy cp "{{blob url}}" "D:\\home\\site\wwwroot\\DownloadFiles\\request-fastqfiles\\RA8395AB\\RA8395AB10DO10_10_S1_L001_R1_001.fastq.gz"
Folder: D:\home\site\wwwroot\DownloadFilesrequest-fastqfiles\RA8395AB Exists?: True
Output: at 9/16/2020 11:36:53 AM
Error: panic: mkdir : The system cannot find the path specified.

goroutine 1 [running]:
github.com/Azure/azure-storage-azcopy/common.PanicIfErr(...)
/home/vsts/work/1/s/common/lifecyleMgr.go:482
main.main()
/home/vsts/work/1/s/main.go:55 +0x562

[mohsha-msft]

Hey @ramteju ,

Please share the AzCopy logs of the failed job with me so that I can look more into your issue.

[ramteju]

Hi @mohsha-msft ,

log files not created for the above command only we got response as
panic: mkdir : The system cannot find the path specified.

goroutine 1 [running]:
github.com/Azure/azure-storage-azcopy/common.PanicIfErr(...)
/home/vsts/work/1/s/common/lifecyleMgr.go:482
main.main()
/home/vsts/work/1/s/main.go:55 +0x562

[exglade]

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

AzCopy version 10.6.0

Which platform are you using? (ex: Windows, Mac, Linux)

Windows on Azure Function for Powershell

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

./azcopy.exe sync ""https://srcstgacc.blob.core.windows.net/myfolder"" ""https://dststgacc.blob.core.windows.net/backup-container/myfolder"" --recursive --delete-destination=true

What problem was encountered?

ERROR: panic: mkdir : The system cannot find the path specified.

How can we reproduce the problem in the simplest way?

Run the command above in Azure function.

Note: I've yet to reproduce the problem consistently. The command sometimes works, sometimes it doesn't.

Have you found a mitigation/solution?

None.

[exglade]

I have the same error but with a different scenario.

I wrote a durable function in PowerShell for Azure function. I want to sync all containers from a storage account to a virtual directory in another storage account.

Example:
https://srcstgacc.blob.core.windows.net/myfolder -> https://dststgacc.blob.core.windows.net/backup-container/myfolder

The durable function flow:

1. [Http Trigger] Start orchestrator
2. [Orchestrator] Prepare the variables (with activity to be deterministic): source & destination uri with SAS token, source container names
3. [Orchestrator] Start an activity for syncing.
4. [Activity] Sync source to destination.
5. [Orchestrator] Await & repeat step 3-4 for every container.

Sometimes, some sync activities failed with same error; sometimes, all sync activities failed with same error. I've yet to discover the root cause to consistently reproduce the error.

Additional notes:

• Regarding AzCopy concurrency, I've tried fan-in/fan-out and function chaining pattern. But always 1 execution at a time.
• Source storage account is a live environment.
• There are some successful syncs.

I hope this provides some additional insight to the issue. I'm going to try to perform the script locally to see if the issue would occur.

[exglade]

As mentioned in the log provided by @ramteju, the error was thrown at main.go:line 55.

AzCopy failed to create the folder on the default log path in Azure Function. The default app path is %UserProfile%/.azcopy according to the documentation. I'm not sure how Azure Function handles its file system and permission for %UserProfile%.

Hence, the workaround is setting the location of plans and log files for AzCopy in Azure Function. I've set it to %TEMP%/.azcopy. I hope this solves the problem.

https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-configure#change-the-location-of-the-plan-and-log-files

A reminder that if you are running multiple instances, you have to set again for every instance. I made that mistake when invoking multiple durable function activities.

[LuwkasLima]

In case anyone else finds this issue and for some reason can't make @exglade's solution work. I had to use $env:AZCOPY_JOB_PLAN_LOCATION = $env:temp+'\.azcopy' instead of the %temp% to make it work in Azure functions.

[rahimek]

@LuwkasLima
Hi, I have exactly the same problem as @exglade but unfortunately your solution is not working for me.

I put $env:AZCOPY_JOB_PLAN_LOCATION = $env:temp+'\.azcopy' in my function but I still got the error:

ERROR: panic: mkdir : The system cannot find the path specified. Exception : Type : System.Management.Automation.RemoteException ErrorRecord : Exception : Type : System.Management.Automation.ParentContainsErrorRecordException Message : panic: mkdir : The system cannot find the path specified. HResult : -2146233087 CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordException FullyQualifiedErrorId : RuntimeException Message : panic: mkdir : The system cannot find the path specified. HResult : -2146233087 TargetObject : panic: mkdir : The system cannot find the path specified. CategoryInfo : NotSpecified: (panic: mkdir : The …the path specified.:String) [], RemoteException FullyQualifiedErrorId : NativeCommandError InvocationInfo : MyCommand : AzCopy.exe

I noticed the problem started after I restarted FunctionApp. Any ideas?