Azure-storage-azcopy: azcopy-node - Copy fails on `no SAS token or OAuth token is present and the resource is not public` when key is provided

[Ilanak]

Which version of the AzCopy was used?

@azure-tools/azcopy-node v1.0.0

Note: The version is visible when running AzCopy without any argument

Which platform are you using? (ex: Windows, Mac, Linux)

Node

What command did you run?

var client = new AzCopyClient();

const src = <IRemoteKeyLocation>{
    resourceUri: "https://srcaccount.blob.core.windows.net/",
    accountKey: "xxxxx",
    accountName: "srckey",
    path: "somepath"
}

const dst = <IRemoteKeyLocation>{
    resourceUri: "https://dststorage.blob.core.windows.net/",
    accountName: "dstacount",
    accountKey: "xxxxx",
    path: "copy-test"
}
var jobId = await client.copy(src, dst, {recursive: true});

What problem was encountered?

"failed to perform copy command due to error: no SAS token or OAuth token is present and the resource is not public"
I tried the above with SAS token and received the same error.

How can we reproduce the problem in the simplest way?

Have you found a mitigation/solution?

[zezha-msft]

@JasonYeMSFT could you please chime in here?

[JasonYeMSFT]

@Ilanak You need to explicitly set the location type. For example, if you are using Key, you need something like

const src = <IRemoteKeyLocation>{
    type: "RemoteKey",
    resourceUri: "https://srcaccount.blob.core.windows.net/container_name/", // The trailing slash is required.
    accountKey: "xxxxx",
    accountName: "srckey",
    path: "blobPath"
};
const dst = <IRemoteKeyLocation>{
    type: "RemoteKey",
    resourceUri: "https://srcaccount.blob.core.windows.net/container_name/", // The trailing slash is required.
    accountKey: "xxxxx",
    accountName: "dstKey",
    path: "blobPath"
};

If you are using SAS, change the type to "RemoteSas" and add the property sasToken: "......".

By the way, as reflected in the example above, the "resourceUri" is supposed to be the service endpoint to the container and the "path" property is supposed to be the full resource path. If you move the container name to the path property the url we generate won't be correct.

I have tested with RemoteKey and RemoteSas with explicit type and correcting the resourcUri. RemoteSas works fine but I keeps getting a different error failed to perform copy command due to error: get cached token failed to ensure token fresh, please log in with azcopy's login command again, adal: Refresh request failed. Status Code = '400'. Response body: {"error":"invalid_grant","error_description":"AADSTS700082: The refresh token has expired due to inactivity.. This reproduces even on command only (see example below), @zezha-msft could you help take a look?

export ACCOUNT_NAME=myAccountName;
export ACCOUNT_KEY=mayAccountKey;
./azcopy copy "https://myAccountName.blob.core.windows.net/container/srcblob" "https://myAccountName.blob.core.windows.net/container/destblob";
unset ACCOUNT_NAME;
unset ACCOUNT_KEY;

We always recommend using SAS/Auth over key to authenticate. It isn't necessary and may cause problems if the key in environment variables isn't cleaned properly.

[Ilanak]

Hi,

I changed to SAS token and I was missing the container name in the path.
Now it's working for me.
Didn't get it to work using account name and key.

[MRayermannMSFT]

@Ilanak for your original code, the real root problem is that you can only use 1 RemoteKey location at a time. The name and key are communicated to AzCopy using environment variables. So if you try to use key for both src and dst, then the dst name and key are ignored. There should probably be documentation that explains this since figuring that out requires some digging/connecting the dots. As Jason says though, it is good you are now using SAS. That is much more secure.