Azure-storage-azcopy: File Modification time using azcopy

[brwilkinson]

Is there any way to move files into Azure BLOB/FILE to include the original creation time stamp of the files ?

azcopy v10.

[JohnRusk]

Definitely not for blob, because that's not supported by the blob Service (i.e. you can't fake or modify the creation date for Blobs). For Files, I don't know what's supported on the back end, but we don't have any code to do it in AzCopy itself. I'll mark this as feature request because it might be possible for Files.

Check out Azure File Sync if you need it for Files. (I think that does what you're asking. BTW it's conceptually a different product that azcopy, more about on-going active synchronisation that transfer "jobs" - if I understand correctly)

[brwilkinson]

Thanks @JohnRusk ... appreciated.

I have already made a recommendation for Azure File sync, it does suit the customer requirements for the data migration they are performing. Thanks for the suggestion.

[JohnRusk]

For future reference, I've just remembered that I think there are plans to add the date-preservation feature to AzCopy in a future release. (I.e. not 10.3, which is due out soon, but something later). So that doesn't help you right (stick with File Sync) but it may help future readers of this thread who want the feature in AzCopy.

[dwrightpulse]

+1 on the request to preserve file timestamps when copying with AzCopy.

To preserve timestamps on my copies now, I have had to use file sync with cloud tiering to create a local cache and then robocopy with /copy:dat

[JohnRusk]

For Azure Files, this is currently scheduled for release 10.4. We have no plans to add this feature for Blobs at present, because the Blob Service does not currently support it.

[dwrightpulse]

@JohnRusk - thanks for the update, is there an ETA for the 10.4 release?

[JohnRusk]

Our current policy for AzCopy is that we don't publish release ETAs. Sorry. 10.4 is our next planned release - i.e. we are not planning any other 10.3.x to come before it.

[lesterw1]

@JohnRusk Blobs do support metadata. I suggest adding a --Put-FileDates or similar switch into azcopy copy and azcopy sync which will write the following additional metadata to the target blob:

• CreationTime
• LastWriteTime
• LastAccessTime

Once these values are written (preferably in UTC time or with a time zone indicator), it opens up a whole new world of possibilities, where --put-FileDates can be bi-directional. For example, azcopy sync could use the file dates for synchronization if --Put-FileDates is specified.

Personally, I'd like to be able to sync files across several PCs using Azure storage. The current azcopy sync doesn't work well when syncing 2 or more locations to/from Azure storage.

[JohnRusk]

Great suggestion. We are already working on it, as one of the major features for 10.4. But note that its only possible to make it work on Azure Files. Blob Storage does not currently allow the setting of file dates.

Also note that of the three dates you mentioned, only two will be supported. LastAccessTime will not be.

I'm going to leave this thread open, re-title it "multi machine sync" since that's an important point which I think we should think through a bit more.

[JohnRusk]

On second thoughts, I'll make a separate issue for multi-machine sync

[adreed-msft]

This is now supported in 10.4 via --persist-smb-info. Enjoy!

[vishalaj1]

Hi,

@adreed-msft : Just a note and confirmation that: --persist-smb-info is only for Azure File Share?

Regards,
Vishal

[zezha-msft]

Yes @vishalaj1

[ChandraSM]

Hi,
--persist-smb-info works only if you are trying to copy from one storage account file share to another storage account file share. Shouldn't this be an option while uploading from VM to Azure File Share as well?
https://github.com/microsoft/AzureStorageExplorer/issues/2969

[hedegaardDK]

Can it be right that --persist-smb-info hasn't been developed to also retain folder creation date?

[zezha-msft]

@ChandraSM Yes it works for uploading to Azure File Share as well.

@hedegaardDK No it should work for the folder creation date.

[hedegaardDK]

When using the newest azcopy.exe (v10.8.0) and this below command, it did retain the files date, but the folder was changed to the migrating date and time.
And it is to Azure File share i am migrating.
azcopy.exe copy "E:\Data\blabla\blabla*" "https://blablablabla.file.core.windows.net/azurefileblablabla1/blablabla/blablabla/*?se=2021-01-13T13%3A13%3A14Z&sp=rwl&sv=2018-03-28&sr=s&sig=blablablablabla" --overwrite=prompt --from-to=LocalFile --follow-symlinks --put-md5 --follow-symlinks --preserve-smb-info=true --recursive

[JohnRusk]

@zezha-msft There's a table in the Folders as First Class Citizens design document (or maybe the SMB properties preservation doc), which should record any cases where we can't move the folder dates. (E.g. it explains why we can't and don't move folder _modification_ dates). I don't have access to the document right now, but if you have a moment to check it, it might have something useful.