Duplicati: "GLACIER is not allowed" when running an S3 backup with storage class set to GLACIER

[ghost]

I have:

• [x] searched open and closed issues for duplicates

---

Version info

Duplicati Version: 2.0.1.53
Operating System: Fedora 25
Backend: S3

Bug description

My backup to S3 with the Glacier storage class fails with "GLACIER is not allowed". I've tried using multiple regions that I know support Glacier (us-east-1, us-west-1, us-west-2), but none of them work. I've included the full traceback below.

Steps to reproduce

• set up a backup with s3 as the backend
• choose us-east-1 as the region
• choose Glacier as the storage type
• run the backup

Actual result: the backup fails with the aforementioned error
Expected result: the backup should succeed

Screenshots

n/a

debug log

{"ClassName":"Amazon.S3.AmazonS3Exception","Message":"GLACIER is not allowed.","InnerException":{"ClassName":"Amazon.Runtime.Internal.HttpErrorResponseException","Message":"The remote server returned an error: (400) Bad Request.","InnerException":{"ClassName":"System.Net.WebException","Message":"The remote server returned an error: (400) Bad Request.","InnerException":null,"HelpURL":null,"StackTraceString":" at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x40d8e090 + 0x001cb> in :0 \n at System.Net.HttpWebRequest.GetResponse () <0x40eb9ad0 + 0x0005a> in :0 \n at Amazon.Runtime.Internal.HttpRequest.GetResponse () <0x40f53c70 + 0x00043> in :0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"HResult":-2146233079,"Source":"System","ExceptionMethod":null,"Data":null},"HelpURL":null,"StackTraceString":" at Amazon.Runtime.Internal.HttpRequest.GetResponse () <0x40f53c70 + 0x00303> in :0 \n at Amazon.Runtime.Internal.HttpHandler`1[TRequestContent].InvokeSync (IExecutionContext executionContext) <0x40f60c40 + 0x0023b> in :0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (IExecutionContext executionContext) <0x40f52050 + 0x00034> in :0 \n at Amazon.Runtime.Internal.RedirectHandler.InvokeSync (IExecutionContext executionContext) <0x40f60bc0 + 0x00023> in :0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (IExecutionContext executionContext) <0x40f52050 + 0x00034> in :0 \n at Amazon.Runtime.Internal.Unmarshaller.InvokeSync (IExecutionContext executionContext) <0x40f60b10 + 0x00017> in :0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (IExecutionContext executionContext) <0x40f52050 + 0x00034> in :0 \n at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync (IExecutionContext executionContext) <0x40f60aa0 + 0x00017> in :0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (IExecutionContext executionContext) <0x40f52050 + 0x00034> in :0 \n at Amazon.Runtime.Internal.ErrorHandler.InvokeSync (IExecutionContext executionContext) <0x40f609b0 + 0x00027> in :0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"HResult":-2146233088,"Source":"AWSSDK.Core","ExceptionMethod":null,"Data":null,"Response":{"StatusCode":"BadRequest","IsSuccessStatusCode":false,"ContentType":"application/xml","ContentLength":-1,"ResponseBody":{}}},"HelpURL":null,"StackTraceString":" at Duplicati.Library.Main.Operation.BackupHandler.HandleFilesystemEntry (ISnapshotService snapshot, Duplicati.Library.Main.BackendManager backend, System.String path, FileAttributes attributes) <0x40ee1820 + 0x01f53> in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (ISnapshotService snapshot, Duplicati.Library.Main.BackendManager backend) <0x40ee0dd0 + 0x0064f> in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, IFilter filter) <0x40e3f3f0 + 0x01757> in :0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"HResult":-2146233088,"Source":"Duplicati.Library.Main","ExceptionMethod":null,"Data":null,"errorCode":"InvalidArgument","errorType":"Sender","requestId":"83B57CF992B5EF26","statusCode":"BadRequest","AmazonId2":"redacted","ResponseBody":null}

[kenkendk]

The docs explain this:
https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html

You cannot specify GLACIER as the storage class at the time that you create an object. You create GLACIER objects by first uploading objects using STANDARD, RRS, or STANDARD_IA as the storage class. Then, you transition these objects to the GLACIER storage class using lifecycle management. For more information, see Object Lifecycle Management.

There is an old (read: very old) guide to setting it up here:
https://sites.google.com/a/duplicati.com/duplicati/news/howtouseglaciertostorebackups

[ghost]

Thanks for that. Based on that guide, it sounds like choosing "Glacier" for the storage class will never work (all of the Glacier transitioning appears to happen within the S3 console). If that's the case, should that be removed as an option from the drop down?

[ManasN]

Hi, is this still the case?

[LukBukkit]

Now it works if you use the storage type GLACIER, but all the index files get also put in glacier and I don't know if we want that (see the linked old guide).
And the backup won't come to an end, because it wants to upload one last file with a negative size.