Aws-cli: aws s3 sync not working for restored glacier objects

[thagorx]

aws_cli_version = aws-cli/1.14.9 Python/2.7.13 Linux/4.9.76-3.78.amzn1.x86_64 botocore/1.8.13

I bulk restored 2 folders (both relatively large 10.5 TiB and 200GiB) in my bucket from glacier about 18h ago. Now I want to copy them to another bucket with aws s3 sync

aws s3 sync s3://mybucket/folder1/ s3://mybucket2/
aws s3 sync s3://mybucket/folder2/ s3://mybucket2/

but I just get the error message:
warning: Skipping file s3://mybucket/folder1/part-02431-c31bc063-e362-4d8b-9239-e515359ed4bf-c000.snappy.parquet. Object is of storage class GLACIER. Unable to perform copy operations on GLACIER objects. You must restore the object to be able to the perform operation. See aws s3 copy help for additional parameter options to ignore or force these transfers.

when I fetch the Metadata with aws s3api head-object --bucket mybucket --key folder1/part1.parquet this is the response:

{
    "Restore": "ongoing-request=\"false\", expiry-date=\"Sat, 14 Apr 2018 00:00:00 GMT\"", 
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Wed, 20 Dec 2017 15:02:48 GMT", 
    "ContentLength": 59408357, 
    "ETag": "\"49b981b95eaf62abed0880c0db7c1200-8\"", 
    "StorageClass": "GLACIER", 
    "Metadata": {}
}

which looks like it should do

when I try copy with aws s3 cp s3://mybucket/folder1/part1.parquet s3://mybucket2/ it works, but
--recursive does not work aws s3 cp --recursive s3://mybucket/folder1/ s3://mybucket2/ same error as above

[stealthycoin]

The error message mentions looking at the help for additional parameters for forcing glacier. Did you try --force-glacier-transfer?

[colemanja91]

@stealthycoin I am seeing the exact same issue. Trying with --force-glacier-transfer gives the following:

> aws s3 cp s3://mybucket/myobject ./ --force-glacier-transfer
download failed: s3://mybucket/myobject to ./myobject An error occurred (InvalidObjectState) when calling the GetObject operation: The operation is not valid for the object's storage class

[stealthycoin]

@colemanja91 You need to restore the object. If you can't copy it then it isn't restored. All --force- glacier-transfer does is ignore the warning and try anyway.

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[tdmalone]

I came here with the same issue - getting the same error when doing a sync from one bucket to another. I had done a restore, and calling head-object confirmed it had completed, but adding --force-glacier-transfer solved the problem.