Aws-cli: aws s3 sync --delete: delete files _after_ uploading

Created on 9 Jul 2015  路  9Comments  路  Source: aws/aws-cli

Hello,

With awscli==1.7.37, when using s3 sync --delete, is there a way to have the files be deleted after the rest is uploaded? It seems like they鈥檙e always deleted before uploading.

$ aws s3 sync --delete dist s3://bucket
delete: s3://bucket/scripts/bundle.65406bc3e1133110.js
upload: dist/index.html to s3://bucket/index.html
upload: dist/scripts/bundle.9ad9b21077147aa5.js to s3://bucket/scripts/bundle.9ad9b21077147aa5.js
upload: dist/styles/bundle.6de17660ab595a44.css to s3://bucket/styles/bundle.6de17660ab595a44.css
feature-request s3 s3syncstrategy

Most helpful comment

A quick workaround is to run the sync command twice, once without --delete, then with the --delete flag.

All 9 comments

What's the reasoning for waiting to delete until after all uploads are complete? If we did that, then we'd have to keep every delete request in memory, which could consume a large amount of memory for large buckets.

@mtdowling all the reasoning that鈥檚 behind rsync鈥檚 --delete-after.

For example, if the transfer fails while syncing, let鈥檚 say right after the delete, I鈥檝e deleted the old files, but haven鈥檛 added any of the new, which could be a problem.

Interesting. I suppose we could also evaluate adding other delete strategies similar to the delete options offered by rsync. I'll mark this as a feature request, and we'll add it to our backlog.

A quick workaround is to run the sync command twice, once without --delete, then with the --delete flag.

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We鈥檝e imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it鈥檚 a text-only import of the original post into UserVoice, we鈥檒l still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

Based on community feedback, we have decided to return feature requests to GitHub issues.

Any updates on this? the feature delete_after is important IMHO in order to clean up source systems that have been synced

@mehditlili use rclone instead.

Run your command twice, second time with --delete will delete the files after and won't re-upload the files the second time you run it.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ypant picture ypant  路  3Comments

KimberleySDU picture KimberleySDU  路  3Comments

pawelkilian picture pawelkilian  路  3Comments

ikim23 picture ikim23  路  3Comments

alexejk picture alexejk  路  3Comments