Django-storages: Upload files > 5 gb with S3BotoStorage class?

[pacahon]

Hello. Is it possible to upload files > 5gb (and less < 50 gb) to s3 storage in multipart mode? I can see something related in S3BotoStorageFile class, but looks like it's not used in _save method, only set_contents_from_file which try to upload the whole file as I guess.

[ZuluPro]

@pacahon : Sorry for the latency, I must look deeper before help you

[pacahon]

@ZuluPro I hope that the answer https://github.com/boto/boto3/issues/789
Looks like we can replace put method with upload_fileojb. Will test it ASAP.

[ZuluPro]

@pacahon I think we're going to remove boto3 from django-storages.
Boto is compatible with Python 3, so no need to maintains a second almost similar backend.

[pacahon]

@ZuluPro Boto3, the next version of Boto, is now stable and recommended for general use. Maybe remove boto? :)

[mbarrien]

@ZuluPro once again, as Boto maintainers themselves say, Boto is effectively deprecated for Boto 3. http://boto.readthedocs.io/en/latest/ says this multiple times on the page:

"Note Boto3, the next version of Boto, is now stable and recommended for general use. It can be used side-by-side with Boto in the same project, so it is easy to start using Boto3 in your existing projects as well as new projects. Going forward, API updates and all new feature work will be focused on Boto3."

Notice the last sentence. Boto 2 is effectively dead. Removing Boto3 support in favor of something that has been stated by the maintainers to be dead is not the wisest move.

[jschneier]

We are definitely going to keep both. Boto3 is the future but tons and tons of projects rely on the old Boto backend. I'm just going to WONT-FIX boto issues that are pains in the future and point them at Boto3.