Django-storages: Support for Backblaze B2 Cloud Storage

[chickahoona]

Backblaze has quite a name for backups and I "recently" discovered that they also have an API.
Their pricing seems to beat GCP / Azure and AWS S3 by far.
https://www.backblaze.com/b2/cloud-storage-pricing.html

They have a command line tool here for python, that should handle the "hard work" e.g. keeping track of the auth token expiry, a worker thread pool..., so the implementation should be straight forward:
https://github.com/Backblaze/B2_Command_Line_Tool
(Their API layere can be accessed here https://github.com/Backblaze/B2_Command_Line_Tool/blob/master/b2/api.py)

I also found their official "SDK" here for B2:
https://pypi.org/project/b2sdk/

[techlover1]

With the merge of #807 B2 is now working with the apache-libcloud backend. Install apache-libcloud then add the following to your settings.py

LIBCLOUD_PROVIDERS = {
    'backblaze': {
        'type': 'libcloud.storage.types.Provider.BACKBLAZE_B2',
        'user': 'Key ID',
        'key': 'Secret key',
        'bucket': 'Bucket Name',
    }
}
DEFAULT_LIBCLOUD_PROVIDER = 'backblaze'
DEFAULT_FILE_STORAGE = 'storages.backends.apache_libcloud.LibCloudStorage'
MEDIA_URL = 'https://f000.backblazeb2.com/file/%s/' % (LIBCLOUD_PROVIDERS['backblaze']['bucket'])

[chickahoona]

Thank you very much for your reply! Sadly the libcloud implementation is too "stiff" to be used by me, as one cannot use for example multiple accounts of the same type (e.g. multiple backblaze accounts) or configurations coming out of the database and so on.

I usually use django storage directly. e.g.:

from django.core.files.storage import get_storage_class

storage_class = 'storages.backends.gcloud.GoogleCloudStorage'
kwargs = {
    project_id: 'abc-123456',
    bucket_name: 'abc',
    ...
}
storage = get_storage_class(storage_class )(**kwargs)

The LibCloudStorage sadly has no possibility to specify the class / kwargs for the storage directly and always uses the ones from the settings.

If one would adjust the __init__ method of the LibCloudStorage class from:

@deconstructible
class LibCloudStorage(Storage):

    def __init__(self, provider_name=None, option=None):
        ...
        self.provider = settings.LIBCLOUD_PROVIDERS.get(provider_name)
        ...

to something like this:

@deconstructible
class LibCloudStorage(Storage):

    def __init__(self, provider_name=None, option=None, **kwargs):
        ...
        self.provider = settings.LIBCLOUD_PROVIDERS.get(provider_name, kwargs)
        ...

the one could do the same approach as before with the "native" storage backends. e.g.

from django.core.files.storage import get_storage_class

storage_class = 'libcloud.storage.types.Provider.BACKBLAZE_B2'
kwargs = {
    'type': 'libcloud.storage.types.Provider.BACKBLAZE_B2',
    'user': 'Key ID',
    'key': 'Secret key',
    'bucket': 'Bucket Name',
}
storage = get_storage_class(storage_class )(**kwargs)

I am happy to create the PR if you would accept it. (currently there are 38 open PRs so I wanted to ask before wasting potentially my time)

[ehossack]

Wanted to bump this one.
Having this would also help me, especially since there's some limited configurability with the backblaze options.

There's less PRs now!

[ehossack]

Since there was no movement here, I implemented my own library. Hopefully it can be of use to someone else:
https://pypi.org/project/django-backblaze-b2/

[esquire900]

B2 now has an S3 compatible API. Getting it to work is as simple as using the boto s3 storage with a configuration like:

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = os.getenv('B2_USER')
AWS_SECRET_ACCESS_KEY = os.getenv('B2_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('B2_BUCKET')
AWS_S3_CUSTOM_DOMAIN = 'f002.backblazeb2.com/file/B2_BUCKET'
AWS_S3_ENDPOINT_URL = 'https://B2_BUCKET.s3.us-west-002.backblazeb2.com' - exact url stated in b2 bucket overview page

[boyarzun]

B2 now has an S3 compatible API. Getting it to work is as simple as using the boto s3 storage with a configuration like:

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = os.getenv('B2_USER')
AWS_SECRET_ACCESS_KEY = os.getenv('B2_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('B2_BUCKET')
AWS_S3_CUSTOM_DOMAIN = 'f002.backblazeb2.com/file/B2_BUCKET'
AWS_S3_ENDPOINT_URL = 'https://B2_BUCKET.s3.us-west-002.backblazeb2.com' - exact url stated in b2 bucket overview page

It works!

But, when I replace a file, other hidden files appear in the bucket. File versioning is disabled in my B2 Bucket.

This is the structure:

• 98b4e3e7-0415-4658-9ae7-8ed043b3ed16.css (The second one I uploaded, was supposed to eliminate the first one, but the other two files were generated)
• a7dd8a09-eba3-463d-9631-6f45bb04ad7a.css (2) * (The first one I uploaded)
  
  
  
  • a7dd8a09-eba3-463d-9631-6f45bb04ad7a.css (hidden)
  
  
  • a7dd8a09-eba3-463d-9631-6f45bb04ad7a.css
  
  

I'm using django-cleanup. It automatically deletes files for FileField, ImageField, and subclasses. When a FileField’s value is changed and the model is saved, the old file is deleted.