Bentoml: Support Google Cloud Storage in BentoML Model Registry (YataiService)

[parano]

Is your feature request related to a problem? Please describe.

Currently, besides the file system API(local, NFS, AWS EFS, or any cloud storage with POSIX support), the only object storage that BentoML YataiService's storage backend supports today is AWS S3.

For example, a user can configure YataiService to store all BentoML SavedBundle files to a s3 bucket:

docker run -p 3000:3000 -p 50051:50051 \ 
  -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... \ 
  bentoml/yatai-service:0.7.2 \ 
  --db-url postgresql://scott:tiger@localhost:5432/bentomldb \ 
  --repo-base-url s3://my-bentoml-repo/

Users can also save and load directly to/from s3 cloud storage. e.g.:

svc = IrisClassifier()
svc.pack('model', model)
svc.save("s3://my-bucket/path")

bentoml run predict s3://my-bucket/path --input=s3://my-bucket/test.csv

This feature request is to add the same support for Google Cloud Storage. Simply replacing the s3 paths in the example above to a GCS path, everything should just work.

[Korusuke]

Hey @parano, I would like to work on this!
Although it might take me a while to implement this as I have never really worked with S3/GCS.

[parano]

Thanks @Korusuke, this is a non-trivial task and indeed it might take some time to get familiar with the codebase and S3/GCS. Happy to jump on a call and help you walk through some of the components related to building this if you think that could be helpful, I'm always available in slack in you have any questions too.

[Korusuke]

Thanks @parano
I will try some stuff out over the weekend.
And if by Monday I am not able to understand somethings, we can hop on a call!