Amplify-cli: Support for multiple buckets?

[ffxsam]

In the documentation examples:

Amplify.configure(
    Auth: {
        identityPoolId: 'XX-XXXX-X:XXXXXXXX-XXXX-1234-abcd-1234567890ab', //REQUIRED - Amazon Cognito Identity Pool ID
        region: 'XX-XXXX-X', // REQUIRED - Amazon Cognito Region
        userPoolId: 'XX-XXXX-X_abcd1234', //OPTIONAL - Amazon Cognito User Pool ID
        userPoolWebClientId: 'XX-XXXX-X_abcd1234', //OPTIONAL - Amazon Cognito Web Client ID
    },
    Storage: {
        bucket: '', //REQUIRED -  Amazon S3 bucket
        region: 'XX-XXXX-X', //OPTIONAL -  Amazon service region
    });

What if the web app needs to interact with more than one bucket? It would be nice to have a system where we could specify several and interact with them via their names.

    Storage: {
      bucketOne: {
        bucket: '', //REQUIRED -  Amazon S3 bucket
        region: 'XX-XXXX-X', //OPTIONAL -  Amazon service region
      },
      bucketTwo: { ... }
    });

[tim-thompson]

Don't know what the progress on this has been but as it is still open thought I would comment.
I had this same restriction today and after some digging found that it is possible to pass a bucket option into various calls as follows:

Storage.vault.get(key, {bucket: 'alternative-bucket-name'});

Using this I've managed to successfully use multiple buckets in the same app. If it is not specified then it defaults back to the bucket in the global configuration for Amplify.

[jnreynoso]

Hi, what is valud @tim-thompson ? Storage.vault

[annjawn]

Is there any update on this? Support for multiple buckets is a really desirable feature.

[tim-thompson]

@annjawn I posted a solution further up this page that works for all my scenarios. If you need more info then I've written about it on my blog in more detail - http://tim-thompson.co.uk/aws-amplify-multiple-buckets.

[annjawn]

@tim-thompson i have tried the storage.vault method but it did not work for me for some reason. Also, it looks like only get works with storage.vault however the code suggests otherwise. I've found a solution btw. I am doing storage.config() before each operation by setting the appropriate bucket name. It's less than efficient, but it's getting the job done.

[rizerzero]

@annjawn Hi, do you have a blog post on your method ? thanks in advance 👍

[10ky]

If you are able to get content off of a bucket using this statement:

Storage.vault.get(key, {bucket: 'alternative-bucket-name'});

It would be a security issue. Unless you allow it in an IAM role attached the a user. I believe amplify uses this role "s3_amplify_...". This role should be modified automatically according to your aws-exports.js file when you do amplify push. I don't see how the above statement would affect "amplify push".

[10ky]

@mlabieniec is this feature request removed from the aws-amplify milestone on Jul 19? I thought this is a good feature to have. I think I have a use case where all my resized photos in S3 can be in a separate bucket. Right now, the S3image and album library resizes photo at the client side. If my photo files are very large, that would not be desirable. If the resize file is put in the same directory as in the private user directory, a lambda trigger would not work because S3 trigger does not support regular expression prefix match.

[ngocketit]

It would be very convenient to have this supported. Currently, I have to call Amplify.configure() with new bucket every time I want to do something with non-default bucket.

[hoang-innomizetech]

We are also looking for this feature, we are building an app that requires access to multiple buckets, so it would be better if we don't have to specify the bucket when configuring amplify (or we can use default bucket), some APIs also need to allow us specify bucket such as get pre-signed url

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dylan-westbury]

Something similar with configuring API, you can specify an array of endpoints. Array of buckets could be nice

[DmitriWolf]

I also would like to see a feature added to Amplify to support the use of multiple buckets.
Great work. Thank you all, contributors.

[Off2Race]

I used @tim-thompson's suggestion and it worked for me as well. The documentation for Storage.get probably needs to be updated but the following works fine:

Storage.get(key1, {bucket: 'bucket-1'});  
Storage.get(key2, {bucket: 'bucket-2'});  

I've only tried it for "public" access (any authenticated user of my app) but looking at the code I don't see a reason why it wouldn't work in other scenarios too. In effect, the bucket you specify during the Amplify.configure appears to be a default that can be overridden.

[jimji1005]

above only works if buckets are in the same region, unfortunately. 🤦‍♂

[Ramesh-Chathuranga]

if you want to add more s3 bucket to your project, use Configuration when uploading file.
this is an example code for multi s3 bucket.

function=(fileName, file, isUser=false)=>{
  if(isUser){
  Storage.configur({ AWSS3: {
          bucket: 'bucketA',
          region: "us-exxxx"
        }});
  return Storage.put(fileName, file, {
        contentType: file.type,
      });
}else {
   Storage.configur({ AWSS3: {
          bucket: 'bucketB',
          region: "us-exxxx"
        }});
  return Storage.put(fileName, file, {
        contentType: file.type,
      });
}}

[dtelaroli]

I'm used to use one bucket and event trigger by environment/account.
It would be great a native cli support for that.

[aelbokl]

I am just commenting to keep the bot from killing this thread. This feature is much needed and has many use cases.

[KesleyDavid]

Also needing this feature

[cody1024d]

Adding my need for this functionality too

[PatrykMilewski]

Would be nice to have that

[sammartinez]

cc @renebrandel that we need Amplify CLI to do the implementation first prior to us doing anything on Amplify JS

[r0zar]

Does https://github.com/aws-amplify/amplify-cli/issues/3977 solve for this use case? I imagine that it would.

[stleon]

amplify import storage
Scanning for plugins...
Plugin scan successful
? Please select from one of the below mentioned services: S3 bucket - Content (Images, audio, video, etc.)
Amazon S3 storage was already added to your project.

It would be very useful if there were any support for multiple storages.