Aws-sdk-js: S3: NoSuchBucket Error when AWS.S3 is configured inside a test

[rgmembreno]

Hi all, I'm getting an error when trying to read or write to an S3 Bucket depending on where I initialize the S3 client. The following code fails with a NoSuchBucket error returned from S3:

describe('AWS Client', function () {
      var AWS = require('aws-sdk')
      var logger = require('winston')

      it('puts data', function (done) {
       **var s3Client = new AWS.S3({
        params: { Bucket: <BucketName> },
        accessKeyId:<accessKey>,
        secretAccessKey: <secretAccessKey>
        region: 'us-east-1'
      })**
        var uploadParams = {
          Key: 'helloWorld.txt',
          Body: 'hello world'
        }

        s3Client.upload(uploadParams, function (err, data) {
          logger.info(err || 'no err')
          if (err) return done(err)
          logger.info('succeeded', data.Location)
          done()
        })
      })
    })

"NoSuchBucket: The specified bucket does not exist
at Request.extractError (C:\<...>\node_modules\aws-sdk\lib\services\s3.js:568:35)
at Request.callListeners (C:\<...>\node_modules\aws-sdk\lib\sequential_executor.js:105:20)
at Request.emit (C:\<...>\node_modules\aws-sdk\lib\sequential_executor.js:77:10)"

However, if I initialize my S3 client outside the test I don't get a NoSuchBucket error and I'm able to read and write to my bucket

describe('AWS Client', function () {
      var AWS = require('aws-sdk')
      var logger = require('winston')
      **var s3Client = new AWS.S3({
        params: { Bucket: <BucketName> },
        accessKeyId:<accessKey>,
        secretAccessKey: <secretAccessKey>
        region: 'us-east-1'
      })**

      it('puts data', function (done) {
        var uploadParams = {
          Key: 'helloWorld.txt',
          Body: 'hello world'
        }

        s3Client.upload(uploadParams, function (err, data) {
          logger.info(err || 'no err')
          if (err) return done(err)
          logger.info('succeeded', data.Location)
          done()
        })
      })
    })

We also see this error when we configure the client as an object property: "this.S3Client = ...". Could this be a configuration error or an issue with the S3 client code?

I tested with the latest aws-sdk v 2.72.2 and on our current version 2.2.4 and got the same results. I am running node v0.12.15 locally and on Opsworks. I get the same errors with two different IAM users--one has admin access, and the other has an IAM policy only allowing access to this Bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::/*"
]
}
]
}

Thanks in advance, any help is appreciated.

[chrisradek]

@rgmembreno
Can you construct your S3 client with logger: console in your configuration as well? That will log out what parameters each S3 method was called with. It'll let you validate which Bucket is being used when making the upload call in both of your above examples.

[rgmembreno]

Hi @chrisradek here are the logged responses:

Success:
2017-06-17T01:59:07.054Z - info: [AWS s3 200 0.804s 0 retries] putObject({ Body: ,
Bucket: 'myBucket',
Key: 'helloWorld.txt' })

Failed:
2017-06-17T01:59:07.054Z - info: [AWS s3 404 0.144s 0 retries] putObject({ Body: ,
Bucket: 'myBucket',
Key: 'helloWorld.txt' })

The bucket names are the same in both cases, but there are 404 errors when the S3 client is declared inside the test enclosure. This also squares with another test I made with the admin IAM user: I can see all of my buckets when I declare the S3 client as a global value, but I only see a handful of buckets (and not "myBucket") when I call the S3 client in the test itself.

Note that all tests were done in aws-sdk v 2.2.4. I can try again with the latest version tomorrow, but I don't think I'll get different results.

[rgmembreno]

Hi @chrisradek do you need any more info from me? This issue is blocking one of our clients so I'd like to dig into this error before we can make a fix for it in our application.

[rgmembreno]

Hi everyone,

It's been almost 2 weeks with no other input, and this is a showstopper issue for us for one client. Is this an issue with the aws-sdk or is it just a configuration issue? Any insight on why we lack permissions to access our buckets depending on where the S3 client is declared?

[chrisradek]

@rgmembreno
Where your S3 client is declared really should have no affect on your tests. Do any of your tests/beforeEach/afterEach functions modify the S3 client, or AWS.config?

Can you log out your s3Client.config in both cases and see how they differ?

[rgmembreno]

Hi @chrisradek
Thanks for reaching out. The code snippets I posted were the entire test suite--no beforeEach/afterEach functions were used. The S3 client is declared and configured and then left unchanged in both cases.

[srchase]

@rgmembreno

Were you able to log the s3Client.config inside and outside a test to see how they differ?

Is this still an open issue for you?

[rgmembreno]

Hi @srchase

Yes, I was able to figure this out, but I didn't update this case until now. Thanks for all the help from @chrisradek and the team--we were able to track the root cause with their help. I'm closing the case now.

[lmajano]

What was the issue @rgmembreno I am hitting the same issue

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.