Dpl: Pin the version of the aws-sdk gem

[cotsog]

Since the last couple of months, when a new version of the aws-sdk gem is released, we are seeing a lot of deployments failing because one of the gem's dependencies (i.e. aws-sdk-resources) isn't released at the same time. Here's a sample of the error our customers are seeing:

Installing deploy dependencies
ERROR: Could not find a valid gem 'aws-sdk-resources' (= 2.10.88) in any repository
ERROR: Possible alternatives: aws-sdk-resources
/home/travis/.rvm/rubies/ruby-2.2.7/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- aws-sdk (LoadError)

This happens because the version of aws-sdk in our Gemfile is lax:

gem 'aws-sdk', '~> 2.10.39', '< 3.0'

Hence I would suggest we pin the version of the aws-sdk gem to prevent this from happening again in the future.

Thank you!

[danielnguyen]

Any updates and/or available workaround for this issue?

[jdub]

Getting the same / similar issue with the S3 provider at the moment:

Fetching: dpl-1.8.43.gem (100%)
Successfully installed dpl-1.8.43
1 gem installed

Installing deploy dependencies
ERROR:  Could not find a valid gem 'aws-sdk-resources' (= 2.10.97) in any repository
ERROR:  Possible alternatives: aws-sdk-resources

/home/travis/.rvm/rubies/ruby-2.2.7/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- aws-sdk (LoadError)
    from /home/travis/.rvm/rubies/ruby-2.2.7/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:59:in `require'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider.rb:85:in `requires'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider/s3.rb:6:in `<class:S3>'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider/s3.rb:5:in `<class:Provider>'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider/s3.rb:4:in `<module:DPL>'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider/s3.rb:3:in `<top (required)>'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider.rb:59:in `const_get'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider.rb:59:in `block in new'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/cli.rb:41:in `fold'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/provider.rb:56:in `new'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/cli.rb:31:in `run'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/lib/dpl/cli.rb:7:in `run'
    from /home/travis/.rvm/gems/ruby-2.2.7/gems/dpl-1.8.43/bin/dpl:5:in `<top (required)>'
    from /home/travis/.rvm/gems/ruby-2.2.7/bin/dpl:23:in `load'
    from /home/travis/.rvm/gems/ruby-2.2.7/bin/dpl:23:in `<main>'

[alexbecker]

Currently happening to us as well. Unable to deploy to any of our environments.

[alexbecker]

Fixed for now but it'll happen again unless the version is pinned.

[stale[bot]]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically closing the issue. This is often because the request was already solved in some way and it just wasn't updated or it's no longer applicable. If that's not the case, please do feel free to either reopen this issue or open a new one. We'll gladly take a look again! You can read more here: https://blog.travis-ci.com/2018-03-09-closing-old-issues

[m2jobe]

A little dissapointing that this is still happening. AWS SDK RUBY 2.11.393 was released yesterday and since then all builds previously working and set with an aws_access_key_id are not outputting the following

/home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call': The AWS Access Key Id you provided does not exist in our records. (Aws::S3::Errors::InvalidAccessKeyId)
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/s3_dualstack.rb:24:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/s3_accelerate.rb:34:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/idempotency_token.rb:18:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/seahorse/client/plugins/response_target.rb:21:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/seahorse/client/request.rb:70:in `send_request'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-core-2.11.393/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-resources-2.11.393/lib/aws-sdk-resources/request.rb:24:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-resources-2.11.393/lib/aws-sdk-resources/operations.rb:41:in `call'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/aws-sdk-resources-2.11.393/lib/aws-sdk-resources/operation_methods.rb:19:in `block in add_operation'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-elastic_beanstalk-1.10.13/lib/dpl/provider/elastic_beanstalk.rb:131:in `upload'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-elastic_beanstalk-1.10.13/lib/dpl/provider/elastic_beanstalk.rb:49:in `push_app'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/lib/dpl/provider.rb:199:in `block in deploy'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/lib/dpl/cli.rb:41:in `fold'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/lib/dpl/provider.rb:199:in `deploy'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/lib/dpl/cli.rb:32:in `run'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/lib/dpl/cli.rb:7:in `run'
    from /home/travis/.rvm/gems/ruby-2.4.5/gems/dpl-1.10.13/bin/dpl:5:in `<top (required)>'
    from /home/travis/.rvm/gems/ruby-2.4.5/bin/dpl:23:in `load'
    from /home/travis/.rvm/gems/ruby-2.4.5/bin/dpl:23:in `<main>'

If versions arent going to pinned can basics tests like these be run to ensure this doesnt affect teams development flow?

[svenfuchs]

@m2jobe thanks for the report.

i believe this is not related to v2.11.393, but https://changelog.travis-ci.com/more-secure-handling-of-secure-values-in-configuration-126390. you haven't given a link to your build log, so i cannot verify it's this same issue, but we have seen a couple reports with the same backtrace earlier today.

do you have any instance of secure: "plain_unencrypted_string" in your build configs? if so, please try removing the secure: bit

[m2jobe]

@svenfuchs Thanks so much for the response, that was exactly it for both key id and secret!