Packer: Regression - No valid credential sources found for AWS Builder.

[hegyre]

• Packer version = 1.2.0
• Host platform = Windows 8.1 (and Windows 10 for my colleague which reproduced the issue)

Got No valid credential sources found for AWS Builder.

First of all, with Packer 1.1.3, there is no problem, it works like a charm with the config below
The problem appeared only after upgrading to Packer 1.2.0

I don't use direct credentials in Packer file, nor with environment variables, nor with -var parameter
I always use my AWS credential file at %USERPROFILE%/.aws/credentials and %USERPROFILE%/.aws/config
Note that the config file contains the AWS profile which is working great with either AWS Cli and Packer 1.1.3

The same issue happens with a colleague for which I asked to try to reproduce the issue.

The ami.json file contains (important part only):

{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": "",
    "aws_profile"   : "scaling",

    "location"      : "ie",
    "project_region": "eu-west-1",
    "ManagedBy"     : "Packer"
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "profile": "{{user `aws_profile`}}",
      "region": "{{user `project_region`}}",
...

The file %USERPROFILE%/.aws/credentials contains:

[default]
aws_access_key_id = ***********
aws_secret_access_key = ***********

[another-cred]
...
[another-cred]
...

The file %USERPROFILE%/.aws/config contains:

[default]
region = eu-west-1

[profile scaling]
region = eu-west-1
role_arn = arn:aws:iam::*******:role/my_role
#role_arn = arn:aws:iam::*********:role/CrossAccountAdministrator
source_profile = default
#mfa_serial = arn:aws:iam::*******:mfa/****

[profile another_one]
...

Result with Packer 1.2.0:

amazon-ebs output will be in this color.

Build 'amazon-ebs' errored: No valid credential sources found for AWS Builder. Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials for more information on providing credentials f
or the AWS Builder.

==> Some builds didn't complete successfully and had errors:
--> amazon-ebs: No valid credential sources found for AWS Builder. Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials for more information on providing credentials for the AWS B
uilder.

==> Builds finished but no artifacts were created.

[SwampDragons]

It looks like the AWS sdk expects the credentials and config files to be in %USERPROFILE%\.aws; not %USERDATA%/.aws; when I put the file under %USERPROFILE% there I have no problem running a packer build.

We did update the aws sdk in the 1.2.0 release, but it looks like it's always been this way -- I'm surprised this ever worked for you.

Going off this It looks like you can override these defaults by setting the env vars AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE

SDK code

SDK docs

[SwampDragons]

I'm going to close this because we already have in our docs where the file needs to be: https://www.packer.io/docs/builders/amazon.html#shared-credentials-file

If you still can't get this working after moving your file to the correct place, please reopen.

[hegyre]

@SwampDragons ,
sorry, my bad on this, the env variable in the issue description is wrong, but I have these 'credentials' and 'config' files in the correct folder %USERPROFILE%/.aws/ as you've mentionned (and so in the docs), not in %USERDATA%/.aws/

Sorry for the mess (bad copy//paste...)

Could you please re-open the issue (I don't have the right to do so) ?
I'll update the description correctly

[SwampDragons]

@hegyre I still can't reproduce; I can use the aws credentials files just fine, if they're in the right place. The only way I can reproduce your error is by deleting the files from %USERPROFILE%/.aws -- are you sure you're running Packer as the right user?

[hegyre]

Hello @SwampDragons,

As told, If I use packer 1.1.3 instead of packer 1.2.0, then it works fine again.
And my co-worker can reproduce it.

Maybe a thing I've omitted: I use a cross account (without MFA) and that is why I need the profile (in the config file)
If you're still there I can show you with a teamviewer session

[salvianreynaldi]

I'm having No valid credential sources found for AWS Builder as well when running packer 1.2.0 in AWS CodeBuild using IAM Task Role. v 1.1.2 worked fine
template: https://github.com/traveloka/traveloka-bake-ami-helper/blob/0.1.0/aws-ebs-traveloka-ansible.json

[rlees85]

I can't get Packer 1.2.0 to do anything in AWS where I require an IAM role to be assumed with AWS_PROFILE.

Worked fine before... I'm on Linux, and my .aws folder is in the correct/same place.

Downgrading to 1.1.3 works with no issues

[SwampDragons]

Okay, I'm still flying blind here re: repro but it seems related to assuming roles. Looking through the 1.2.0 changelog, this seems like a possible culprit: https://github.com/hashicorp/packer/pull/5764 . Here's a windows binary of the master branch with 5764 reverted -- @hegyre can you give it a whirl and let me know if your problem still exists? packer.zip

[SwampDragons]

and here's the same binary built for linux
packer.zip

[hegyre]

Hello @SwampDragons ,

Yeah this version is working fine!
However are you sure this is #5764 ? Could we have a 1.2.1-dev windows version of Master without the revert, in order to ensure this is effectively #5764 ?

[SwampDragons]

Sure, though we haven't merged much in since the release:
packer.zip

[hegyre]

@SwampDragons ,
Thx, just tested with your new build, without the revert, and having the issue again.
So you've discovered the correct culprit :)

[SwampDragons]

Now to figure out _why_...

[hegyre]

Well, could it be possible that the issue is inside aws-sdk-go ?

Have a look at that PR: https://github.com/aws/aws-sdk-go/pull/1770
Fix https://github.com/aws/aws-sdk-go/issues/1771
Related to awslabs/amazon-ecr-credential-helper#84

It seems the same issue we have, and has been fixed recently in v1.12.72 : https://github.com/aws/aws-sdk-go/pull/1779

So you may just need to update your go sdk ?

[mwhooker]

@hegyre interesting, good catch! Here's a windows binary with the sdk updated if you don't mind testing

packer-windows.zip

[ikthorn]

New packer user, was pulling my hair out seeing this same behavior on 1.2.0 on OSX, with MFA+assumerole.
Dropping back to 1.1.3 and it works a treat.

[hegyre]

@mwhooker , unfortunately still seeing the issue :(

Is it possible that you have to add an import:
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
and / or
"github.com/aws/aws-sdk-go/internal/shareddefaults"
and/or
"github.com/aws/aws-sdk-go/aws/defaults"

In file https://github.com/hashicorp/packer/blob/master/builder/amazon/common/access_config.go

I'm asking this because of these addiotns in the commits: https://github.com/aws/aws-sdk-go/commit/33ba3859ae4a8204a009f78f17d9c29dfb934c5f

[mwhooker]

okay, I will try to reproduce locally and work on a solution

[mwhooker]

Thanks all for the help identifying and debugging this. sorry for the regression! I've made a PR in #5914 that I believe resolves this. In the PR you will find compiled binaries for windows and darwin. If anyone could help test, it would be much appreciated

[hegyre]

@mwhooker , @SwampDragons
I confirm the Windows build in your PR works fine for me!
Thank you for your reactivity on this!

[ikthorn]

@mwhooker not sure why it wouldn't work, but I confirm Darwin build works as well!

[SwampDragons]

\o/ 🎉 thanks for all of your quick turnaround on testing. This will make it into the 1.2.1 release.

[ashokei]

@mwhooker This issue is still seen in 1.2.1 for Linux x64. It works if i go back to 1.1.3.

[rickard-von-essen]

@ashokei please open a new issue with all details requested in the issue template.

[JustAMan]

@ashokei @rickard-von-essen
I've opened new issue #5986, though I'm experiencing it on Windows 10.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.