Packer: amazon-ebs: Error waiting for SSH: handshake failed: ssh: unable to authenticate

[c2cschools]

I noticed that this problem has occurred before in #130 , but I am not sure if this is the same issue.

I am attempting to build my images in stages. I go from the official release ami to a "raw" image, then from the "raw" image to a "bootstrapped" image.

official -> raw -> bootstrapped -> base

Where "raw" is basically a local copy of the official image, bootstrapped includes Puppet and Docker, and the base image is the result of a Puppet Apply run that installs and configures our commons.

I am creating AMIs/images for Ubuntu 12.04 and CentOS 6.4.

With Ubuntu, I have successfully completed the "raw" and "bootstrapped" images. With CentOS, the "raw" image builds fine (though I had to increase the ssh_timeout setting). However, when going from "raw" to "bootstrapped" with CentOS, it fails with:

==> amazon-ebs: Error waiting for SSH: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

It's worth noting that my "raw" build executes a shell script, but that shell script is _empty_, so no filesystem changes have occurred, that I am aware of, that could have caused this problem.

command

export PACKER_LOG=1
packer build packer-config.json

packer-config.json

{    
    "builders": [
        {
            "type": "amazon-ebs",
            "access_key": "...",
            "secret_key": "...",
            "region": "us-east-1",
            "source_ami": "ami-8bf0c4e2",
            "instance_type": "m1.small",
            "ssh_username": "root",
            "ami_name": "c2c-puppet-docker-centos-6.4 {{isotime | clean_ami_name}}",
            "ami_description": "C2C Bootstrapped Image for CentOS 6.4 - Includes Puppet and Docker - Built with Packer.io",
            "ssh_timeout" : "10m"
        }
    ],

    "provisioners": [
        {
            "type": "shell",
            "scripts": [
                "scripts/docker.sh",
                "scripts/puppet.sh"
            ],
            "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E '{{ .Path }}'"
        }
    ],
    "post-processors": [
        {
            "type": "vagrant",
            "keep_input_artifact": true
        }
    ]
}

output

2014/01/04 11:45:37 Packer Version: 0.5.1  eeefde8fc71444fdfc77711dc0e2789fa9b3b7d2
2014/01/04 11:45:37 Packer Target OS/Arch: linux amd64
2014/01/04 11:45:37 Built with Go Version: go1.2
2014/01/04 11:45:37 Detected home directory from env var: /home/ubuntu
2014/01/04 11:45:37 Attempting to open config file: /home/ubuntu/.packerconfig
2014/01/04 11:45:37 File doesn't exist, but doesn't need to. Ignoring.
2014/01/04 11:45:37 Packer config: &{PluginMinPort:0 PluginMaxPort:0 Builders:map[amazon-ebs:packer-builder-amazon-ebs amazon-chroot:packer-builder-amazon-chroot docker:packer-builder-docker openstack:packer-builder-openstack virtualbox-iso:packer-builder-virtualbox-iso virtualbox-ovf:packer-builder-virtualbox-ovf amazon-instance:packer-builder-amazon-instance digitalocean:packer-builder-digitalocean googlecompute:packer-builder-googlecompute qemu:packer-builder-qemu vmware-iso:packer-builder-vmware-iso vmware-vmx:packer-builder-vmware-vmx] Commands:map[build:packer-command-build fix:packer-command-fix inspect:packer-command-inspect validate:packer-command-validate] PostProcessors:map[vagrant:packer-post-processor-vagrant vsphere:packer-post-processor-vsphere] Provisioners:map[ansible-local:packer-provisioner-ansible-local chef-solo:packer-provisioner-chef-solo file:packer-provisioner-file puppet-masterless:packer-provisioner-puppet-masterless shell:packer-provisioner-shell salt-masterless:packer-provisioner-salt-masterless]}
2014/01/04 11:45:37 Setting cache directory: /home/ubuntu/c2c-virtual-env/_build/packer/aws/centos/6.4/bootstrapped/packer_cache
2014/01/04 11:45:37 Environment.Cli: []string{"build", "packer-config.json"}
2014/01/04 11:45:37 command + args: []string{"build", "packer-config.json"}
2014/01/04 11:45:37 Loading command: build
2014/01/04 11:45:37 Creating plugin client for path: /usr/local/packer/packer-command-build
2014/01/04 11:45:37 Starting plugin: /usr/local/packer/packer-command-build []string{"/usr/local/packer/packer-command-build"}
2014/01/04 11:45:37 Waiting for RPC address for: /usr/local/packer/packer-command-build
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Plugin build against Packer 'eeefde8fc71444fdfc77711dc0e2789fa9b3b7d2'
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Plugin minimum port: 10000
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Plugin maximum port: 25000
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Plugin address: unix /tmp/packer-plugin419864228
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Waiting for connection...
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Serving a plugin connection...
2014/01/04 11:45:37 Executing command: build
2014/01/04 11:45:37 Loading builder: amazon-ebs
2014/01/04 11:45:37 Creating plugin client for path: /usr/local/packer/packer-builder-amazon-ebs
2014/01/04 11:45:37 Starting plugin: /usr/local/packer/packer-builder-amazon-ebs []string{"/usr/local/packer/packer-builder-amazon-ebs"}
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Reading template: packer-config.json
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Creating build: amazon-ebs
2014/01/04 11:45:37 Waiting for RPC address for: /usr/local/packer/packer-builder-amazon-ebs
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Plugin build against Packer 'eeefde8fc71444fdfc77711dc0e2789fa9b3b7d2'
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Plugin minimum port: 10000
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Plugin maximum port: 25000
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Plugin address: unix /tmp/packer-plugin295778018
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Waiting for connection...
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Serving a plugin connection...
2014/01/04 11:45:37 Loading post-processor: vagrant
2014/01/04 11:45:37 Creating plugin client for path: /usr/local/packer/packer-post-processor-vagrant
2014/01/04 11:45:37 Starting plugin: /usr/local/packer/packer-post-processor-vagrant []string{"/usr/local/packer/packer-post-processor-vagrant"}
2014/01/04 11:45:37 Waiting for RPC address for: /usr/local/packer/packer-post-processor-vagrant
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Plugin build against Packer 'eeefde8fc71444fdfc77711dc0e2789fa9b3b7d2'
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Plugin minimum port: 10000
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Plugin maximum port: 25000
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Plugin address: unix /tmp/packer-plugin962302759
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Waiting for connection...
2014/01/04 11:45:37 /usr/local/packer/packer-post-processor-vagrant: 2014/01/04 11:45:37 Serving a plugin connection...
2014/01/04 11:45:37 Loading provisioner: shell
2014/01/04 11:45:37 Creating plugin client for path: /usr/local/packer/packer-provisioner-shell
2014/01/04 11:45:37 Starting plugin: /usr/local/packer/packer-provisioner-shell []string{"/usr/local/packer/packer-provisioner-shell"}
2014/01/04 11:45:37 Waiting for RPC address for: /usr/local/packer/packer-provisioner-shell
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Plugin build against Packer 'eeefde8fc71444fdfc77711dc0e2789fa9b3b7d2'
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Plugin minimum port: 10000
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Plugin maximum port: 25000
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Plugin address: unix /tmp/packer-plugin536453881
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Waiting for connection...
2014/01/04 11:45:37 /usr/local/packer/packer-provisioner-shell: 2014/01/04 11:45:37 Serving a plugin connection...
2014/01/04 11:45:37 ui: amazon-ebs output will be in this color.
amazon-ebs output will be in this color.
2014/01/04 11:45:37 ui:

2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Build debug mode: false
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Force build: false
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Preparing build: amazon-ebs
2014/01/04 11:45:37 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:37 Config: {PackerConfig:{PackerBuildName:amazon-ebs PackerBuilderType:amazon-ebs PackerDebug:false PackerForce:false PackerUserVars:map[]} AccessConfig:{AccessKey:<Filtered> SecretKey:<Filtered> RawRegion:us-east-1} AMIConfig:{AMIName:c2c-puppet-docker-centos-6.4 2014-01-04T11-45-37Z AMIDescription:C2C Bootstrapped Image for CentOS 6.4 - Includes Puppet and Docker - Built with Packer.io AMIUsers:[] AMIGroups:[] AMIProductCodes:[] AMIRegions:[] AMITags:map[]} BlockDevices:{AMIMappings:[] LaunchMappings:[]} RunConfig:{AssociatePublicIpAddress:false AvailabilityZone: IamInstanceProfile: InstanceType:m1.small RunTags:map[] SourceAmi:ami-8bf0c4e2 RawSSHTimeout:10m SSHUsername:root SSHPort:22 SecurityGroupId: SecurityGroupIds:[] SubnetId: TemporaryKeyPairName:packer 52c7f461-5531-19cd-3cc0-f189b99a296f UserData: UserDataFile: VpcId: sshTimeout:600000000000} tpl:0xc2100c3c20}
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Waiting on builds to complete...
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Starting build run: amazon-ebs
2014/01/04 11:45:37 /usr/local/packer/packer-command-build: 2014/01/04 11:45:37 Running builder: amazon-ebs
==> amazon-ebs: Creating temporary keypair: packer 52c7f461-5531-19cd-3cc0-f189b99a296f
2014/01/04 11:45:37 ui: ==> amazon-ebs: Creating temporary keypair: packer 52c7f461-5531-19cd-3cc0-f189b99a296f
==> amazon-ebs: Creating temporary security group for this instance...
2014/01/04 11:45:38 ui: ==> amazon-ebs: Creating temporary security group for this instance...
2014/01/04 11:45:38 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:38 Temporary group name: packer 52c7f462-da4e-aefb-5c6f-05d7469d5b5a
==> amazon-ebs: Authorizing SSH access on the temporary security group...
2014/01/04 11:45:38 ui: ==> amazon-ebs: Authorizing SSH access on the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
2014/01/04 11:45:39 ui: ==> amazon-ebs: Launching a source AWS instance...
    amazon-ebs: Instance ID: i-7428bc54
2014/01/04 11:45:39 ui:     amazon-ebs: Instance ID: i-7428bc54
==> amazon-ebs: Waiting for instance (i-7428bc54) to become ready...
2014/01/04 11:45:39 ui: ==> amazon-ebs: Waiting for instance (i-7428bc54) to become ready...
2014/01/04 11:45:39 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:45:39 Waiting for state to become: running
2014/01/04 11:46:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:11 Waiting for SSH, up to timeout: 10m0s
==> amazon-ebs: Waiting for SSH to become available...
2014/01/04 11:46:11 ui: ==> amazon-ebs: Waiting for SSH to become available...
2014/01/04 11:46:19 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:19 TCP connection to SSH ip/port failed: dial tcp 10.180.230.59:22: no route to host
2014/01/04 11:46:28 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:28 TCP connection to SSH ip/port failed: dial tcp 10.180.230.59:22: no route to host
2014/01/04 11:46:33 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:33 TCP connection to SSH ip/port failed: dial tcp 10.180.230.59:22: connection refused
2014/01/04 11:46:38 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:38 Attempting SSH connection...
2014/01/04 11:46:38 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:38 reconnecting to TCP connection for SSH
2014/01/04 11:46:38 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:38 handshaking with SSH
2014/01/04 11:46:39 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:39 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:39 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:39 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:39 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:39 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 Attempting SSH connection...
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 reconnecting to TCP connection for SSH
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 handshaking with SSH
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:44 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:44 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 Attempting SSH connection...
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 reconnecting to TCP connection for SSH
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 handshaking with SSH
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:50 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:50 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 Attempting SSH connection...
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 reconnecting to TCP connection for SSH
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 handshaking with SSH
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:46:55 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:46:55 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 Attempting SSH connection...
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 reconnecting to TCP connection for SSH
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 handshaking with SSH
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:00 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:00 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 Attempting SSH connection...
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 reconnecting to TCP connection for SSH
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 handshaking with SSH
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:06 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:06 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 Attempting SSH connection...
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 reconnecting to TCP connection for SSH
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 handshaking with SSH
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:11 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:11 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:16 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:16 Attempting SSH connection...
2014/01/04 11:47:16 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:16 reconnecting to TCP connection for SSH
2014/01/04 11:47:16 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:16 handshaking with SSH
2014/01/04 11:47:17 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:17 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:17 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:17 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:17 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:17 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 Attempting SSH connection...
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 reconnecting to TCP connection for SSH
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 handshaking with SSH
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:22 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:22 Detected authentication error. Increasing handshake attempts.
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 Attempting SSH connection...
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 reconnecting to TCP connection for SSH
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 handshaking with SSH
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 handshake error: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 SSH handshake err: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:27 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:27 Detected authentication error. Increasing handshake attempts.
==> amazon-ebs: Error waiting for SSH: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2014/01/04 11:47:27 ui error: ==> amazon-ebs: Error waiting for SSH: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
==> amazon-ebs: Terminating the source AWS instance...
2014/01/04 11:47:27 ui: ==> amazon-ebs: Terminating the source AWS instance...
2014/01/04 11:47:28 /usr/local/packer/packer-builder-amazon-ebs: 2014/01/04 11:47:28 Waiting for state to become: terminated
==> amazon-ebs: Deleting temporary security group...
2014/01/04 11:48:16 ui: ==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
2014/01/04 11:48:16 ui: ==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.
2014/01/04 11:48:16 ui: Build 'amazon-ebs' finished.
2014/01/04 11:48:16 /usr/local/packer/packer-command-build: 2014/01/04 11:48:16 Builds completed. Waiting on interrupt barrier...

==> Builds finished. The artifacts of successful builds are:
2014/01/04 11:48:16 ui:
==> Builds finished. The artifacts of successful builds are:
2014/01/04 11:48:16 machine readable: amazon-ebs,artifact-count []string{"0"}
2014/01/04 11:48:16 waiting for all plugin processes to complete...
2014/01/04 11:48:16 /usr/local/packer/packer-post-processor-vagrant: plugin process exited
2014/01/04 11:48:16 /usr/local/packer/packer-provisioner-shell: plugin process exited
2014/01/04 11:48:16 [ERR] Error reading stream direction: EOF
2014/01/04 11:48:16 [INFO] Mux connection loop exiting
2014/01/04 11:48:16 [ERR] Error reading stream direction: EOF
2014/01/04 11:48:16 [INFO] Mux connection loop exiting
2014/01/04 11:48:16 [ERR] Error reading stream direction: read unix /tmp/packer-plugin419864228: connection reset by peer
2014/01/04 11:48:16 [INFO] Mux connection loop exiting
2014/01/04 11:48:16 /usr/local/packer/packer-command-build: plugin process exited
2014/01/04 11:48:16 [ERR] Error reading stream direction: EOF
2014/01/04 11:48:16 [INFO] Mux connection loop exiting
2014/01/04 11:48:16 /usr/local/packer/packer-builder-amazon-ebs: plugin process exited

[intabulas]

I am also seeing this today

[tomasca]

Also seeing the same thing. Tried with packer 0.5.1 today. Had same thing with 0.4.1 before.

Even when launching the Packer-created "raw" image manually via EC2 console it is not possible to manually login with SSH and keypair. It asks for password.

Only issue with CentOS. Works as expected with Ubuntu and RHEL.

[kevinmartin]

I get the same error. Using CentOS.

[shahab32mohsen]

Same here, I get the same error using CentOs. I can successfully create the Virtual network, Cloud service, VM, Disk, .... but at the end before running provisioners it fails with exactly same error! Has anybody found a solution yet?:)

[inbravo]

The same issue is with me. Tried with,

1. Medium instance of Ubuntu (ubuntu-precise-12.04-amd64-server-20120616-5cc062dc-4b61-4424-bc43-416e51483a1f-ami-82fa58eb.1 (ami-967edcff))
2. Received this error: Error waiting for SSH: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

[ddeboer]

Same error message here with a CentOS 6.5 image.

[dsummersl]

Same here, Amazon Linux AMI ami-bba18dd2

[mwedgwood-rmn]

The problem is (arguably) the timing of when the vanilla CentOS image gets its ssh keypair from the metadata server. The bit of code that does it is in rc.local, and that runs _after_ SSH starts up. Depending on when packer tries to authenticate, it might catch a listening SSH daemon but no authorized key for the root user.

It would be awesome for packer to optionally retry failed SSH authentication.

[felin-arch]

I just ran into this problem and spent an hour trying to figure out what the hack is happening. Here is my scenario which is pretty similar to yours. I wanted to set up a minimal environment in amazon based on CentOS 6. All I wanted is a fully patched AMI so I could start provisioning on top of that. I ran into the same handshake failure problem.

Here is what is happening:
The original CentOS ami has a script (/etc/rc.d/rc.local) which fetches the public-key from amazon and sets it up for the root account. This script only tries to fetch the keys if the root account does not have any associated keys with it (/root/.ssh/authorized_keys does not exist). This is the case with the base image, but once you "repackage" it, it will hold the initial public keys (fetched from amazon during this "provisioning" session), hence when you boot up your own AMI the script will not fetch the keys from amazon (because root already has keys).

I simply did an "rm -rf /root/.ssh/authorized_keys" after patching up the AMI. In my case this had nothing to do with packer, it was the CentOS key fetching script which held the solution.

[blandonnimrat]

@felin-arch - thank you so much for writing that up. My hair is (somewhat) intact because of your gentlemanly contribution.

[mwedgwood-rmn]

@felin-arch, this does indeed fix one case of the problem described by the OP, but the problem also exists when going from official -> raw. If @mitchellh considers this one closed, I can file a new bug detailing the issue from the official image.

My workaround was to build a new base image by installing into a chroot and configuring cloud-init to handle the key setup, removing the bits in the dist rc.local.

[i-sam]

@felin-arch @mwedgwood-rmn Guys, so is the problem was solved in somehow, still see the same problem with Centos6.5.
"==> amazon-ebs: Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain"
Or just one way - workaround with manual rebuilding base ami?

[felin-arch]

@i-sam,
_short_: this is not an issue with packer. The above described two workarounds can be used to avoid this behavior.
_long_: I do not think packer can do anything reasonable to solve this issue. Packer works as it should, the inconsistency comes from the way the official images fetch keys. There are two workarounds above, both essentially tweak the key fetching method. As @mwedgwood-rmn, described he handles key setup in cloud-init removing the the original key setup code from rc.local. I just removed the root key right before saving the AMI so the original rc.local script would fetch it again on the next boot.

[i-sam]

@felin-arch Ok, thank you for fast answer. Got it.

[mitchellh]

I agree with @felin-arch. Sorry guys.

[igmar]

I think the actual cause is different : The "provided" AMI's thet AWS offers add the key to the ec2-user user. That is kinda hardcoded, and enforced by cloud-init script.

For me, adding "ssh_username": "ec2-user" made it work (on a CentOS AMI), ubuntu is needed when using an Ubuntu based AMI.
In both cases, root will not work.

[kyptin]

I can confirm @igmar's solution, which seems like the right one to me. Even apart from Packer, trying to ssh into a CentOS-based AMI as user ubuntu will time out, but changing the user to ec2-user succeeds.

[jjfeiler]

For reference, the contents of /etc/rc.d/rc.local on the official CentOS 6.5 image are as follows:

$ cat /etc/rc.d/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# set a random pass on first boot
if [ -f /root/firstrun ]; then 
  dd if=/dev/urandom count=50|md5sum|passwd --stdin root
  passwd -l root
  rm /root/firstrun
fi

if [ ! -d /root/.ssh ]; then
  mkdir -m 0700 -p /root/.ssh
  restorecon /root/.ssh
fi
# Get the root ssh key setup
ReTry=0
while [ ! -f /root/.ssh/authorized_keys ] && [ $ReTry -lt 10 ]; do
  sleep 2
  curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /root/.ssh/pubkey
  if [ 0 -eq 0 ]; then
    mv /root/.ssh/pubkey /root/.ssh/authorized_keys
  fi
  ReTry=$[Retry+1]
done
chmod 600 /root/.ssh/authorized_keys && restorecon /root/.ssh/authorized_keys

[hyperfocus1338]

Does rebuilding the official Ubuntu base image with "rm -rf /root/.ssh/authorized_keys" also work as a workaround to this issue?

[felin-arch]

@hyperfocus1337 I believe so. That is basically what I did.

[hyperfocus1338]

I'll try it out and report back.

[hyperfocus1338]

Unfortunately didn't work out for me. I made a ec2 instance manually with the default Ubuntu 14.04 AMI (ami-9eaa1cf6), and then I logged in through ssh and ran the command "rm -rf /root/.ssh/authorized_keys". After that I saved a snapshot.

Running the template again with the self created source ami and ebs snapshot gives me the same problem. I have shared my packer template here: https://github.com/hyperfocus1337/packer-amazon-ebs-template

Any hints on how I can further troubleshoot this issue?

[felin-arch]

You are trying to ssh to the machines as the user ubuntu. You removed the keys for the root user. You have to check the exact way Ubuntu fetches keys and alter your method.

I do not know how Ubuntu fetches keys, but I guess it does the same thing as CentOS. The only difference is that it adds the keys under the ubuntu user instead of root.

[hyperfocus1338]

This time I removed both the authorized_keys file for the root and ubuntu user which didn't work. Also only removing it for the Ubuntu user didn't work. Also tried using a private ssh key, can't even manually login with it.

Is there another way to make the amazon-ebs builder work with Ubuntu? Where can I find the Ubuntu version of the "/etc/rc.d/rc.local" script on CentOS 6.5? Is my "ami_block_device_mappings" setting correctly set on my template, I'm not sure if I fully understand it.

And once Packer automatically creates an ssh key, where does it store it for later access? The documentation only mentions the -debug flag. Doesn't sound like it's the only way to retrieve it.

[felin-arch]

@hyperfocus1337, You will have to check what Ubuntu does to fetch the keys. As I remember when you use -debug it saves the key to the current directory (I think it also echoed the key location to the console).

[hyperfocus1338]

Thanks once again for fast response. Can you elaborate a little bit more on "what Ubuntu does to fetch keys"? Tried to research it but I don't know where to start. What files/directories should I look into?

Should I look through Upstart or Systemd files? Since Ubuntu doesn't use the init system like CentOS.

[felin-arch]

When you fire up an EC2 instance you can specify a key you want to use to access that machine. The VM needs to set up this key before you can access it. Various distributions use different ways of fetching the correct public key from AWS.
One thing is common though: they all fetch it from http://169.254.169.254... This is an internal address available to all VMs running on EC2. Ubuntu seems to use the Amazon provided "cloud-init" method to fetch keys (this is just guessing). Basically you are looking for something that only runs at startup.

Yes init.d would be a good start. Cloud-init must have some documentation that should help you.

[hyperfocus1338]

I think this is the script, it's located at /usr/lib/cloud-init/write-ssh-key-fingerprints. A list of all cloud init files on Ubuntu 14.04 is located here: http://packages.ubuntu.com/trusty/all/cloud-init/filelist.

#!/bin/sh

logger_opts="-p user.info -t ec2"

# rhels' version of logger_opts does not support long
# for of -s (--stderr), so use short form.
logger_opts="$logger_opts -s"

# Redirect stderr to stdout
exec 2>&1

fp_blist=",${1},"
key_blist=",${2},"
{
echo
echo "#############################################################"
echo "-----BEGIN SSH HOST KEY FINGERPRINTS-----"
for f in /etc/ssh/ssh_host_*key.pub; do
    [ -f "$f" ] || continue
    read ktype line < "$f"
    # skip the key if its type is in the blacklist
    [ "${fp_blist#*,$ktype,}" = "${fp_blist}" ] || continue
    ssh-keygen -l -f "$f"
done
echo "-----END SSH HOST KEY FINGERPRINTS-----"
echo "#############################################################"

} | logger $logger_opts

echo "-----BEGIN SSH HOST KEY KEYS-----"
for f in /etc/ssh/ssh_host_*key.pub; do
    [ -f "$f" ] || continue
    read ktype line < "$f"
    # skip the key if its type is in the blacklist
    [ "${key_blist#*,$ktype,}" = "${key_blist}" ] || continue
    cat $f
done
echo "-----END SSH HOST KEY KEYS-----"

Another cloud init configuration option which might be interesting is: https://cloudinit.readthedocs.org/en/latest/topics/examples.html#configure-instances-ssh-keys. There are also three ssh modules: https://cloudinit.readthedocs.org/en/latest/topics/modules.html#ssh.

I'm not good with scripting languages and maybe too inexperienced to figure this out myself. But I'll try to make sense of it and report back. But in the meantime can someone help me verify what it does and help determine what steps I should take to resolve this issue?

[felin-arch]

The script you posted above does not do any key fetching. I googled around and you may need to use the ec2-user to log in. Before trying to automate things, you may want to do all the steps manually to make sure you understand what is happening when packer is in play.

[hyperfocus1338]

What do you mean by the ec2-user? Your Amazon IAM username? Login with that in the Packer template? Tried it with:

• Ec2 user (my Amazon IAM username) and custom defined Amazon auto-generated private_key_file
• Ec2 user without private_key_file
• Ec2 user with default Ubuntu 14.04 AMI: ami-9eaa1cf6

Unfortunately didn't work yet. Clueless on how to proceed.

@sethvargo @mitchellh The amazon-ebs builder is not much use for Ubuntu users at this time without any clear instructions on how to make the SSH login work, even though it's not an issue on the Packer side. And I'm guessing many users use both EC2 and Ubuntu. Would love to be able to have this integrated with Atlas.

This is my template: https://github.com/hyperfocus1337/packer-amazon-ebs-template

[dirtybirdnj]

@felin-arch This was the fix for an issue I was having, thank you!

[EliasGoldberg]

for CentOS 7 ssh_username must be set to centos to avoid the error message:

Build 'amazon-ebs' errored: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

[justintilson]

@EliasGoldberg's solution worked for me.

[alfonso-cabrera]

@EliasGoldberg 's solution also worked for me on CentOS 7.

[andreabolandrina]

ssh_username set as centos worked for me too (on CentOs 7)

[gvenkat]

I can confirm that @EliasGoldberg's solution worked for me. Thanks a bunch.

[shyam2j]

Yes it woked for me also with ubuntu user. was giving error with default template

[davidmcl]

Setting the 'correct' user fixed the error for me as well - thanks to all.

FYI - there is a list of SSH users based on the instance type in a Tip at: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html

here is the tip verbiage at this point...

Tip
For Amazon Linux, the user name is ec2-user. For RHEL, the user name is ec2-user or root. For Ubuntu, the user name is ubuntu or root. For Centos, the user name is centos. For Fedora, the user name is ec2-user. For SUSE, the user name is ec2-user or root. Otherwise, if ec2-user and root don't work, check with your AMI provider.

dbm

[codekipple]

Interestingly I was hitting this problem and the cause was the name I was using in the source_ami_filter.
See below I was using Base-* but in fact the base AMI I wanted to build off was just called base-1511969253.

"builders": [{
  "source_ami_filter": {
    "filters": {
      "virtualization-type": "hvm",
      "name": "Base-*",
      "root-device-type": "ebs"
    },
    "most_recent": true
  }
}]

If I got the name completely wrong than it would fail early saying it could not find the AMI. But using the name with the wrong case seemed to allow it to find the AMI but then fail at the SSH connection part triggering this error:- Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain.

I was using ubuntu/images/*ubuntu-xenial-16.04-amd64-server-* to create the base image. Not sure if that's relevant. Could be that this case sensitive behaviour affects non ubuntu AMI's

[SwampDragons]

@codekipple that's really interesting... maybe we should add a line to the docs stating that the ami name is case sensitive.

[rickard-von-essen]

@codekipple the biggest problem with that filter is that you _don't_ specify owner or the AMI, so you get the latest public AMI which matches your filter. "Always" specify owners.

[codekipple]

@rickard-von-essen ah ok, I'm new to packer and I'm trying to make a system to create AMI's in 4 different AWS accounts. I omitted the owners on purpose so it worked for all accounts and I didn't have to keep track of the owners ID.

[alirezaAbdi]

I solved it by not forgetting to add the public_key_pair (of the aws account)
I had the same error message "amazon-ebs: Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain" and problem was that I had forgotten to provide SSH_KEYPAIR_NAME to the dockerized Packer. After adding kaypair name, it works now!

[Mistawes]

FWIW, I commented the original 'ec2-user' username and kept getting this error due to how it's pulled from .kitchen.yml. You should only have one reference to 'username'!

[santoshr1016]

"builders" : [
        {
            "type" : "amazon-ebs",
            "profile" : "your-aws-profile",
            "region" : "{{user `region`}}",
            "instance_type" : "t2.micro",
            "source_ami" : "ami-XXXX",
            "communicator": "ssh",
            "ssh_username" : "ubuntu",
            "ssh_keypair_name": "XXXX",
            "ssh_private_key_file": "/path/to/XXXX.pem",
            "ami_name" : "Ubuntu-Sample-AMI",
            "ami_description" : "Some message",
            "run_tags" : {
                "Name" : "Hello World",
                "Tool" : "Packer",
                "Author" : "XXXX"
            }
        }
  ]

It is very important to give correct username for different linux flavors, which are.
RHEL: ec2-user
Amazon Linux: ec2-user
CentOS: centos
Ubuntu: ubuntu

I hope this saves your time!!!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.