Packer: ami_block_device_mappings is not work

Created on 31 Oct 2014 · 6Comments · Source: hashicorp/packer

ephemeral disk not available, but i could be aws cli.
is a bug??

$ packer --version
Packer v0.7.2
$ uname -v
Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64

{
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "<your access_key>",
    "secret_key": "<your secret_key>",
    "region": "ap-northeast-1",
    "source_ami": "ami-b1fe9bb0",
    "ami_virtualization_type": "paravirtual",
    "instance_type": "c3.large",
    "ssh_username": "ec2-user",
    "ssh_timeout": "5m",
    "ami_block_device_mappings": [
      { "device_name": "/dev/sdb", "virtual_name": "ephemeral0" },
      { "device_name": "/dev/sdc", "virtual_name": "ephemeral1" }
    ]
    "vpc_id": "<your vpc id>",
    "subnet_id": "<your subnet id>",
    "security_group_id": "<your security group id>",
    "associate_public_ip_address": true
  }]
}

aws ec2 run-instances \
  --image-id ami-b1fe9bb0 --count 1 --instance-type c3.large \
  --key-name <your key name> --security-group-ids <your security group id> --subnet-id <your subnet id> --associate-public-ip-address \
  --block-device-mappings '[{"DeviceName":"/dev/sdb","VirtualName":"ephemeral0"},{"DeviceName":"/dev/sdc","VirtualName":"ephemeral1"}]'

$ PACKER_LOG=1 packer build -debug test.json 
2014/10/31 10:17:10 Packer Version: 0.7.2  
2014/10/31 10:17:10 Packer Target OS/Arch: darwin amd64
2014/10/31 10:17:10 Built with Go Version: go1.3.3
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: amazon-chroot = /usr/local/bin/packer-builder-amazon-chroot
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: amazon-ebs = /usr/local/bin/packer-builder-amazon-ebs
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: amazon-instance = /usr/local/bin/packer-builder-amazon-instance
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: digitalocean = /usr/local/bin/packer-builder-digitalocean
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: docker = /usr/local/bin/packer-builder-docker
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: googlecompute = /usr/local/bin/packer-builder-googlecompute
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: null = /usr/local/bin/packer-builder-null
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: openstack = /usr/local/bin/packer-builder-openstack
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: parallels-iso = /usr/local/bin/packer-builder-parallels-iso
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: parallels-pvm = /usr/local/bin/packer-builder-parallels-pvm
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: qemu = /usr/local/bin/packer-builder-qemu
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: virtualbox-iso = /usr/local/bin/packer-builder-virtualbox-iso
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: virtualbox-ovf = /usr/local/bin/packer-builder-virtualbox-ovf
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: vmware-iso = /usr/local/bin/packer-builder-vmware-iso
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: vmware-vmx = /usr/local/bin/packer-builder-vmware-vmx
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: build = /usr/local/bin/packer-command-build
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: fix = /usr/local/bin/packer-command-fix
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: inspect = /usr/local/bin/packer-command-inspect
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: validate = /usr/local/bin/packer-command-validate
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: compress = /usr/local/bin/packer-post-processor-compress
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: docker-import = /usr/local/bin/packer-post-processor-docker-import
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: docker-push = /usr/local/bin/packer-post-processor-docker-push
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: docker-save = /usr/local/bin/packer-post-processor-docker-save
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: docker-tag = /usr/local/bin/packer-post-processor-docker-tag
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: vagrant = /usr/local/bin/packer-post-processor-vagrant
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: vagrant-cloud = /usr/local/bin/packer-post-processor-vagrant-cloud
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: vsphere = /usr/local/bin/packer-post-processor-vsphere
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: ansible-local = /usr/local/bin/packer-provisioner-ansible-local
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: chef-client = /usr/local/bin/packer-provisioner-chef-client
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: chef-solo = /usr/local/bin/packer-provisioner-chef-solo
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: file = /usr/local/bin/packer-provisioner-file
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: puppet-masterless = /usr/local/bin/packer-provisioner-puppet-masterless
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: puppet-server = /usr/local/bin/packer-provisioner-puppet-server
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: salt-masterless = /usr/local/bin/packer-provisioner-salt-masterless
2014/10/31 10:17:10 [DEBUG] Discoverd plugin: shell = /usr/local/bin/packer-provisioner-shell
2014/10/31 10:17:10 Detected home directory from env var: /Users/iwai
2014/10/31 10:17:10 Detected home directory from env var: /Users/iwai
2014/10/31 10:17:10 Attempting to open config file: /Users/iwai/.packerconfig
2014/10/31 10:17:10 File doesn't exist, but doesn't need to. Ignoring.
2014/10/31 10:17:10 Packer config: &{DisableCheckpoint:false DisableCheckpointSignature:false PluginMinPort:10000 PluginMaxPort:25000 Builders:map[qemu:/usr/local/bin/packer-builder-qemu virtualbox-ovf:/usr/local/bin/packer-builder-virtualbox-ovf amazon-chroot:/usr/local/bin/packer-builder-amazon-chroot digitalocean:/usr/local/bin/packer-builder-digitalocean docker:/usr/local/bin/packer-builder-docker null:/usr/local/bin/packer-builder-null parallels-pvm:/usr/local/bin/packer-builder-parallels-pvm vmware-vmx:/usr/local/bin/packer-builder-vmware-vmx amazon-ebs:/usr/local/bin/packer-builder-amazon-ebs amazon-instance:/usr/local/bin/packer-builder-amazon-instance parallels-iso:/usr/local/bin/packer-builder-parallels-iso virtualbox-iso:/usr/local/bin/packer-builder-virtualbox-iso vmware-iso:/usr/local/bin/packer-builder-vmware-iso googlecompute:/usr/local/bin/packer-builder-googlecompute openstack:/usr/local/bin/packer-builder-openstack] Commands:map[build:/usr/local/bin/packer-command-build fix:/usr/local/bin/packer-command-fix inspect:/usr/local/bin/packer-command-inspect validate:/usr/local/bin/packer-command-validate] PostProcessors:map[compress:/usr/local/bin/packer-post-processor-compress docker-import:/usr/local/bin/packer-post-processor-docker-import docker-push:/usr/local/bin/packer-post-processor-docker-push docker-save:/usr/local/bin/packer-post-processor-docker-save docker-tag:/usr/local/bin/packer-post-processor-docker-tag vagrant:/usr/local/bin/packer-post-processor-vagrant vagrant-cloud:/usr/local/bin/packer-post-processor-vagrant-cloud vsphere:/usr/local/bin/packer-post-processor-vsphere] Provisioners:map[shell:/usr/local/bin/packer-provisioner-shell ansible-local:/usr/local/bin/packer-provisioner-ansible-local chef-client:/usr/local/bin/packer-provisioner-chef-client chef-solo:/usr/local/bin/packer-provisioner-chef-solo file:/usr/local/bin/packer-provisioner-file puppet-masterless:/usr/local/bin/packer-provisioner-puppet-masterless puppet-server:/usr/local/bin/packer-provisioner-puppet-server salt-masterless:/usr/local/bin/packer-provisioner-salt-masterless]}
2014/10/31 10:17:10 Setting cache directory: /Users/iwai/Work/jswf-test-deploy-config/packer_cache
2014/10/31 10:17:10 Environment.Cli: []string{"build", "-debug", "test.json"}
2014/10/31 10:17:10 command + args: []string{"build", "-debug", "test.json"}
2014/10/31 10:17:10 Loading command: build
2014/10/31 10:17:10 Creating plugin client for path: /usr/local/bin/packer-command-build
2014/10/31 10:17:10 Starting plugin: /usr/local/bin/packer-command-build []string{"/usr/local/bin/packer-command-build"}
2014/10/31 10:17:10 Detected home directory from env var: /Users/iwai
2014/10/31 10:17:10 Waiting for RPC address for: /usr/local/bin/packer-command-build
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Plugin build against Packer ''
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Plugin minimum port: 10000
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Plugin maximum port: 25000
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Plugin address: unix /var/folders/4f/zv6g6bn15sz5x2xrfyc605l80000gn/T/packer-plugin611235164
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Waiting for connection...
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Serving a plugin connection...
2014/10/31 10:17:10 Executing command: build
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Reading template: test.json
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Creating build: amazon-ebs
2014/10/31 10:17:10 Loading builder: amazon-ebs
2014/10/31 10:17:10 Creating plugin client for path: /usr/local/bin/packer-builder-amazon-ebs
2014/10/31 10:17:10 Starting plugin: /usr/local/bin/packer-builder-amazon-ebs []string{"/usr/local/bin/packer-builder-amazon-ebs"}
2014/10/31 10:17:10 Waiting for RPC address for: /usr/local/bin/packer-builder-amazon-ebs
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Plugin build against Packer ''
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Plugin minimum port: 10000
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Plugin maximum port: 25000
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Plugin address: unix /var/folders/4f/zv6g6bn15sz5x2xrfyc605l80000gn/T/packer-plugin407523055
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Waiting for connection...
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Serving a plugin connection...
Debug mode enabled. Builds will not be parallelized.
2014/10/31 10:17:10 ui: Debug mode enabled. Builds will not be parallelized.
amazon-ebs output will be in this color.
2014/10/31 10:17:10 ui: amazon-ebs output will be in this color.

2014/10/31 10:17:10 ui: 
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Build debug mode: true
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Force build: false
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Preparing build: amazon-ebs
2014/10/31 10:17:10 packer-builder-amazon-ebs: 2014/10/31 10:17:10 Config: {PackerConfig:{PackerBuildName:amazon-ebs PackerBuilderType:amazon-ebs PackerDebug:true PackerForce:false PackerUserVars:map[]} AccessConfig:{AccessKey:<Filtered> SecretKey:<Filtered> RawRegion:ap-northeast-1 Token:} AMIConfig:{AMIName:jswf-admovie-1414718230 AMIDescription: AMIVirtType:paravirtual AMIUsers:[] AMIGroups:[] AMIProductCodes:[] AMIRegions:[] AMITags:map[Name:Jswf Admovie 2014-10-31T01-17-10Z Environment:prd] AMIEnhancedNetworking:false} BlockDevices:{AMIMappings:[{DeleteOnTermination:false DeviceName:/dev/sdb Encrypted:false IOPS:0 NoDevice:false SnapshotId: VirtualName:ephemeral0 VolumeType: VolumeSize:0} {DeleteOnTermination:false DeviceName:/dev/sdc Encrypted:false IOPS:0 NoDevice:false SnapshotId: VirtualName:ephemeral1 VolumeType: VolumeSize:0}] LaunchMappings:[]} RunConfig:{AssociatePublicIpAddress:true AvailabilityZone: IamInstanceProfile: InstanceType:c3.large RunTags:map[] SourceAmi:ami-b1fe9bb0 SpotPrice: SpotPriceAutoProduct: RawSSHTimeout:5m SSHUsername:ec2-user SSHPrivateKeyFile: SSHPrivateIp:false SSHPort:22 SecurityGroupId: SecurityGroupIds:[sg-4d071d2f] SubnetId:subnet-24b2b450 TemporaryKeyPairName:packer 5452e316-8356-4112-2882-27fc918e817a UserData: UserDataFile: VpcId:vpc-4dadb02f sshTimeout:300000000000} tpl:0xc20811e0a0}
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Debug enabled, so waiting for build to finish: amazon-ebs
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Starting build run: amazon-ebs
2014/10/31 10:17:10 packer-command-build: 2014/10/31 10:17:10 Running builder: amazon-ebs
==> amazon-ebs: Inspecting the source AMI...
2014/10/31 10:17:10 ui: ==> amazon-ebs: Inspecting the source AMI...
==> amazon-ebs: Pausing after run of step 'StepSourceAMIInfo'. Press enter to continue. 2014/10/31 10:17:11 ui: ask: ==> amazon-ebs: Pausing after run of step 'StepSourceAMIInfo'. Press enter to continue.

2014/10/31 10:17:27 ui: scan err: unexpected newline
==> amazon-ebs: Creating temporary keypair: packer 5452e316-8356-4112-2882-27fc918e817a
2014/10/31 10:17:27 ui: ==> amazon-ebs: Creating temporary keypair: packer 5452e316-8356-4112-2882-27fc918e817a
    amazon-ebs: Saving key for debug purposes: ec2_amazon-ebs.pem
2014/10/31 10:17:27 ui:     amazon-ebs: Saving key for debug purposes: ec2_amazon-ebs.pem
==> amazon-ebs: Pausing after run of step 'StepKeyPair'. Press enter to continue. 2014/10/31 10:17:27 ui: ask: ==> amazon-ebs: Pausing after run of step 'StepKeyPair'. Press enter to continue.

2014/10/31 10:17:43 ui: scan err: unexpected newline
2014/10/31 10:17:43 packer-builder-amazon-ebs: 2014/10/31 10:17:43 Using specified security groups: [sg-4d071d2f]
==> amazon-ebs: Pausing after run of step 'StepSecurityGroup'. Press enter to continue. 2014/10/31 10:17:43 ui: ask: ==> amazon-ebs: Pausing after run of step 'StepSecurityGroup'. Press enter to continue.

2014/10/31 10:17:44 ui: scan err: unexpected newline
==> amazon-ebs: Launching a source AWS instance...
2014/10/31 10:17:44 ui: ==> amazon-ebs: Launching a source AWS instance...
    amazon-ebs: Instance ID: i-54560f4d
2014/10/31 10:17:45 ui:     amazon-ebs: Instance ID: i-54560f4d
==> amazon-ebs: Waiting for instance (i-54560f4d) to become ready...
2014/10/31 10:17:45 ui: ==> amazon-ebs: Waiting for instance (i-54560f4d) to become ready...
2014/10/31 10:17:45 packer-builder-amazon-ebs: 2014/10/31 10:17:45 Waiting for state to become: running
2014/10/31 10:17:45 packer-builder-amazon-ebs: 2014/10/31 10:17:45 Allowing 300s to complete (change with AWS_TIMEOUT_SECONDS)
    amazon-ebs: Public DNS: ec2-54-65-12-210.ap-northeast-1.compute.amazonaws.com
2014/10/31 10:18:03 ui:     amazon-ebs: Public DNS: ec2-54-65-12-210.ap-northeast-1.compute.amazonaws.com
    amazon-ebs: Public IP: 54.65.12.210
2014/10/31 10:18:03 ui:     amazon-ebs: Public IP: 54.65.12.210
    amazon-ebs: Private IP: 10.0.0.121
2014/10/31 10:18:03 ui:     amazon-ebs: Private IP: 10.0.0.121
==> amazon-ebs: Pausing after run of step 'StepRunSourceInstance'. Press enter to continue. 2014/10/31 10:18:03 ui: ask: ==> amazon-ebs: Pausing after run of step 'StepRunSourceInstance'. Press enter to continue.

2014/10/31 10:20:13 ui: scan err: unexpected newline
2014/10/31 10:20:13 packer-builder-amazon-ebs: 2014/10/31 10:20:13 Waiting for SSH, up to timeout: 5m0s
==> amazon-ebs: Waiting for SSH to become available...
2014/10/31 10:20:13 ui: ==> amazon-ebs: Waiting for SSH to become available...
2014/10/31 10:20:13 packer-builder-amazon-ebs: 2014/10/31 10:20:13 Attempting SSH connection...
2014/10/31 10:20:13 packer-builder-amazon-ebs: 2014/10/31 10:20:13 reconnecting to TCP connection for SSH
2014/10/31 10:20:13 packer-builder-amazon-ebs: 2014/10/31 10:20:13 handshaking with SSH
==> amazon-ebs: Connected to SSH!
2014/10/31 10:20:14 ui: ==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Pausing after run of step 'StepConnectSSH'. Press enter to continue. 2014/10/31 10:20:14 ui: ask: ==> amazon-ebs: Pausing after run of step 'StepConnectSSH'. Press enter to continue.

bug buildeamazon

Source

iwai

All 6 comments

add more information.
i'm login on debug mode.

iwai on 31 Oct 2014

There are two different mappings available in the amazon builder: _ami_block_device_mappings_ and _launch_block_device_mappings_. _launch_block_device_mappings_ is used when booting an instance prior to the build process; _ami_block_device_mappings_ is used to instruct the AMI creation process.

As an example, the following specifies a single SSD-backed 1000-iops EBS drive for /dev/sda during the build process, but replaces that with a SSD-backed drive for the AMI - along with other ephemeral and storage drives which are not needed during the build process.

{
  "builders": [
    {
      "type": "amazon-ebs",
      "name": "amazon-ebs-mysql_mongodb_gridfs",
      "...": "...",
      "ami_block_device_mappings": [
        {
          "device_name": "/dev/sda",
          "volume_size": 16,
          "delete_on_termination": true,
          "volume_type": "gp2"
        },
        {
          "device_name": "/dev/sdb",
          "virtual_name": "ephemeral0"
        },
        {
          "device_name": "/dev/sdc",
          "virtual_name": "ephemeral1"
        },
        {
          "device_name": "/dev/sdd",
          "virtual_name": "ephemeral2"
        },
        {
          "device_name": "/dev/sde",
          "virtual_name": "ephemeral3"
        },
        {
          "device_name": "/dev/sdg",
          "volume_size": 20,
          "delete_on_termination": "false",
          "volume_type": "gp2"
        },
        {
          "device_name": "/dev/sdh",
          "volume_size": 1000,
          "delete_on_termination": false,
          "volume_type": "standard"
        }
      ],
      "launch_block_device_mappings": [
        {
          "device_name": "/dev/sda",
          "volume_size": 16,
          "delete_on_termination": true,
          "volume_type": "io1",
          "iops": 200
        }
      ]
    }
  ]
}