Packer: Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot

Created on 15 Mar 2019 · 18Comments · Source: hashicorp/packer

@rcaught @azr on latest 1.4 packer I am getting

1552628049,,ui,message, amazon-ebs: Waiting for all copies to complete... 
1552628360,,ui,say,==> amazon-ebs: Modifying attributes on AMI (ami-0906556ff67d70be8)... 
1552628360,,ui,message, amazon-ebs: Modifying: users 1552628360,,ui,error,==> amazon-ebs: Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.
==> amazon-ebs: status code: 400%!(PACKER_COMMA) request id: 38a0a0d6-1d0f-4b27-afce-607e85e78805 
1552628360,,ui,say,==> amazon-ebs: Deregistering the AMI because cancellation%!(PACKER_COMMA) error or it was temporary (encrypt_boot was set)...
1552628360,,ui,say,==> amazon-ebs: Cancelling the spot request...
1552628360,,ui,say,==> amazon-ebs: Terminating the source AWS instance...

Kindly note I don't want my boot/root volume to be encrypted, an additional volume that I am attaching needs to be encrypted and while sharing new ami created should also have that encrypted snapshot.

bug buildeamazon

Source

ff-agrawal

All 18 comments

Please share your buildfile and you systems informations as stated in here ! 🙂

azr on 15 Mar 2019

@azr you can find debugs logs at https://gist.github.com/ff-agrawal/7cc59057c4b230d71deb6870c2dcecb5

I hope with this log file, you will get info about things in my buildfile as well, let me know in case you want me to attach that explicitly as well

Its a Linux system
Host platform (uname -a etc.) = Linux 4.14.101-75.76.amzn1.x86_64 #1 SMP Tue Feb 19 18:14:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Packer version is 1.4 taken from https://github.com/hashicorp/packer/files/2953566/packer.zip

Template: https://gist.github.com/ff-agrawal/a0b79472c1c9dabb94478363f617f033

ff-agrawal on 15 Mar 2019

@azr I believe you have all the information now to debug further

ff-agrawal on 15 Mar 2019

@ff-agrawal, I need your json buildfile or a simplified version of it so that it's easier to reproduce on my side, this would save me a lot of time 🙂

azr on 15 Mar 2019

@azr I did provided the link of the template in my previous comment https://github.com/hashicorp/packer/issues/7410#issuecomment-473230994

You can remove all the chef part and can trying creating ami with root non encrypted volume and another volume encrypted post that try sharing the ami say to Frankfurt region

Kindly let me know in case if anything is still needed

ff-agrawal on 15 Mar 2019

👍1

Thanks for adding it there, I will look into it 🙂

azr on 15 Mar 2019

@azr any update on the fix

ff-agrawal on 21 Mar 2019

Howdy @ff-agrawal, I've been tweaking around with your build file and I think a fix is that you should just not pass any encrypt_boot setting.

See, it's impossible to make an encrypted AMI unencrypted - AWS will deny it - but #6787 allows to not pass the encrypted setting to retain what the source image has as an encrypted setting.

In your case, if the source image's boot is not encrypted, then no problemo it will just stay unencrypted.

azr on 21 Mar 2019

@azr I think you misunderstood the issue, I am not making encrypted AMI unencrypted, my goal is to simply copy ami with the additional volume encrypted from one region to another region using.
The operation is pretty straight forward when done form UI and I would expect the same should be happening when done via packer.
If you need additional info we can do hangout session wherein I can walk you through my requirement, please note it was happening fine earlier before we upgraded to packer ~1.3.1 version.

ff-agrawal on 22 Mar 2019

Ah sorry - yes - I'm looking into it! 🙂

azr on 22 Mar 2019

Okay, so I just tested it and like I said before you just need to remove "encrypt_boot": false from your build.

By putting "encrypt_boot": false in your config file, packer will try to unencrypt the encrypted drive during the snapshot phase.

Here's a simplified version of your config file that worked for me :

{
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "us-east-1",
      "source_ami_filter": {
        "filters": {
          "virtualization-type": "hvm",
          "name": "amzn-ami-hvm-????.??.?.????????-x86_64-gp2",
          "root-device-type": "ebs"
        },
        "most_recent": true,
        "owners": "amazon"
      },
      "launch_block_device_mappings": [
        {
          "device_name": "/dev/xvda",
          "volume_size": "20",
          "volume_type": "gp2",
          "delete_on_termination": "true"
        },
        {
          "device_name": "/dev/sdb",
          "volume_size": "500",
          "volume_type": "gp2",
          "delete_on_termination": "true",
          "encrypted": true
        }
      ],
      "instance_type": "t2.micro",
      "communicator": "ssh",
      "ssh_pty": "true",
      "ssh_username": "ec2-user", 
      "associate_public_ip_address": "true",
      "ami_name": "solr-test"
    }

Screenshot 2019-03-22 at 12 11 14

I will add log warnings ( and probably a doc entry ) to make this more obvious.

azr on 22 Mar 2019

@azr where you able to able to transfer automatically say from us-east-1 to eu-central-1 via packer job

ff-agrawal on 22 Mar 2019

Yes ! I did so by just adding: "ami_regions": ["eu-central-1"],
~Internally, copying to a region is the same operation as creating the AMI.~ < that last statement is not true, but it still worked 🙂

azr on 22 Mar 2019

let me give it a try

ff-agrawal on 22 Mar 2019

👍1

@azr I tried but it failed
template used: https://gist.github.com/ff-agrawal/aa0d63ad38d230ec468f2a388f290e2a

packer used: https://github.com/hashicorp/packer/files/2953566/packer.zip

Error logs

`2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:35 [INFO] 858622 bytes written for 'stdout' 2019/03/22 13:30:35 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:35 [INFO] RPC client: Communicator ended with: 0 2019/03/22 13:30:35 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [INFO] 858622 bytes written for 'stdout' 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [INFO] RPC client: Communicator ended with: 0 1553261435,,ui,message, amazon-ebs: Creating configuration file 'knife.rb' 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [INFO] 226 bytes written for 'uploadData' 2019/03/22 13:30:35 [INFO] 226 bytes written for 'uploadData' 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Opening new ssh session 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Starting remote scp process: scp -vt /tmp/packer-chef-client 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Started SCP session, beginning transfers... 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Copying input data into temporary file so we can read the length 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] scp: Uploading knife.rb: perms=C0644 size=226 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] SCP session complete, closing stdin pipe. 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Waiting for SSH session to complete. 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] scp stderr (length 25): Sink: C0644 226 knife.rb 1553261435,,ui,say,==> amazon-ebs: Cleaning up chef node... 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] Opening new ssh session 2019/03/22 13:30:35 packer: 2019/03/22 13:30:35 [DEBUG] starting remote command: sudo knife node delete packer-solr-1553261125 -y -c /tmp/packer-chef-client/knife.rb 1553261436,,ui,message, amazon-ebs: Deleted node[packer-solr-1553261125] 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:36 [INFO] 38 bytes written for 'stdout' 2019/03/22 13:30:36 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:36 [INFO] RPC client: Communicator ended with: 0 2019/03/22 13:30:36 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [INFO] 38 bytes written for 'stdout' 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [INFO] RPC client: Communicator ended with: 0 1553261436,,ui,say,==> amazon-ebs: Cleaning up chef client... 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [DEBUG] Opening new ssh session 2019/03/22 13:30:36 packer: 2019/03/22 13:30:36 [DEBUG] starting remote command: sudo knife client delete packer-solr-1553261125 -y -c /tmp/packer-chef-client/knife.rb 1553261437,,ui,message, amazon-ebs: Deleted client[packer-solr-1553261125] 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:37 [INFO] 40 bytes written for 'stdout' 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:37 [INFO] RPC client: Communicator ended with: 0 2019/03/22 13:30:37 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] 40 bytes written for 'stdout' 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] RPC client: Communicator ended with: 0 1553261437,,ui,message, amazon-ebs: Removing directory: /tmp/packer-chef-client 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [DEBUG] Opening new ssh session 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [DEBUG] starting remote command: sudo rm -rf '/tmp/packer-chef-client' 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stdout' 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:37 [INFO] RPC client: Communicator ended with: 0 2019/03/22 13:30:37 [INFO] RPC endpoint: Communicator ended with: 0 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stdout' 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] 0 bytes written for 'stderr' 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 [INFO] RPC client: Communicator ended with: 0 2019/03/22 13:30:37 [INFO] (telemetry) ending chef-client 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 Error asking for input: machine-readable UI can't ask 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 Error asking for input: machine-readable UI can't ask 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 Error asking for input: machine-readable UI can't ask 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 Error asking for input: machine-readable UI can't ask 2019/03/22 13:30:37 packer: 2019/03/22 13:30:37 Error asking for input: machine-readable UI can't ask 1553261437,,ui,say,==> amazon-ebs: Creating AMI solr-2019-03-22_012525 from instance i-09fe2106ba88fff2a 1553261437,,ui,message, amazon-ebs: AMI: ami-0c3b1705ea1688300 1553261437,,ui,say,==> amazon-ebs: Waiting for AMI to become ready... 2019/03/22 13:35:54 packer: 2019/03/22 13:35:54 Error asking for input: machine-readable UI can't ask 1553261754,,ui,say,==> amazon-ebs: Copying AMI (ami-0c3b1705ea1688300) to other regions... 1553261754,,ui,message, amazon-ebs: Copying to: eu-central-1 1553261754,,ui,message, amazon-ebs: Waiting for all copies to complete... 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 1553262065,,ui,say,==> amazon-ebs: Modifying attributes on AMI (ami-0c3b1705ea1688300)... 1553262065,,ui,message, amazon-ebs: Modifying: users 1553262065,,ui,error,==> amazon-ebs: Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.\n==> amazon-ebs: status code: 400%!(PACKER_COMMA) request id: 9eb68dc8-c948-4698-961d-909a81e090a9 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 1553262065,,ui,say,==> amazon-ebs: Deregistering the AMI because cancellation%!(PACKER_COMMA) error or it was temporary (encrypt_boot was set)... 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:05 packer: 2019/03/22 13:41:05 Error asking for input: machine-readable UI can't ask 1553262065,,ui,say,==> amazon-ebs: Cancelling the spot request... 1553262065,,ui,say,==> amazon-ebs: Terminating the source AWS instance... 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 1553262111,,ui,say,==> amazon-ebs: Cleaning up any extra volumes... 1553262111,,ui,say,==> amazon-ebs: No volumes to clean up%!(PACKER_COMMA) skipping 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 1553262111,,ui,say,==> amazon-ebs: Deleting temporary keypair... 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:51 packer: 2019/03/22 13:41:51 Error asking for input: machine-readable UI can't ask 2019/03/22 13:41:51 [INFO] (telemetry) ending amazon-ebs 2019/03/22 13:41:51 Waiting on builds to complete... 2019/03/22 13:41:51 Builds completed. Waiting on interrupt barrier... 2019/03/22 13:41:51 [INFO] (telemetry) Finalizing. 1553262111,,ui,error,Build 'amazon-ebs' errored: Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.\n status code: 400%!(PACKER_COMMA) request id: 9eb68dc8-c948-4698-961d-909a81e090a9 1553262111,,error-count,1 1553262111,,ui,error,\n==> Some builds didn't complete successfully and had errors: 1553262111,amazon-ebs,error,Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.\n status code: 400%!(PACKER_COMMA) request id: 9eb68dc8-c948-4698-961d-909a81e090a9 1553262111,,ui,error,--> amazon-ebs: Error modify AMI attributes: InvalidParameter: Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot.\n status code: 400%!(PACKER_COMMA) request id: 9eb68dc8-c948-4698-961d-909a81e090a9 1553262111,,ui,say,\n==> Builds finished but no artifacts were created. 2019/03/22 13:41:51 waiting for all plugin processes to complete... 2019/03/22 13:41:51 /home/ec2-user/workspace/Packer-AMI_Build/packerapp/packer: plugin process exited 2019/03/22 13:41:51 /home/ec2-user/workspace/Packer-AMI_Build/packerapp/packer: plugin process exited 2019/03/22 13:41:51 /home/ec2-user/workspace/Packer-AMI_Build/packerapp/packer: plugin process exited [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: script returned exit code 1 Finished: FAILURE

ff-agrawal on 22 Mar 2019

This isn't a Packer bug -- it's an AWS behavior. https://aws.amazon.com/premiumsupport/knowledge-center/create-unencrypted-volume-cmk/

If you want to share the volume, you need to encrypt it with a non-default key.

SwampDragons on 25 Mar 2019

👍1

Oh right, I didn't realize that - sorry about losing both our times on this - I think this issue can be closed then!

azr on 26 Mar 2019

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.