Aws-cli: aws ssm put-parameters ignores trailing new line characters

[chgangaraju]

I am trying to insert a SSH key into parameter store. After a lot of struggle by inserting key using console, this issue helps us to fix the missing new line characters. Even though we are able to get the content properly, _it is trimming the new line character in the last_.

put-parameter:

aws ssm put-parameter --name "ssh_key" --type "SecureString" --overwrite --value "-----BEGIN RSA PRIVATE KEY-----   
some content here
-----END RSA PRIVATE KEY-----

"

get-parameters:

aws ssm get-parameters --names ssh_key --with-decryption --region eu-west-1 --output text 2>&1 | sed 's/.*----BEGIN/----BEGIN/'

----BEGIN RSA PRIVATE KEY-----
some content here
-----END RSA PRIVATE KEY-----

[jamesls]

I'm not able to reproduce the issue. Following your example I get:

$ aws ssm put-parameter --name "ssh_key" --type "SecureString" --overwrite --value "-----BEGIN RSA PRIVATE KEY-----
> some content here
> -----END RSA PRIVATE KEY-----
> "

Now when I get the param:

$ aws ssm get-parameter --name ssh_key --with-decryption
{
    "Parameter": {
        "Type": "SecureString",
        "Name": "ssh_key",
        "Value": "-----BEGIN RSA PRIVATE KEY-----\nsome content here\n-----END RSA PRIVATE KEY-----\n"
    }
}

Note how there's a new line in the "Value" key. So if I try to extract just that value:

$ aws ssm get-parameter --name ssh_key --with-decryption --output text --query Parameter.Value
-----BEGIN RSA PRIVATE KEY-----
some content here
-----END RSA PRIVATE KEY-----

It works as expected. We can also verify the newline using hexdump:

 $ aws ssm get-parameter --name ssh_key --with-decryption --output text --query Parameter.Value   | hexdump -C
00000000  2d 2d 2d 2d 2d 42 45 47  49 4e 20 52 53 41 20 50  |-----BEGIN RSA P|
00000010  52 49 56 41 54 45 20 4b  45 59 2d 2d 2d 2d 2d 0a  |RIVATE KEY-----.|
00000020  73 6f 6d 65 20 63 6f 6e  74 65 6e 74 20 68 65 72  |some content her|
00000030  65 0a 2d 2d 2d 2d 2d 45  4e 44 20 52 53 41 20 50  |e.-----END RSA P|
00000040  52 49 56 41 54 45 20 4b  45 59 2d 2d 2d 2d 2d 0a  |RIVATE KEY-----.|
00000050  0a                                                |.|
00000051

Note the "0a" at the end which is the newline char, i.e hex(ord('\n'))

[chgangaraju]

Thank you @jamesls , hexdump helps to debug this issue. I can see 0a character. You can close this issue.

This is a problem with sed command which i used to parse Parameter.Value. I am skipping - character in the first.
sed 's/.*----BEGIN/----BEGIN/'

I am using fedora and installed awscli using rpm. I could not use --query option.

➜  ~ aws --version
aws-cli/1.11.109 Python/3.6.2 Linux/4.11.11-300.fc26.x86_64 botocore/1.5.72

 ~ aws ssm get-parameter --name ssh_key --with-decryption --output text --query Parameter.Value    

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument operation: Invalid choice, valid choices are:

add-tags-to-resource                     | cancel-command                          
create-activation                        | create-association                      
create-association-batch                 | create-document                         
create-maintenance-window                | create-patch-baseline                   
delete-activation                        | delete-association                      
delete-document                          | delete-maintenance-window               
delete-parameter                         | delete-patch-baseline                   
deregister-managed-instance              | deregister-patch-baseline-for-patch-group
deregister-target-from-maintenance-window | deregister-task-from-maintenance-window 
describe-activations                     | describe-association                    
describe-automation-executions           | describe-available-patches              
describe-document                        | describe-document-permission            
describe-effective-instance-associations | describe-effective-patches-for-patch-baseline
describe-instance-associations-status    | describe-instance-information           
describe-instance-patch-states           | describe-instance-patch-states-for-patch-group
describe-instance-patches                | describe-maintenance-window-execution-task-invocations
describe-maintenance-window-execution-tasks | describe-maintenance-window-executions  
describe-maintenance-window-targets      | describe-maintenance-window-tasks       
describe-maintenance-windows             | describe-parameters                     
describe-patch-baselines                 | describe-patch-group-state              
describe-patch-groups                    | get-automation-execution                
get-command-invocation                   | get-default-patch-baseline              
get-deployable-patch-snapshot-for-instance | get-document                            
get-inventory                            | get-inventory-schema                    
get-maintenance-window                   | get-maintenance-window-execution        
get-maintenance-window-execution-task    | get-parameter-history                   
get-parameters                           | get-patch-baseline                      
get-patch-baseline-for-patch-group       | list-associations                       
list-command-invocations                 | list-commands                           
list-document-versions                   | list-documents                          
list-inventory-entries                   | list-tags-for-resource                  
modify-document-permission               | put-inventory                           
put-parameter                            | register-default-patch-baseline         
register-patch-baseline-for-patch-group  | register-target-with-maintenance-window 
register-task-with-maintenance-window    | remove-tags-from-resource               
send-command                             | start-automation-execution              
stop-automation-execution                | update-association                      
update-association-status                | update-document                         
update-document-default-version          | update-maintenance-window               
update-managed-instance-role             | update-patch-baseline                   
help                                    


Invalid choice: 'get-parameter', maybe you meant:

  * get-parameters
  * put-parameter
  * delete-parameter