Aws-cli: Modify Instance Attribute UserData with file:// doesn't work

[gregbook]

I'm having issues with modifying the instance attribute user-data of one of my instances please see below:

This works with some text:
aws ec2 modify-instance-attribute --attribute userData --instance-id i-362098dd --user-data "Value=MyNewUserData"

This doesn't:
aws ec2 modify-instance-attribute --attribute userData --instance-id i-362098dd --user-data file://userdata.txt
Error parsing parameter '--user-data': Expected: '=', received: 'r' for input:

I am using the following CLI:
aws-cli/1.11.63 Python/2.7.13 Darwin/16.3.0 botocore/1.5.26

Thanks for your help.

[stealthycoin]

My guess would be that file is formatted incorrectly, can you give us both the file and the --debug output?

[gregbook]

Debug output:

2017-03-21 02:30:25,362 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.63 Python/2.7.13 Darwin/16.3.0 botocore/1.5.26
2017-03-21 02:30:25,362 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'modify-instance-attribute', '--attribute', 'userData', '--instance-id', 'i-362098dd', '--user-data', 'file://userdata.txt', '--profile', 'gregor', '--debug']
2017-03-21 02:30:25,363 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x110b75398>
2017-03-21 02:30:25,363 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x110879d70>
2017-03-21 02:30:25,363 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2017-03-21 02:30:25,367 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/ec2/2016-11-15/service-2.json
2017-03-21 02:30:25,468 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.ec2: calling handler <function register_retries_for_service at 0x11049c7d0>
2017-03-21 02:30:25,468 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ec2
2017-03-21 02:30:25,474 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <functools.partial object at 0x110728f70>
2017-03-21 02:30:25,474 - MainThread - awscli.customizations.removals - DEBUG - Removing operation: import-instance
2017-03-21 02:30:25,474 - MainThread - awscli.customizations.removals - DEBUG - Removing operation: import-volume
2017-03-21 02:30:25,474 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <function add_waiters at 0x110b7d758>
2017-03-21 02:30:25,478 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/ec2/2016-11-15/waiters-2.json
2017-03-21 02:30:25,482 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'dry-run', <awscli.arguments.BooleanArgument object at 0x111920510>), (u'no-dry-run', <awscli.arguments.BooleanArgument object at 0x111920550>), (u'instance-id', <awscli.arguments.CLIArgument object at 0x111920590>), (u'attribute', <awscli.arguments.CLIArgument object at 0x1119205d0>), (u'value', <awscli.arguments.CLIArgument object at 0x111920610>), (u'block-device-mappings', <awscli.arguments.ListArgument object at 0x111920650>), (u'source-dest-check', <awscli.arguments.CLIArgument object at 0x111920690>), (u'disable-api-termination', <awscli.arguments.CLIArgument object at 0x1119206d0>), (u'instance-type', <awscli.arguments.CLIArgument object at 0x111920710>), (u'kernel', <awscli.arguments.CLIArgument object at 0x111920750>), (u'ramdisk', <awscli.arguments.CLIArgument object at 0x111920790>), (u'user-data', <awscli.arguments.CLIArgument object at 0x1119207d0>), (u'instance-initiated-shutdown-behavior', <awscli.arguments.CLIArgument object at 0x111920810>), (u'groups', <awscli.arguments.ListArgument object at 0x111920850>), (u'ebs-optimized', <awscli.arguments.CLIArgument object at 0x111920890>), (u'sriov-net-support', <awscli.arguments.CLIArgument object at 0x1119208d0>), (u'ena-support', <awscli.arguments.CLIArgument object at 0x111920910>)])
2017-03-21 02:30:25,482 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function add_streaming_output_arg at 0x110b759b0>
2017-03-21 02:30:25,482 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function _rename_arg at 0x110b83de8>
2017-03-21 02:30:25,482 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function _rename_arg at 0x110b9c320>
2017-03-21 02:30:25,482 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <functools.partial object at 0x110b79d60>
2017-03-21 02:30:25,483 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function add_cli_input_json at 0x110882c80>
2017-03-21 02:30:25,483 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function unify_paging_params at 0x110b06398>
2017-03-21 02:30:25,488 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/ec2/2016-11-15/paginators-1.json
2017-03-21 02:30:25,488 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ec2.modify-instance-attribute: calling handler <function add_generate_skeleton at 0x110af6758>
2017-03-21 02:30:25,489 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ec2.modify-instance-attribute: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x111772e90>>
2017-03-21 02:30:25,504 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ec2.modify-instance-attribute: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x111920fd0>>
2017-03-21 02:30:25,505 - MainThread - botocore.hooks - DEBUG - Event operation-args-parsed.ec2.modify-instance-attribute: calling handler <functools.partial object at 0x111856cb0>
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.dry-run: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.instance-id: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ec2.modify-instance-attribute: calling handler <awscli.argprocess.ParamShorthandParser object at 0x11085f8d0>
2017-03-21 02:30:25,506 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'i-362098dd' for parameter "instance_id": u'i-362098dd'
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.attribute: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ec2.modify-instance-attribute: calling handler <awscli.argprocess.ParamShorthandParser object at 0x11085f8d0>
2017-03-21 02:30:25,506 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'userData' for parameter "attribute": u'userData'
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.value: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,506 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.block-device-mappings: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.source-dest-check: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.disable-api-termination: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.instance-type: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.kernel: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.ramdisk: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.modify-instance-attribute.user-data: calling handler <function uri_param at 0x110711a28>
2017-03-21 02:30:25,507 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ec2.modify-instance-attribute: calling handler <awscli.argprocess.ParamShorthandParser object at 0x11085f8d0>
2017-03-21 02:30:25,507 - MainThread - awscli.argprocess - DEBUG - Parsing param --user-data as shorthand
2017-03-21 02:30:25,508 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 197, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 333, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 474, in __call__
    parsed_args, self.arg_table)
  File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 530, in _build_call_parameters
    arg_object.add_to_params(service_params, value)
  File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 453, in add_to_params
    unpacked = self._unpack_argument(value)
  File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 463, in _unpack_argument
    cli_argument=self, value=value)
  File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 476, in _emit_first_response
    responses = self._emit(name, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 473, in _emit
    return self._event_emitter.emit(name, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/awscli/argprocess.py", line 317, in __call__
    cli_argument, value, service_name, operation_name)
  File "/usr/local/lib/python2.7/site-packages/awscli/argprocess.py", line 341, in _parse_as_shorthand
    raise ParamError(cli_argument.cli_name, str(e))
ParamError: Error parsing parameter '--user-data': Expected: '=', received: 'c' for input:
#cloud-config

coreos:
^
  etcd:
    name: "node001"
    # generate a new token for each unique cluster from https://discovery.etcd.io/new
    discovery: "https://discovery.etcd.io/<token>"
    # multi-region and multi-cloud deployments need to use $public_ipv4
    addr: "$public_ipv4:4001"
    peer-addr: "$private_ipv4:7001"

2017-03-21 02:30:25,511 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

Userdata is a simple txt file with some cloud-config directives:
userdata.txt

[SleepyBrett]

same issue

[gregbook]

By any chance anyone can shade some light on this issue? Thanks a lot

[arukaen]

@gregbook Did you encode your textfile? base64 userdata.txt > userdata64.txt and then use file://userdata64.txt

Did this on my mac no problem. Using awscli-1.11.101

[coltaan]

Convert your file into base64 and use the command below:
aws ec2 modify-instance-attribute --attribute userData --value file://user-data.base64 --instance-id \

[gregbook]

@arukaen I did try your method, I still have the issue.
aws-cli/1.11.138 Python/2.7.13 Darwin/16.7.0 botocore/1.6.5

@coltaan the cloud provider I am using doesn't support modifying attributes this way...

[Suresh-TechM]

@coltaan Your answer worked for me. Thanks!

[justnance]

@gregbook - Just following up. Is this still an issue?

modify-instance-attribute is used on a stopped instance. With this command, the CLI does not perform base64 encoding of the user data for you. As such, the following base64 command would be used on a Linux platform:

base64 my_script.txt >my_script_base64.txt

Do you recall if --instance-id i-362098dd was in a stopped state at the time you ran the command. Understandably this is an old issue and the information may not be available any longer.

It also appears the format of the file://userdata.txt appears to be incorrect which would have caused: Error parsing parameter '--user-data': Expected: '=', received: 'r' for input:

[gregbook]

@justnance Thanks for following up on this. It was a while ago so I haven't tried recently. The instance was in a stopped state. You can close the issue.

[justnance]

@gregbook - Thanks for the update. Closing issue.