Aws-cli: Error parsing parameter '--multipart-upload': Expected: '=', received: '' for input:

[charudut]

once i finished uploading ALL the parts, i tried to run "complete-multipart-upload" using the following command
"aws s3api complete-multipart-upload --endpoint-url http:// --bucket aws --key 'multipart/01' --upload-id --multipart-upload file://file.txt"

trying to run this from a ubuntu linux and the command just froze after throwing out the error mentioned in the title. am i doing something incorrectly?

Thanks,
-cjs

[kyleknap]

Can you provide the following information to help debug this as I cannot reproduce:
1) Debug logs. You can do this by appending --debug to the command you ran.
2) How you installed the CLI? Like was it using pip or some other installation method?

[charudut]

hello kyleknap thanks for replying.

here are the details you asked for:

root@vm-cshetty-1:~# aws --version
aws-cli/1.10.38 Python/2.7.6 Linux/3.13.0-63-generic botocore/1.4.28

i first installed via apt-get and then i ran pip install.

here's the debug log:

root@vm-cshetty-1:~/NAS2# aws s3api complete-multipart-upload --endpoint-url http://<ip>/s3/ --bucket aws --key 'multipart/01' --upload-id 00000155-6bd7-4883-a03f-dd148f162449 --multipart-upload file://999m.file --debug
2016-06-20 23:48:07,669 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.10.38 Python/2.7.6 Linux/3.13.0-63-generic botocore/1.4.28
2016-06-20 23:48:07,669 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3api', 'complete-multipart-upload', '--endpoint-url', 'http://<ip>/s3/', '--bucket', 'aws', '--key', 'multipart/01', '--upload-id', '00000155-6bd7-4883-a03f-dd148f162449', '--multipart-upload', 'file://999m.file', '--debug']
2016-06-20 23:48:07,669 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x7fe8a748cc80>
2016-06-20 23:48:07,669 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x7fe8a786a668>
2016-06-20 23:48:07,669 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2016-06-20 23:48:07,671 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/dist-packages/botocore/data/s3/2006-03-01/service-2.json
2016-06-20 23:48:07,703 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler <function register_retries_for_service at 0x7fe8a7cb6b90>
2016-06-20 23:48:07,703 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: s3
2016-06-20 23:48:07,706 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api: calling handler <function add_waiters at 0x7fe8a749e1b8>
2016-06-20 23:48:07,709 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/dist-packages/botocore/data/s3/2006-03-01/waiters-2.json
2016-06-20 23:48:07,711 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'bucket', <awscli.arguments.CLIArgument object at 0x7fe8a6e5ed90>), (u'key', <awscli.arguments.CLIArgument object at 0x7fe8a6e6e050>), (u'multipart-upload', <awscli.arguments.CLIArgument object at 0x7fe8a6e6e090>), (u'upload-id', <awscli.arguments.CLIArgument object at 0x7fe8a6e6e0d0>), (u'request-payer', <awscli.arguments.CLIArgument object at 0x7fe8a6e6e110>)])
2016-06-20 23:48:07,711 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.complete-multipart-upload: calling handler <function add_streaming_output_arg at 0x7fe8a7496320>
2016-06-20 23:48:07,711 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.complete-multipart-upload: calling handler <function add_cli_input_json at 0x7fe8a7874578>
2016-06-20 23:48:07,711 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.complete-multipart-upload: calling handler <function unify_paging_params at 0x7fe8a74b36e0>
2016-06-20 23:48:07,713 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/lib/python2.7/dist-packages/botocore/data/s3/2006-03-01/paginators-1.json
2016-06-20 23:48:07,714 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.complete-multipart-upload: calling handler <function add_generate_skeleton at 0x7fe8a772cc08>
2016-06-20 23:48:07,714 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.complete-multipart-upload: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x7fe8a6e6e150>>
2016-06-20 23:48:07,714 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.complete-multipart-upload: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7fe8a7179f10>>
2016-06-20 23:48:07,716 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.complete-multipart-upload.bucket: calling handler <function uri_param at 0x7fe8a7893f50>
2016-06-20 23:48:07,716 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.s3.complete-multipart-upload: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7fe8a749b650>
2016-06-20 23:48:07,716 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'aws' for parameter "bucket": u'aws'
2016-06-20 23:48:07,716 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.complete-multipart-upload.key: calling handler <function uri_param at 0x7fe8a7893f50>
2016-06-20 23:48:07,716 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.s3.complete-multipart-upload: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7fe8a749b650>
2016-06-20 23:48:07,717 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'multipart/01' for parameter "key": u'multipart/01'
2016-06-20 23:48:07,717 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.complete-multipart-upload.multipart-upload: calling handler <function uri_param at 0x7fe8a7893f50>
2016-06-20 23:48:36,494 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.s3.complete-multipart-upload: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7fe8a749b650>
2016-06-20 23:48:36,494 - MainThread - awscli.argprocess - DEBUG - Parsing param --multipart-upload as shorthand
2016-06-20 23:48:52,260 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 186, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 379, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 520, in __call__
    parsed_args, self.arg_table)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 576, in _build_call_parameters
    arg_object.add_to_params(service_params, value)
  File "/usr/local/lib/python2.7/dist-packages/awscli/arguments.py", line 453, in add_to_params
    unpacked = self._unpack_argument(value)
  File "/usr/local/lib/python2.7/dist-packages/awscli/arguments.py", line 463, in _unpack_argument
    cli_argument=self, value=value)
  File "/usr/local/lib/python2.7/dist-packages/awscli/arguments.py", line 476, in _emit_first_response
    responses = self._emit(name, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/awscli/arguments.py", line 473, in _emit
    return self._event_emitter.emit(name, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/awscli/argprocess.py", line 317, in __call__
    cli_argument, value, service_name, operation_name)
  File "/usr/local/lib/python2.7/dist-packages/awscli/argprocess.py", line 341, in _parse_as_shorthand
    raise ParamError(cli_argument.cli_name, str(e))
ParamError: Error parsing parameter '--multipart-upload': Expected: '=', received: '' for input:

and it's stuck at this output.

let me know if you need further details from me.

thanks,
-cjs

[kyleknap]

Interesting, I am not exactly sure what is happening in terms of why you are getting this error. I have a feeling it has something to do with how the Debian package manager is packaging the CLI as there have been quite a few issues in the past where the package managers actually removed or modified code when packaging the CLI. Furthermore, these packages of the CLI are all maintained by other individuals so we do not have much control or insight on how they are packaging the CLI.

So I would recommend trying to completely uninstalling the CLI with apt-get and then reinstalling with pip (preferably in a virtual environment), which is a method of installation we do maintain. Let me know if that helps.

[jamesls]

I was able to repro the issue. The error is due to the contents of 999m.file contents, which we don't log in our debug logs. For example, I get a similiar error message if I specify an empty file, or a file with invalid shorthand syntax. Double check the contents of that file.

If you're still having issues, please provide the contents of the 999.mfile you're using for the --multipart-upload argument and I'd be happy to reopen and take another look.

[charudut]

hello jamesls, thank you for looking into the issue. i generated the 999m.file using "fallocate -l 999m 999m.file". from what i know fallocate reserves space in disk but doesn't write anything to it.

[codesurf42]

I found the same error message returned when the json file was ok but uploaded parts were smaller than expected size (~5 MB/part) so s3 couldn't finish such upload transaction.
Checking what was exactly reported by --debug helped in this case.

[shilpa1016]

aws s3api complete-multipart-upload --multipart-upload file://C:UsersLillyDesktopS3_POCfileparts.json --bucket bucket-name --key object-name --upload-id wEGRHEqUA.7deiJHGLj0kezR5yp0wPIEOeqZJrTmC_9cdyuQ0vgNKbOkBWWFayCRWooj0mIsw02hQPTjcW2pgG6BIcj0fLZGDd67B2ytTzydw.lkskOkuqn57OkozWYb

I got the same issue and doubted whether there is some parsing error in my json file. But there was not. The file:// prefix is a must and is used to load the JSON structure from a file in the local folder named mpustruct. (Note: The json structure should be properly formed as per the aws specification)

Specifying the parts directly in the command and not form an external json file can also work
--multipart-upload Parts=[{ETag=c89bc55d61534a407e6ea376e0eedcc0,PartNumber=1},{ETag=44c7af3f06fd63a36593826be3cb7684,PartNumber=2}]

[darryn02]

+1 for using file:// prefix, that fixed it for me (for file in current directory use file://./file.json). Thanks!