Aws-cli: "parameter-filters" not working for "aws ssm get-parameters-by-path" ?

[jamietsao]

Unless I'm mistaken, the documentation for get-parameters-by-path says you can filter the parameter results via parameter-filters: http://docs.aws.amazon.com/cli/latest/reference/ssm/get-parameters-by-path.html

I can't seem to get this to work:

AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path --parameter-filters Key=tag:xyz  --path /my/path

This returns ALL parameters in /my/path instead of only those tagged with xyz.

describe-parameters has the same parameter-filters and it works:

AWS_DEFAULT_REGION=us-west-2 aws ssm describe-parameters --parameter-filters Key=tag:xyz

This will correctly return only parameters tagged with xyz.

Am I missing something?

[joguSD]

Could you post debug logs by adding --debug?
It would seem that this should be possible and as long as the cli is properly sending the request this might actually be an issue with the service.

[jamietsao]

I've created two test parameters under /test:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path  --path /test
{
    "Parameters": [
        {
            "Type": "String",
            "Name": "/test/foo",
            "Value": "bar"
        },
        {
            "Type": "String",
            "Name": "/test/hello",
            "Value": "world"
        }
    ]
}

I added a baz tag to one of them:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm describe-parameters --parameter-filters Key=tag:baz
{
    "Parameters": [
        {
            "LastModifiedUser": "arn:aws:iam::<redacted>:user/<redacted>",
            "LastModifiedDate": 1506377527.11,
            "Type": "String",
            "Name": "/test/foo"
        }
    ]
}

But when I try to filter by baz tag using get-parameters-by-path, it still returns both:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path --parameter-filters Key=tag:baz --path /test --debug
2017-09-25 15:16:02,231 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.150 Python/2.7.10 Darwin/16.7.0 botocore/1.7.8
2017-09-25 15:16:02,231 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['ssm', 'get-parameters-by-path', '--parameter-filters', 'Key=tag:baz', '--path', '/test', '--debug']
2017-09-25 15:16:02,231 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x109a9d578>
2017-09-25 15:16:02,231 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x1097b9d70>
2017-09-25 15:16:02,233 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/ssm/2014-11-06/service-2.json
2017-09-25 15:16:02,275 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.ssm: calling handler <function register_retries_for_service at 0x109375758>
2017-09-25 15:16:02,275 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ssm
2017-09-25 15:16:02,279 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ssm: calling handler <function add_waiters at 0x109aa6668>
2017-09-25 15:16:02,284 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'path', <awscli.arguments.CLIArgument object at 0x10a2fb190>), (u'recursive', <awscli.arguments.BooleanArgument object at 0x10a2fb1d0>), (u'no-recursive', <awscli.arguments.BooleanArgument object at 0x10a2fb210>), (u'parameter-filters', <awscli.arguments.ListArgument object at 0x10a2fb250>), (u'with-decryption', <awscli.arguments.BooleanArgument object at 0x10a2f0dd0>), (u'no-with-decryption', <awscli.arguments.BooleanArgument object at 0x10a2fb290>), (u'max-results', <awscli.arguments.CLIArgument object at 0x10a2fb2d0>), (u'next-token', <awscli.arguments.CLIArgument object at 0x10a2fb310>)])
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_streaming_output_arg at 0x109a9d7d0>
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_cli_input_json at 0x1097c2b90>
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function unify_paging_params at 0x109a16d70>
2017-09-25 15:16:02,288 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/ssm/2014-11-06/paginators-1.json
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_generate_skeleton at 0x109a0e1b8>
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ssm.get-parameters-by-path: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10a2fb350>>
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ssm.get-parameters-by-path: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10a2d6150>>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.path: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ssm.get-parameters-by-path: calling handler <awscli.argprocess.ParamShorthandParser object at 0x109aa83d0>
2017-09-25 15:16:02,291 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'/test' for parameter "path": u'/test'
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.recursive: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.parameter-filters: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ssm.get-parameters-by-path: calling handler <awscli.argprocess.ParamShorthandParser object at 0x109aa83d0>
2017-09-25 15:16:02,291 - MainThread - awscli.argprocess - DEBUG - Parsing param --parameter-filters as shorthand
2017-09-25 15:16:02,292 - MainThread - awscli.arguments - DEBUG - Unpacked value of [u'Key=tag:baz'] for parameter "parameter_filters": [{u'Key': u'tag:baz'}]
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.with-decryption: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.max-results: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.next-token: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.cli-input-json: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.generate-cli-skeleton: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event calling-command.ssm.get-parameters-by-path: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10a2fb350>>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event calling-command.ssm.get-parameters-by-path: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10a2d6150>>
2017-09-25 15:16:02,292 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2017-09-25 15:16:02,292 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2017-09-25 15:16:02,293 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2017-09-25 15:16:02,293 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2017-09-25 15:16:02,293 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/endpoints.json
2017-09-25 15:16:02,329 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.ssm: calling handler <function add_generate_presigned_url at 0x1093459b0>
2017-09-25 15:16:02,329 - MainThread - botocore.args - DEBUG - The s3 config key is not a dictionary type, ignoring its value of: None
2017-09-25 15:16:02,341 - MainThread - botocore.endpoint - DEBUG - Setting ssm timeout as (60, 60)
2017-09-25 15:16:02,342 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: ssm
2017-09-25 15:16:02,342 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.ssm.GetParametersByPath: calling handler <function generate_idempotent_uuid at 0x1093751b8>
2017-09-25 15:16:02,343 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetParametersByPath) (verify_ssl=True) with params: {'body': '{"Path": "/test", "ParameterFilters": [{"Key": "tag:baz"}]}', 'url': u'https://ssm.us-west-2.amazonaws.com/', 'headers': {'User-Agent': 'aws-cli/1.11.150 Python/2.7.10 Darwin/16.7.0 botocore/1.7.8', 'Content-Type': u'application/x-amz-json-1.1', 'X-Amz-Target': u'AmazonSSM.GetParametersByPath'}, 'context': {'auth_type': None, 'client_region': 'us-west-2', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x10a4b9d50>}, 'query_string': '', 'url_path': '/', 'method': u'POST'}
2017-09-25 15:16:02,343 - MainThread - botocore.hooks - DEBUG - Event request-created.ssm.GetParametersByPath: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10a4b9cd0>>
2017-09-25 15:16:02,343 - MainThread - botocore.hooks - DEBUG - Event choose-signer.ssm.GetParametersByPath: calling handler <function set_operation_specific_signer at 0x1093750c8>
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-amz-json-1.1
host:ssm.us-west-2.amazonaws.com
x-amz-date:20170925T221602Z
x-amz-target:AmazonSSM.GetParametersByPath

content-type;host;x-amz-date;x-amz-target
472e8bd620027d5bcbfae2b28aae0c138963d1ce4201a45657ba3eef65d15667
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20170925T221602Z
20170925/us-west-2/ssm/aws4_request
8634a86420a036bb701d34ff2f827b56fa7dd9792956370682ba5b4df729d4af
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - Signature:
9e491139a8fa51c3164aed3d5eee90d826f082c5495a8634854c7969382e1543
2017-09-25 15:16:02,345 - MainThread - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [POST]>
2017-09-25 15:16:02,345 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): ssm.us-west-2.amazonaws.com
2017-09-25 15:16:02,541 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST / HTTP/1.1" 200 122
2017-09-25 15:16:02,542 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amzn-requestid': '1dd11356-a23f-11e7-96b2-41ae96746da4', 'date': 'Mon, 25 Sep 2017 22:16:02 GMT', 'content-length': '122', 'content-type': 'application/x-amz-json-1.1'}
2017-09-25 15:16:02,542 - MainThread - botocore.parsers - DEBUG - Response body:
{"Parameters":[{"Name":"/test/foo","Type":"String","Value":"bar"},{"Name":"/test/hello","Type":"String","Value":"world"}]}
2017-09-25 15:16:02,543 - MainThread - botocore.hooks - DEBUG - Event needs-retry.ssm.GetParametersByPath: calling handler <botocore.retryhandler.RetryHandler object at 0x10a2d60d0>
2017-09-25 15:16:02,543 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2017-09-25 15:16:02,543 - MainThread - awscli.formatter - DEBUG - RequestId: 1dd11356-a23f-11e7-96b2-41ae96746da4
{
    "Parameters": [
        {
            "Type": "String",
            "Name": "/test/foo",
            "Value": "bar"
        },
        {
            "Type": "String",
            "Name": "/test/hello",
            "Value": "world"
        }
    ]
}

As you can see, the filtering works for describe-parameters, but not get-parameters-by-path even though the documentation for parameter-filters is identical for both commands.

[jamietsao]

@joguSD - any updates?

[joguSD]

As far as I can tell the CLI is passing along the correct arguments in the same manner in both cases so this is ultimately a difference in the service behavior between the two API calls. You should open up a support ticket to get a hold of the service team, I'll try pinging them as well. Closing out the issue here as there's no actionable item in this repo.

[satterly]

The documentation says it's not supported...

GetParametersByPath
Retrieve parameters in a specific hierarchy. For more information, see Working with Systems Manager Parameters in the AWS Systems Manager User Guide.
Note
This API action doesn't support filtering by tags.

https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParametersByPath.html

[jamietsao]

Yeah, that note wasn't there when I posted this over 2 years ago. Thanks.

[Obirah]

Just if someone stumbles across this issue like I did: I worked around this restriction by labeling the parameters.

[Michael1d]

Genius @Obirah ! Spent hours trying to work out why the tag functionality didn't work. Found your comment - sorted! Cheers :)

[cosmok]

Another option is to use describe-parameters instead (when we need to filter by tags) and then pipe the result to get-parameter.

See:
https://www.trk7.com/blog/aws-ssm-parameter-store-iam-tag-support/