Boto3: InvalidTypeException when attempting to set access policies through update_elasticsearch_domain_config

[fperks]

Anytime i try to update the access policies through update_elasticsearch_domain_config, i get the following error:

botocore.exceptions.ClientError: An error occurred (InvalidTypeException) when calling the UpdateElasticsearchDomainConfig operation: Error setting policy: [{"Sid":"fsasaafffff","Effect":"Allow","Principal":{"AWS":"*"},"Action":"es:*","Resource":"arn:aws:es:us-west-2:XXXXXXXXX:domain/int-XXXXXX-XXXXX/*"}]}}]

It isn't the policy that seems to be the error, since i can set the exact same policy through awscli or the ui. Which leads me to believe i am passing it incorrectly somehow. I have tried every combination of storing it as a file, minified, unminifed, passing it in directly as a raw string, doing a json.dumps on it first, etc.

[mtdowling]

Can you provide your boto3 version number, and an example of how you're sending the request?

[csakoda]

@mtdowling We're seeing this on boto3 1.2.1

[melbit-kevinwessel]

Please try using this:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "fsasaafffff",
"Effect": "Allow",
"Principal": {
"AWS": "_"
},
"Action": "es:_",
"Resource": "arn:aws:es:us-west-2:XXXXXXXXX:domain/int-XXXXXX-XXXXX/*"
}
]
}

[asemt]

Same problem here, while trying to create a domain:
botocore.exceptions.ClientError: An error occurred (InvalidTypeException) when calling the CreateElasticsearchDomain operation: Error setting policy: [...]
We're using boto3 version 1.2.2.
I tried to use the modified policy suggested above with the removed * in the _Principal_ definition, but with no success.

[fiunchinho]

I'm hitting the same issue. Any workarounds?

Boto3 Version: 1.2.3

[pit]

Same error here.

aws es update-elasticsearch-domain-config --profile company --region us-east-1 --domain-name company-search --access-policies '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "_" }, "Action": [ "es:_" ], "Condition": { "IpAddress": { "aws:SourceIp": [ "A.B.C.D" ] } } "Resource": "*", } ] }'

Debug log: update-elasticsearch-domain-config.txt

[santhoshjoy]

Any solution. I am facing the same. I tried in aws console and CLI as well

[quiver]

I can successfully execute update_elasticsearch_domain_config API from boto3 and CLI as follows.

boto3

version : Boto3/1.2.5 Python/2.7.10 Darwin/15.3.0 Botocore/1.3.30

One thing to note is that you have to pass policy as strings not as JSON(dict).

import json
import boto3

#boto3.set_stream_logger(name='botocore') # for debugging
client = boto3.client('es')
access_policy = {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:ap-northeast-1:123456789012:domain/foo/*"
    }
  ]
}

client.update_elasticsearch_domain_config(
  DomainName='foo',
  AccessPolicies=json.dumps(access_policy)
)

AWS CLI

$ aws --version
aws-cli/1.10.8 Python/2.7.10 Darwin/15.3.0 botocore/1.3.30
$ cat access.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:ap-northeast-1:123456789012:domain/foo/*"
    }
  ]
}

$ aws es update-elasticsearch-domain-config --domain-name foo --access-policies file://access.json
...  UPDATED POLICIES WILL BE RETURNED

[aramirez-es]

I'm facing the same issue since I've updated to boto3 1.4.0 although it worked in my previous installed version (1.3.0). The same policy, introduced through AWS Console, works as expected. Don't tested aws-cli.

[fiunchinho]

any news on this? Still facing this issue almost one year later :(

[aramirez-es]

I'm sorry, it was my fault. I'm using Ansible with a custom python module (which uses boto3) and the way I was declaring access-policy was wrong.

[ryano144]

@aramirez-es can you comment on what you had to change? I'm currently experiencing the same issue on aws-cli 1.11.48 & boto-core 1.5.11.

[meadowlark-bradsher]

I am seeing this error with boto3 1.4.4. I also see an error when directly submitting via the access policy UI.

[garrardkitchen]

I am seeing this error with aws-cli/1.11.149 Python/2.7.9 Windows/8 botocore/1.7.7. I am surprised to see that this is still an issue after such a long time. I am now using @quiver AWS CLI suggestion. It is a real shame this can't be done via CF. It ruins the click to deploy methodology and makes the deployment process disjointed.

[willthames]

I had this problem when attempting to use an IAM based access policy but forgot to pass VPCOptions. Just in case anyone else had the same issue.

[digitalsanctum]

For posterity I also had a similar issue and turned out I was referring to an IAM role that didn't exist in the policy.

Not obvious at all from the "InvalidTypeException" exception message :(

[ashwinaggarwal]

I was getting this error when setting Principal with wildcard.

Works:

AccessPolicies:
    Version: '2012-10-17'
    Statement:
    - Effect: Allow
        Principal:
        AWS:
            - Fn::Join:
                - ":"
                - - arn:aws:iam
                - ''
                - Ref: AWS::AccountId
                - role/${self:service.name}-${opt:stage, 'dev'}-${self:provider.region}-lambdaRole
        Action: 'es:*'
        Resource:
        - Fn::Join:
            - ":"
            - - arn:aws:es
                - ${self:provider.region}
                - Ref: AWS::AccountId
                - domain/${self:service.name}-${opt:stage, 'dev'}/*

Doesn't work:

AccessPolicies:
    Version: '2012-10-17'
    Statement:
    - Effect: Allow
        Principal:
        AWS:
            - Fn::Join:
                - ":"
                - - arn:aws:iam
                - ''
                - Ref: AWS::AccountId
                - role/${self:service.name}-${opt:stage, 'dev'}*
        Action: 'es:*'
        Resource:
        - Fn::Join:
            - ":"
            - - arn:aws:es
                - ${self:provider.region}
                - Ref: AWS::AccountId
                - domain/${self:service.name}-${opt:stage, 'dev'}/*

Notice the * in the end of principal.

[swetashre]

@fperks - Does solution provided by @quiver work for you ? I am marking this as needs_sample considering many people are affected by this. We will see if we can provide some code example that can help everyone.

[sheteshrikant54]

For posterity I also had a similar issue and turned out I was referring to an IAM role that didn't exist in the policy.

Not obvious at all from the "InvalidTypeException" exception message :(

Finally, something worked !!
Yes, it's not at all obvious from the "InvalidTypeException" exception message :(