Aws-sam-cli: sam local invoke fails with AccessDeniedException when working with Layers

[kulkaamb]

Description:

When you use sam local invoke on a Lambda having layers, it fails with

Error: Credentials provided are missing lambda:Getlayerversion policy that is needed to download the layer or you do not have permission to download the layer

The --profile used to run the command had access lambda:* access but sam still fails with this error. Access is verified with aws commandline.

Steps to reproduce the issue:

1. Declare a template with Layers, something like
   Layers:
   - !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:layer:XRayLayer:${XRayLayerVersion}"
2. When Lambda is locally invoked for testing :
   sam local invoke -e .test-datatest-sqs-event.json myservice --profile default --debug --region us-east-1
   function invocation fails with

Error: Credentials provided are missing lambda:Getlayerversion policy that is needed to download the layer or you do not have permission to download the layer

1. Running the function in debug mode does not help, it does not reveal any additional details.
2. Running the function with start-lambda in debug mode and invoking the lambda with aws cli gives more insights, and there is a bug

sam local start-lambda --profile default --debug --region us-east-1

GetLayer call in the background is making call to a dummy account 123456789012 and not the actual account you are working with : log shows something like this

botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the GetLayerVersion operation: User: arn:aws:sts::[account]:assumed-role/[Role]/user is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:us-east-1:123456789012:layer:XRayLayer:6 and that's the bug
it should be using the account from aws --profile

See attached log for both sam local start-lambda and local invoke giving details.

sam bugreport.txt

Observed result:
AccessDeniedException

Expected result:
sam is supposed to download the layer specified in the sam template.

Workaround:
if you hardcode the account in layer ARN in template, it downloads the layer and then subsequent runs.

[jlhood]

Transferring to SAM CLI repo.

[cvamsikrishna11]

Hello all
Did anyone found a solution to this? even I am facing the same issue :(
But it's working fine if I hardcode AWS account ID
As mentioned by @kulkaamb
${AWS::AccountId} --> returning dummy AWS account ID: 123456789012

[SergioFloresG]

any update on this case?
I have the same problem.

[github4es]

any updates? I need to invoke a lambda running locally but it is connecting to an S3 bucket that is locked. I need to know how do I add permissions to the S3 bucket so that it lets my lambda running locally to read an S3 object?

[SergioFloresG]

https://github.com/aws/aws-sam-cli/blob/0c77ad035ea6408e0fad778c0e1c55e4db0bff55/samcli/commands/local/cli_common/invoke_context.py#L189

session = boto3.session.Session(profile_name=self._aws_profile, region_name=self._aws_region)
lambda_client = session.client('lambda')
layer_downloader = LayerDownloader(self._layer_cache_basedir, self.get_cwd(), lambda_client)

[PierreKiwi]

I have "fixed" this issue by using Mappings but it will be way more elegant to solve this problem by either interpolating !Sub:

Mappings:
  common:
    us-east-2:
      arnlayerpowertools: "arn:aws:lambda:us-east-2:ACCOUNT_ID:layer:aws-lambda-powertools-python-layer:5"

    us-west-2:
      arnlayerpowertools: "arn:aws:lambda:us-west-2:ACCOUNT_ID:layer:aws-lambda-powertools-python-layer:5"

...
    Layers:
        - !FindInMap [common, !Ref AWS::Region, arnlayerpowertools]

Also frustrating that SAM does not support passing the Lambda Layer ARN via a AWS::SSM::Parameter::Value<String> parameter (which is even better I think).

[Bradleywboggs]

The problem is broader than just parameterizing Layers. There doesn't seem to be away to override the value of the pseudo parameter AWS::AccountId for local function invocation in any node in the templates, which means I always have to hardcode the actual account Id in my templates when developing locally and then remember to swap it out so I have a properly parameterized template for deploying to the various accounts/envs. The other way around I found was pass in accountId as a parameter override and just reference that in my templates, but even that fails in the Layers node.