Copilot-cli: ValidationError: [/Resources/TaskRole/Type/ManagedPolicyArns] 'null' values are not allowed in templates

[ivan-myob]

copilot svc package gives me task with null ManagedPolicyArns. I've attached copilot folder with add-ons stack (if I remove addons stack it works fine).

AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation template that represents a load balanced web service on Amazon ECS.
Parameters:
  AppName:
    Type: String
  EnvName:
    Type: String
  ServiceName:
    Type: String
  ContainerImage:
    Type: String
  ContainerPort:
    Type: Number
  RulePath:
    Type: String
  TaskCPU:
    Type: String
  TaskMemory:
    Type: String
  TaskCount:
    Type: Number
  HTTPSEnabled:
    Type: String
    AllowedValues: [true, false]
  LogRetention:
    Type: Number
  AddonsTemplateURL:
    Description: 'URL of the addons nested stack template within the S3 bucket.'
    Type: String
    Default: ""
  HealthCheckPath:
    Type: String
  TargetContainer:
    Type: String
  TargetPort:
    Type: Number
Conditions:
  HTTPLoadBalancer:
    !Not
      - !Condition HTTPSLoadBalancer
  HTTPSLoadBalancer:
    !Equals [!Ref HTTPSEnabled, true]
  HasAddons: # If a bucket URL is specified, that means the template exists.
    !Not [!Equals [!Ref AddonsTemplateURL, ""]]
  HTTPRootPath: # If we're using path based routing and use the root path, we have some special logic
    !Equals [!Ref RulePath, "/"]
Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Join ['', [/copilot/, !Ref AppName, '-', !Ref EnvName, '-', !Ref ServiceName]]
      RetentionInDays: !Ref LogRetention

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    DependsOn: LogGroup
    Properties:
      Family: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref ServiceName]]
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      Cpu: !Ref TaskCPU
      Memory: !Ref TaskMemory
      ExecutionRoleArn: !Ref ExecutionRole
      TaskRoleArn: !Ref TaskRole
      ContainerDefinitions:
        - Name: !Ref ServiceName
          Image: !Ref ContainerImage
          PortMappings:
            - ContainerPort: !Ref ContainerPort
          # We pipe certain environment variables directly into the task definition.
          # This lets customers have access to, for example, their LB endpoint - which they'd
          # have no way of otherwise determining.
          Environment:
          - Name: COPILOT_APPLICATION_NAME
            Value: !Sub '${AppName}'
          - Name: COPILOT_SERVICE_DISCOVERY_ENDPOINT
            Value: !Sub '${AppName}.local'
          - Name: COPILOT_ENVIRONMENT_NAME
            Value: !Sub '${EnvName}'
          - Name: COPILOT_SERVICE_NAME
            Value: !Sub '${ServiceName}'
          - Name: COPILOT_LB_DNS
            Value:
              Fn::ImportValue:
                !Sub "${AppName}-${EnvName}-PublicLoadBalancerDNS" 
          - Name: ELASTICSEARCH_ENDPOINT
            Value:
              Fn::GetAtt: [AddonsStack, Outputs.ElasticsearchEndpoint]
          - Name: KIBANA_URL
            Value:
              Fn::GetAtt: [AddonsStack, Outputs.KibanaURL]
          - Name: ELASTICSEARCH_DOMAIN_ARN
            Value:
              Fn::GetAtt: [AddonsStack, Outputs.ElasticsearchDomainARN]
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-region: !Ref AWS::Region
              awslogs-group: !Ref LogGroup
              awslogs-stream-prefix: copilot


  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref ServiceName, SecretsPolicy]]
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - 'ssm:GetParameters'
                Resource:
                  - !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/*'
                Condition:
                  StringEquals:
                    'ssm:ResourceTag/copilot-application': !Sub '${AppName}'
                    'ssm:ResourceTag/copilot-environment': !Sub '${EnvName}'
              - Effect: 'Allow'
                Action:
                  - 'secretsmanager:GetSecretValue'
                Resource:
                  - !Sub 'arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*'
                Condition:
                  StringEquals:
                    'secretsmanager:ResourceTag/copilot-application': !Sub '${AppName}'
                    'secretsmanager:ResourceTag/copilot-environment': !Sub '${EnvName}'
              - Effect: 'Allow'
                Action:
                  - 'kms:Decrypt'
                Resource:
                  - !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/*'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'


  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: 'DenyIAMExceptTaggedRoles'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Deny'
                Action: 'iam:*'
                Resource: '*'
              - Effect: 'Allow'
                Action: 'sts:AssumeRole'
                Resource:
                  - !Sub 'arn:aws:iam::${AWS::AccountId}:role/*'
                Condition:
                  StringEquals:
                    'iam:ResourceTag/copilot-application': !Sub '${AppName}'
                    'iam:ResourceTag/copilot-environment': !Sub '${EnvName}'


  DiscoveryService:
    Type: AWS::ServiceDiscovery::Service
    Properties:
      Description: Discovery Service for the Copilot services
      DnsConfig:
        RoutingPolicy: MULTIVALUE
        DnsRecords:
          - TTL: 60
            Type: A
          - TTL: 60
            Type: SRV
      HealthCheckCustomConfig:
        FailureThreshold: 1
      Name:  !Ref ServiceName
      NamespaceId:
        Fn::ImportValue:
          !Sub '${AppName}-${EnvName}-ServiceDiscoveryNamespaceID'

  Service:
    Type: AWS::ECS::Service
    DependsOn: WaitUntilListenerRuleIsCreated
    Properties:
      Cluster:
        Fn::ImportValue:
          !Sub '${AppName}-${EnvName}-ClusterId'
      TaskDefinition: !Ref TaskDefinition
      DesiredCount: !Ref TaskCount
      PropagateTags: SERVICE
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: ENABLED
          Subnets:
            - Fn::Select:
              - 0
              - Fn::Split:
                - ','
                - Fn::ImportValue: !Sub '${AppName}-${EnvName}-PublicSubnets'
            - Fn::Select:
              - 1
              - Fn::Split:
                - ','
                - Fn::ImportValue: !Sub '${AppName}-${EnvName}-PublicSubnets'
          SecurityGroups:
            - Fn::ImportValue: !Sub '${AppName}-${EnvName}-EnvironmentSecurityGroup'
      DeploymentConfiguration:
        MinimumHealthyPercent: 100
        MaximumPercent: 200
      # This may need to be adjusted if the container takes a while to start up
      HealthCheckGracePeriodSeconds: 60
      LoadBalancers:
        - ContainerName: !Ref TargetContainer
          ContainerPort: !Ref TargetPort
          TargetGroupArn: !Ref TargetGroup
      ServiceRegistries:
        - RegistryArn: !GetAtt DiscoveryService.Arn
          Port: !Ref ContainerPort

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      #  Check if your service is healthy within 20 = 10*2 seconds, compared to 2.5 mins = 30*5 seconds.
      HealthCheckIntervalSeconds: 10 # Default is 30.
      HealthyThresholdCount: 2       # Default is 5.
      HealthCheckTimeoutSeconds: 5
      HealthCheckPath: !Ref HealthCheckPath
      Port: !Ref ContainerPort
      Protocol: HTTP
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: 60                  # Default is 300.
      TargetType: ip
      VpcId:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-VpcId"

  LoadBalancerDNSAlias:
    Type: AWS::Route53::RecordSetGroup
    Condition: HTTPSLoadBalancer
    Properties:
      HostedZoneId:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-HostedZone"
      Comment: !Sub "LoadBalancer alias for service ${ServiceName}"
      RecordSets:
      - Name:
          !Join
            - '.'
            - - !Ref ServiceName
              - Fn::ImportValue:
                  !Sub "${AppName}-${EnvName}-SubDomain"
              - ""
        Type: A
        AliasTarget:
          HostedZoneId:
            Fn::ImportValue:
              !Sub "${AppName}-${EnvName}-CanonicalHostedZoneID"
          DNSName:
            Fn::ImportValue:
              !Sub "${AppName}-${EnvName}-PublicLoadBalancerDNS"

  RulePriorityFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          'use strict';const aws=require("aws-sdk"),priorityForRootRule="50000";let defaultResponseURL,report=function(a,b,c,d,e,f){return new Promise((g,h)=>{const i=require("https"),{URL:j}=require("url");var k=JSON.stringify({Status:c,Reason:f,PhysicalResourceId:d||b.logStreamName,StackId:a.StackId,RequestId:a.RequestId,LogicalResourceId:a.LogicalResourceId,Data:e});const l=new j(a.ResponseURL||defaultResponseURL),m={hostname:l.hostname,port:443,path:l.pathname+l.search,method:"PUT",headers:{"Content-Type":"","Content-Length":k.length}};i.request(m).on("error",h).on("response",a=>{a.resume(),400<=a.statusCode?h(new Error(`Error ${a.statusCode}: ${a.statusMessage}`)):g()}).end(k,"utf8")})};const calculateNextRulePriority=async function(a){var b,c=new aws.ELBv2,d=[];do{const e=await c.describeRules({ListenerArn:a,Marker:b}).promise();d=d.concat(e.Rules),b=e.NextMarker}while(b);let e=1;if(0<d.length){const a=d.map(a=>"default"===a.Priority||a.Priority===priorityForRootRule?0:parseInt(a.Priority)),b=Math.max(...a);e=b+1}return e};exports.nextAvailableRulePriorityHandler=async function(a,b){var c,d,e={};try{switch(a.RequestType){case"Create":d=await calculateNextRulePriority(a.ResourceProperties.ListenerArn),e.Priority=d,c=`alb-rule-priority-${a.LogicalResourceId}`;break;case"Update":case"Delete":c=a.PhysicalResourceId;break;default:throw new Error(`Unsupported request type ${a.RequestType}`);}await report(a,b,"SUCCESS",c,e)}catch(d){console.log(`Caught error ${d}.`),await report(a,b,"FAILED",c,null,d.message)}},exports.withDefaultResponseURL=function(a){defaultResponseURL=a};
      Handler: "index.nextAvailableRulePriorityHandler"
      Timeout: 600
      MemorySize: 512
      Role: !GetAtt 'CustomResourceRole.Arn'
      Runtime: nodejs10.x

  CustomResourceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: "DNSandACMAccess"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
                - elasticloadbalancing:DescribeRules
              Resource: "*"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

  HTTPSRulePriorityAction:
    Condition: HTTPSLoadBalancer
    Type: Custom::RulePriorityFunction
    Properties:
      ServiceToken: !GetAtt RulePriorityFunction.Arn
      ListenerArn:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-HTTPSListenerArn"

  HTTPSListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Condition: HTTPSLoadBalancer
    Properties:
      Actions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward
      Conditions:
        - Field: 'host-header'
          HostHeaderConfig:
            Values:
              - Fn::Join:
                - '.'
                - - !Ref ServiceName
                  - Fn::ImportValue:
                      !Sub "${AppName}-${EnvName}-SubDomain"
      ListenerArn:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-HTTPSListenerArn"
      Priority: !GetAtt HTTPSRulePriorityAction.Priority

  HTTPRulePriorityAction:
    Condition: HTTPLoadBalancer
    Type: Custom::RulePriorityFunction
    Properties:
      ServiceToken: !GetAtt RulePriorityFunction.Arn
      ListenerArn:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-HTTPListenerArn"

  HTTPListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Condition: HTTPLoadBalancer
    Properties:
      Actions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward
      Conditions:
        - Field: 'path-pattern'
          PathPatternConfig:
            Values:
              !If
                - HTTPRootPath
                -
                  - "/*"
                -
                  - !Sub "/${RulePath}"
                  - !Sub "/${RulePath}/*"
      ListenerArn:
        Fn::ImportValue:
          !Sub "${AppName}-${EnvName}-HTTPListenerArn"
      Priority: 
        !If
          - HTTPRootPath
          - 50000 # This is the max rule priority. Since this rule evaluates true for everything, we make sure it is last
          - !GetAtt HTTPRulePriorityAction.Priority

  # Force a conditional dependency from the ECS service on the listener rules.
  # Our service depends on our HTTP/S listener to be set up before it can
  # be created. But, since our environment is either HTTPS or not, we
  # have a conditional dependency (we have to wait for the HTTPS listener
  # to be created or the HTTP listener to be created). In order to have a
  # conditional dependency, we use the WaitHandle resource as a way to force
  # a single dependency. The Ref in the WaitCondition implicitly creates a conditional
  # dependency - if the condition is satisfied (HTTPLoadBalancer) - the ref resolves
  # the HTTPWaitHandle, which depends on the HTTPListenerRule.

  HTTPSWaitHandle:
    Condition: HTTPSLoadBalancer
    DependsOn: HTTPSListenerRule
    Type: AWS::CloudFormation::WaitConditionHandle

  HTTPWaitHandle:
    Condition: HTTPLoadBalancer
    DependsOn: HTTPListenerRule
    Type: AWS::CloudFormation::WaitConditionHandle

  # We don't actually need to wait for the condition to
  # be completed, that's why we set a count of 0. The timeout
  # is a required field, but useless, so we set it to one.
  WaitUntilListenerRuleIsCreated:
    Type: AWS::CloudFormation::WaitCondition
    Properties:
      Handle: !If [HTTPLoadBalancer, !Ref HTTPWaitHandle, !Ref HTTPSWaitHandle]
      Timeout: "1"
      Count: 0

  AddonsStack:
    Type: AWS::CloudFormation::Stack
    Condition: HasAddons
    Properties:
      Parameters:
        App: !Ref AppName
        Env: !Ref EnvName
        Name: !Ref ServiceName
      TemplateURL:
        !Ref AddonsTemplateURL

copilot.zip

[efekarakus]

@ivan-myob Hi Ivan! If you have addons, can you try the command with the --output-dir flag?

$ copilot svc package --output-dir ./infrastructure

Regardless we should handle this error for a friendlier message

[ivan-myob]

I attached zip files with addons to my previous message, but here you go.
infrastructure.zip

[ivan-myob]

Any ideas how I can quickly make this work? Quick and dirty fix will be sufficient as well =)

[efekarakus]

Ooh I think I misunderstood the issue earlier, can you try adding an IAM ManagedPolicy in your addon stack? like described here: https://github.com/aws/copilot-cli/wiki/Additional-AWS-Resources and output the ARN of the policy I think that might fix the issue.

# In your addons/elaticsearch.yaml
Resources:
  # ... Your existing resources
  ESManagedPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: ESPermissions
            Action: es:*
            Effect: Allow
            Resource: *
Outputs:
  KibanaURL:
    Description: Kibana URL
    Value: !Join 
      - ''
      - - !GetAtt 
          - ElasticsearchDomain
          - DomainEndpoint
        - /_plugin/kibana/
  ElasticsearchEndpoint:
    Description: Elasticsearch domain endpoint
    Value: !GetAtt 
      - ElasticsearchDomain
      - DomainEndpoint
  ElasticsearchDomainARN:
    Description: Elasticsearch domain ARN
    Value: !GetAtt 
      - ElasticsearchDomain
      - DomainArn
  ESManagedPolicyARN:
    Value: !Ref ESManagedPolicy

Looks like we have a bug where we expect a ManagedPolicy to be always there in an addon stack.

[efekarakus]

Hi @ivan-myob, the comment above should unblock you and #1075 should fix the issue long-term!

[ivan-myob]

it helped, thank you @efekarakus !