Copilot-cli: How to create HTTPS Load balanced web service?

[ivanbarlog]

Hi there,

I was trying to find any information about how to create HTTPS instead of HTTP "Load Balanced Web Service" but unfortunately I was unable to find any information about this except Setting custom domain for the application which is not working for me.

There are few issues which slightly talks about my problem for instance #1188 or #1342 but I guess that that's not implemented yet - not sure though.

Not sure how the copilot exactly works if I could do manual changes on deployed ALB (which I would preferably won't do if it could be automated) or if there is any mechanism to do a changes right inside manifest.yml

Thanks for any feedback

[efekarakus]

Heya @ivanbarlog !

The tutorial in this repository might also help: https://github.com/kohidave/dns-and-lb-copilot-demo. As long as the domain name registered with Route53 is in your Copilot application's account then the --domain flag should be working.

Setting custom domain for the application which is not working for me.

What part of it didn't work for you?

[ivanbarlog]

Thank you @efekarakus ,

the tutorial you've posted here seems to be very helpful. I need to check it out sometime. Unfortunately I have already scraped the whole copilot infrastructure and replaced it with CloudFormation template. I hoped that copilot could resolve my issues with the ease but I guess it is still a bit confusing/unclear how it works so I guess I will need to revisit it in some time.

Nevertheless I will try the tutorial and I will let you know if it resolves my issues :-)

Regards the --domain not working - I haven't seem to get any particular error, it seems that the copilot just hanged there without doing anything. I have had created the hosted zone within the account for the testing domain I have within another account which shouldn't be problem as we do this most of the time - main account which holds all the domains and sub accounts which have just hosted zones for the domains.

I would need to retry it.

[HiddenDeveloper]

I can confirm the steps worked fine for me. I already had a domain registered in Route53 and used the following commands.

$ copilot app init --domain MY_REGISTERED_DOMAIN

copilot created the infrastructure to manage the services and prompted with a recommendation to run copilot init

$ copilot init

After the command completed the https url was available to access my container.

Great Project. Many Thanks.

[mebibou]

How to have the ALB add a https 443 listener without using the domain flag? my domain is not registered in route53 and when I deploy it only creates a listener on port 80. I added another 443 listener myself but it keeps being deleted on new deploy

[bpottier]

I was able to change the listener rules and not have my changes reverted when running "svc deploy". What is the expected behavior? Also, what is the best practice approach here for applications with domains either managed outside Route53 or in another account?

[kohidave]

How to have the ALB add a https 443 listener without using the domain flag? my domain is not registered in route53 and when I deploy it only creates a listener on port 80. I added another 443 listener myself but it keeps being deleted on new deploy

Hello! So unfortunately, to get HTTPS you need to have a domain name. This is because in order for HTTPS on an ALB, you need to provide your own TLS Certificate. Copilot will generate this for you if you provide a domain name - but without a domain name, you can't create a TLS certificate :(

For non-Route53 hosted domain names, I'd recommend creating a hosted zone in your AWS account in Route53 that has delegated access to that domain. You can do this by:

1. Creating a route53 hosted zone
2. Copying the NS records from that hosted zone into the DNS zone where your DNS name lives
3. Run copilot

I can't think of a way to automate that - the cross-registrar domain name validation.

[kohidave]

I was able to change the listener rules and not have my changes reverted when running "svc deploy". What is the expected behavior? Also, what is the best practice approach here for applications with domains either managed outside Route53 or in another account?

You can change the listener rule and Copilot won't overwrite it unless you change some configuration in your manifest (like path, stickiness, etc) - then it might be overwritten.

For non-AWS hosted domains, I'd recommend the delegation method I mentioned above - essentially creating a Route53 hosted zone with access to your domain.

[bpottier]

How to have the ALB add a https 443 listener without using the domain flag? my domain is not registered in route53 and when I deploy it only creates a listener on port 80. I added another 443 listener myself but it keeps being deleted on new deploy

Hello! So unfortunately, to get HTTPS you need to have a domain name. This is because in order for HTTPS on an ALB, you need to provide your own TLS Certificate. Copilot will generate this for you if you provide a domain name - but without a domain name, you can't create a TLS certificate :(

For non-Route53 hosted domain names, I'd recommend creating a hosted zone in your AWS account in Route53 that has delegated access to that domain. You can do this by:

1. Creating a route53 hosted zone
2. Copying the NS records from that hosted zone into the DNS zone where your DNS name lives
3. Run copilot

I can't think of a way to automate that - the cross-registrar domain name validation.

Is it possible to add the domain after already creating the application?

[kohidave]

Ugh - yea, that's an issue right now - you can't. We're working on revamping our DNS support so you can add the DNS name after the fact and have more flexibility with it.

[bpottier]

Ugh - yea, that's an issue right now - you can't. We're working on revamping our DNS support so you can add the DNS name after the fact and have more flexibility with it.

Bummer. Okay. So I'm guessing I need to destroy the application and re-create it with the domain?

[kohidave]

Ugh - yea, that's an issue right now - you can't. We're working on revamping our DNS support so you can add the DNS name after the fact and have more flexibility with it.

Bummer. Okay. So I'm guessing I need to destroy the application and re-create it with the domain?

For now, I'm afraid so 🙏 I'm really sorry about this - this is the exact use case we're trying to address with our new DNS support design.

[mebibou]

I was able to change the listener rules and not have my changes reverted when running "svc deploy". What is the expected behavior? Also, what is the best practice approach here for applications with domains either managed outside Route53 or in another account?

You can change the listener rule and Copilot won't overwrite it unless you change some configuration in your manifest (like path, stickiness, etc) - then it might be overwritten.

For non-AWS hosted domains, I'd recommend the delegation method I mentioned above - essentially creating a Route53 hosted zone with access to your domain.

Hum I'm not too sure why then but at the beginning I deployed the app in 2 different environments, for the first one I did create the listener directly in the console, and it seems to be sticking there, but for the second environment it kept removing it. So I ended up creating a little script that will add the https listener after I deploy:

#!/bin/sh

SERVICE=$1
ENV=$2

CERTI_ARN="$(aws acm list-certificates | jq -r '.CertificateSummaryList[0] .CertificateArn')"
LB_DNS="$(copilot svc show --name $SERVICE --json | jq '.variables[] | select((.environment == "'$ENV'") and (.name == "COPILOT_LB_DNS")) | .value')"
LB_ARN="$(aws elbv2 describe-load-balancers --output json | jq -r '.LoadBalancers[] | select(.DNSName == '$LB_DNS') | .LoadBalancerArn')"
TG_ARN="$(copilot svc show --name $SERVICE --json --resources | jq -r '.resources | .["'${ENV}'"][] | select((.type == "AWS::ElasticLoadBalancingV2::TargetGroup")) | .physicalID')"

LISTENER_ARN="$(aws elbv2 describe-listeners --load-balancer-arn $LB_ARN --output json | jq -r '.Listeners[] | select(.Port == 443) | .ListenerArn')"
if [ -z "$LISTENER_ARN" ]
then
  aws elbv2 create-listener \
    --load-balancer-arn $LB_ARN \
    --protocol HTTPS \
    --port 443 \
    --ssl-policy "ELBSecurityPolicy-TLS-1-2-Ext-2018-06" \
    --certificates CertificateArn=$CERTI_ARN \
    --default-actions Type=forward,TargetGroupArn=$TG_ARN
else
  echo "Nothing to do."
fi

So to fully-deploy I will do:

copilot deploy --name api --env test && ./update-lb.sh api test

And if the listener is not present (which happens in 1 env but not the other, dont know why really), the script will create the listener correctly.

[mebibou]

How to have the ALB add a https 443 listener without using the domain flag? my domain is not registered in route53 and when I deploy it only creates a listener on port 80. I added another 443 listener myself but it keeps being deleted on new deploy

Hello! So unfortunately, to get HTTPS you need to have a domain name. This is because in order for HTTPS on an ALB, you need to provide your own TLS Certificate. Copilot will generate this for you if you provide a domain name - but without a domain name, you can't create a TLS certificate :(

For non-Route53 hosted domain names, I'd recommend creating a hosted zone in your AWS account in Route53 that has delegated access to that domain. You can do this by:

1. Creating a route53 hosted zone
2. Copying the NS records from that hosted zone into the DNS zone where your DNS name lives
3. Run copilot

I can't think of a way to automate that - the cross-registrar domain name validation.

I think not everyone (like in my case) can just move over the DNS zone to route53 to just create an https listener... seems a lot easier to include a way to create one, doesn't it?

[MatteoGioioso]

@mebibou thank you for that snippet!
In my case I had to add --app ${MY_APP} to the copilot svc show to make it work.
I have re-arranged it in a cloudformation template, in the hope it won't delete it.

Parameters:
  LoadBalancerTargetGroup:
    Type: String
  LoadBalancerArn:
    Type: String
  CertificateArn:
    Type: String

Resources:
  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref LoadBalancerTargetGroup
      LoadBalancerArn: !Ref LoadBalancerArn
      Certificates:
        - CertificateArn: !Ref CertificateArn
      Port: 443
      Protocol: HTTPS

and then I have used your script to inject the parameters:

#!/usr/bin/env bash

export ENV=${1:-dev}
export APP=my-app
export AWS_PROFILE=${APP}-${ENV}
export REGION=my-region-1
export PROJECT=custom-${AWS_PROFILE}
export TEMPLATE_NAME=my-custom.template.yaml
export SERVICE=my-service

set -e

export CERTIFICATE_ARN="$(aws acm list-certificates | jq -r '.CertificateSummaryList[0] .CertificateArn')"
export LOAD_BALANCER_DNS="$(copilot svc show --app ${APP} --name ${SERVICE} --json | jq '.variables[] | select((.environment == "'"$ENV"'") and (.name == "COPILOT_LB_DNS")) | .value')"
export LOAD_BALANCER_ARN="$(aws elbv2 describe-load-balancers --output json | jq -r '.LoadBalancers[] | select(.DNSName == '"$LOAD_BALANCER_DNS"') | .LoadBalancerArn')"
export LOAD_BALANCER_TARGET_GROUP_ARN="$(copilot svc show --app $APP --name $SERVICE --json --resources | jq -r '.resources | .["'"${ENV}"'"][] | select((.type == "AWS::ElasticLoadBalancingV2::TargetGroup")) | .physicalID')"

sam deploy --profile "${AWS_PROFILE}" --region "${REGION}" \
  --template-file ${TEMPLATE_NAME} \
  --stack-name "${PROJECT}" \
  --capabilities CAPABILITY_IAM \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
  Env="${ENV}" \
  LoadBalancerTargetGroup="${LOAD_BALANCER_TARGET_GROUP_ARN}" \
  LoadBalancerArn="${LOAD_BALANCER_ARN}" \
  CertificateArn="${CERTIFICATE_ARN}"