Copilot-cli: bug: prompt users for Dockerfile location if file not found at root

[elhanarinc]

Hello,

We are using monorepo for our product and we have 4 different services which are inside one root folder with their responsible dockerfiles and we have common packages which could be used more than one services. As i understand from the tutorials right now dockerfile location is supported only on the root folder and one sub folder. So i just want to learn about adding custom dockerfile location support will be added for future releases or am i missing something ?

Thanks.

[efekarakus]

Hello @elhanarinc :wave: !

You should be able to modify the manifest to change the location of the Dockerfile to be deeper than 1-level from the root. For example, ./src/<myService>/Dockerfile. Would that work for you?

We're also currently working on https://github.com/aws/copilot-cli/pull/1059, it's in review phase which will allow you to specify the context, dockerfile location, and build args while building the container image.

[elhanarinc]

Hello @elhanarinc 👋 !

You should be able to modify the manifest to change the location of the Dockerfile to be deeper than 1-level from the root. For example, ./src/<myService>/Dockerfile. Would that work for you?

We're also currently working on #1059, it's in review phase which will allow you to specify the context, dockerfile location, and build args while building the container image.

Hello again,

First of all thanks for the explanation 🙏 .

When i tried to run copilot init on the root directory, since i do not have a dockerfile on that level i got this error:
✘ ask svc init: no Dockerfiles found within . or a sub-directory level below

So, in order to solve this issue should i add an empty dockerfile, run copilot init, after the manifest file created change the manifest according to my needs ?

Ps: Since i am using some common packages which are on the root level for the different services i thought that copilot init should be run on the root folder so that docker context actually see and copy them.

Thanks again.

[efekarakus]

Ohh thank you that looks like a bug! A better behavior would be to prompt for the location of the Dockerfile if we can't find any.

You're right I think as a work around for now you can create a dummy Dockerfile at the root and then modify the manifest file. Sorry for the trouble!

[efekarakus]

I'll keep this open for us so that we can provide a better UX by catching the "no Dockerfiles found within . or a sub-directory level below" error :)

[bvtujo]

Hi @elhanarinc, you can also specify a custom Dockerfile location with the --dockerfile flag when running copilot init or copilot svc init. This prevents you from having to edit the manifest or create a placeholder Dockerfile when initializing your services or applications.

$ copilot init --dockerfile service/one/Dockerfile

We'll definitely work to improve the messaging on this error so it's more clear!

[elhanarinc]

Hello again,

Thanks for all help, --dockerfile flag is indeed working but when you give the directory location of the dockerfile as a flag, docker daemon context will see only that sub directory even though you start the copilot from the root folder.

My workaround was copying the dockerfile related with that service into root folder, starting copilot init and after copilot is done deleting the dockerfile.

[oren]

I am having similar issue. This is the first time I am trying copilot. I am on WSL, running ubuntu.

git clone https://github.com/aws-samples/aws-copilot-sample-service example
cd example
copilot init

...
...
...

✔ Linked account 478079566801 and region us-west-2 to application communities-of-excellence.

✔ Created environment test in region us-west-2 under application communities-of-excellence.
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /local/home/oreng/Dockerfile: no such file or directory
✘ build and push image: build Dockerfile at /home/oreng/Dockerfile: building image: exit status 1

my solution was to copy the Dockerfile to /home/oren/

[oren]

that didn't work. I ended up with this error:

✔ Created environment test in region us-west-2 under application communities-of-excellence.
Sending build context to Docker daemon 15.82GB
Step 1/3 : FROM nginx
latest: Pulling from library/nginx
bf5952930446: Pull complete
cb9a6de05e5a: Pull complete
9513ea0afb93: Pull complete
b49ea07d2e93: Pull complete
a5e4a503d449: Pull complete
Digest: sha256:b0ad43f7ee5edbc0effbc14645ae7055e21bc1973aee5150745632a24a752661
Status: Downloaded newer image for nginx:latest
---> 4bb46517cac3
Step 2/3 : EXPOSE 80
---> Running in b74758603d62
Removing intermediate container b74758603d62
---> 5618f2f3687b
Step 3/3 : COPY index.html /usr/share/nginx/html
COPY failed: stat /local/docker/tmp/docker-builder243733838/index.html: no such file or directory
✘ build and push image: build Dockerfile at /home/oreng/Dockerfile: building image: exit status 1

[bvtujo]

Hi @oren, is the --dockerfile flag not working for you? It seems based on the error trace above that like Docker (and Copilot) is not being run in the root of your workspace, which in the code above should be the example directory. We recommend that you run copilot init in the root of your Git repository.

You can also specify the Dockerfile location after initialization by customizing the manifest located at copilot/${SERVICE_NAME}/manifest.yml. The build key defaults to the dockerfile location but can be expanded to a map. In your case it looks like your copilot dir is located at /home/Oren/copilot and dockerfile is at /home/oren/example/Dockerfile so you could update the build field in the manifest to the following:

build:
  dockerfile: example/Dockerfile
  context: example

You can see more about the build field on our wiki.

Hope this helps!

[oren]

note: I am running everything inside example folder.

a few questions:

• --dockerfile didn't work. I can't remember the error message
• what is a workspace?
• what is the purpose of the folder copilot and was it created by the CLI?
• how to delete my app? I tried copilot app delete but it tells me 'couldn't fid an app attached to this workspace'. but copilot app ls shows me an app.

[oren]

I rm the executable and re-installed copilot. I still see an app when running copilot app ls. where is copilot saving this info?

[bvtujo]

Hi @oren, Copilot stores app information locally in the copilot directory at copilot/.workspace, and in the cloud via SSM parameters through your default AWS credentials. You can get rid of the app by deleting the the local .workspace file.

Copilot uses a YAML manifest and Cloudformation to store information about your service configuration, so we create the copilot directory to hold infrastructure files.

[oren]

I still see my app even after deleting ~/copilot:

rm ~/copilot
copilot app show

Only found one application, defaulting to: communities-of-excellence
About

  Name              communities-of-excellence
  URI

Environments
                                                                                                                                      Name              AccountID           Region
  test              478079566801        us-west-2

Services

  Name              Type
  front-end         Load Balanced Web Service  

[iamhopaul123]

Hello @oren, app show gets from SSM param store so locally deleting copilot dir doesn't completely delete the app. You might want to do app delete to completely delete your app.

[oren]

That's didn't help:

copilot app delete
✘ could not find an application attached to this workspace, please run app init first
copilot app ls
communities-of-excellence

[oren]

I also deleted all the cloudformation stacks from my account but I still see the app in the CLI.

[iamhopaul123]

oh that is because the local is out of sync...a workaround to this is to manually delete the param with your copilot app name from your System manager parameter store.

[oren]

deleting 3 entries from 'System manager parameter store' worked. Thanks!

[iamhopaul123]

Thanks @oren. I know it is kind of cumbersome to manually delete scattered resources when your app is out of sync. We should address this problem to make it easier.

[oren]

I am still having issues with the CLI:

copilot app ls
chatbot

copilot app show
✘ get application communities-of-excellence: couldn't find an application named communities-of-excellence in account 478079566801 and region us-west-2

[oren]

actually, i was looking at the wrong place. I have copilot installed on two different environments. so i think i am good for now.

[bvtujo]

@elhanarinc as of our latest release you can configure the context directory passed to the docker daemon by editing the manifest.

Previously, the manifest's build section looked like this:

build: path/to/dockerfile

As of #1059 being merged and released in v0.3.0, you can now specify the build section as a map, like so:

build:
  dockerfile: path/to/dockerfile
  context: different/context/dir

This should solve your monorepo problem!