Docker-letsencrypt-nginx-proxy-companion: Self-signed for development

[asmaloney]

I have some sites I want to migrate to HTTPS, so I want to work on them locally to make sure they work before moving the live sites.

If I use this container as part of my docker infrastructure, how do I use self-signed certs for my development environment?

Can I just create them and put them in the certs directory and everything will work? Or will this container try to check them/renew them with ACME and blow up?

(I'm relatively new to working with HTTPS and trying to understand how to work with it.)

[JrCs]

All you need to do is put your self signed certificates in the nginx certs directory.
Certs are created with this container ONLY if the webserver container defines the LETSENCRYPT_XXXX environment variables.

[asmaloney]

Just to clarify - if I don't set the env vars, this container will still see the self-signed certs in the certs directory and modify the nginx config to pick them up and use them, it just won't try to create or renew them?

[JrCs]

@asmaloney exactly

[asmaloney]

Great, thanks! And thank you again for creating this. Very helpful.

[kachkaev]

@JrCs could you please extend README.md a bit to help newcomers configure the dev environment? My app consists of a few docker-compose.yml files, the first of which sets up nginx/docker-gen/letsencrypt bundle. Other services are started later and expect the traffic to arrive from that bundle. When I'm on localhost, my root domain name is my-project.dev. The services I start are at a.my-project.dev, b.my-project.dev, etc. Because all the traffic is expected to go via https and letsencrypt fails to get the right certs, I can't open anything in the browser. Would be good to know in which folders and in what format to put these dummy certs you are mentioning.

[JrCs]

@kachkaev We don't have dummy certs. Certs are created by letsencrypt container in the nginx certs directory automatically.

[kachkaev]

@JrCs OK, thank you. I'm new to letsencrypt, so might be asking stupid questions. Given the configuration of the services I shared above, how would recommend organising the development environment?

I would like my future dockerized services my-project.com, a.my-project.com and b.my-project.com to stay behind nginx / docker-gen & letsencrypt even in development, while I play with them at my-project.dev, a.my-project.dev and b.my-project.dev. This means that there should be some mocking of https certs on my localhost so that I could open .dev domains in the browser in https mode. This will allow me to test the full stack before putting it to production.

What should I tweak to be able to use the same dockeer-compose.yml files in dev and prod (with different .env, of course)?

[JrCs]

You can't create certs for .dev domain because i don't think the root .dev domain exists.
For your dev environment you can use the .com certificates create by letsencrypt and when you connect to the .dev site allow to use an invalid certificate (a .com certificate for a .dev domain).

You can also access you .dev sites using http instead of https.

[kachkaev]

@JrCs this suggests that I can have a bunch of files on my laptop that represent a certificate for some domain (which is not any of my dev domains, of course). What I should do is to copy these files into /certs/ folder that nginx picks and then rename them somehow so that they represent a.my-project.dev and www.a.my-project.dev, for instance. What is unclear to me is what files to take and where to put them. I'm happy with a security warning in a browser when I access *.dev.

There could be a section in your readme that helped in such case. It could provide a link to the files that represent an invalid certificate and explain where to put these files to mock a successfully generated certificate. Nginx proxy will swallow them and will enable local testing of the complete stack of the docker services.

Would be really good if you could help with this.

[kachkaev]

Or maybe even docker-letsencrypt-nginx-proxy-companion could create some fake dev-mode certificates for my domains if I pass it some flag via an ENV variable? This would be just perfect!

[JrCs]

Just add some links in the nginx cert directory. Each must be name as a .dev domain (like www.a.my-project.dev) that point to the my-project.com multidomain certificate.
Nginx will use any certs that it will find in the nginx cert directory.

But i think it must be easier to access .dev domain directly (without https) and only proxyfied https to http the .com domains with nginx

[kachkaev]

I've been trying to copy and rename different files inside /etc/nginx/certs, but never got to a point when they were picked by nginx.

Looks like local testing of https in a multi-domain project is quite a pain these days even maybe if you are knowledgeable. You first have to obtain a bunch of files that represent some domain and then for each new domain manually go to /etc/nginx/certs and create symlinks. Then there's probably a need to trigger some hook afterwards.

An ideal solution would be just to start the container with something like SELF_SIGNED=true or DUMMY_CERTS=true or ACME_CI_URI=dummy to completely mock letsencrypt requests. Getting a red crossed lock icon for my-super-domain.dev should not take time at all with such a nice bundle of containers sitting in the very front of the web requests (I mean nginx, docker-gen and letsencrypt companion).

@asmaloney @JrCs could you please suggest some little cheat sheet on what to do to enable self-signed (or what ever dummy) certificates for the development domains? Let's assume I don't have any certificate in my local filesystem and I also don't have a production server yet (i.e. I can't copy my-super-domain.com.pem or whatever from there).

[asmaloney]

The solution I used wasn't exactly awesome, but it got me where I needed to go.

I did this a while ago, so I hope I'm not missing steps. I did the following:

• took docker-letsencrypt-nginx-proxy-companion completely out of the loop by not passing LETSENCRYPT_HOST
• generated a self-signed cert: openssl req -x509 -days 365 -keyout localhost.key -out localhost.crt
• copied the localhost.crt and localhost.key files into my certs directory
• changed VIRTUAL_HOST in my run command to point to localhost: -e "VIRTUAL_HOST=localhost"

I only used this on one domain at a time, but I imagine for multiple domains you could generate the certs as above with foo.local instead og localhost and then just make sure your host system resolves foo.local properly?

[kachkaev]

Thanks @asmaloney!

Yeah, this looks a bit tedious to do apply, especially if you want to have multiple domains and use dockercompose. I'm using a combination of variables from .env in my docker-compose.yml and the configs often contain the following:

services:
  web:
    environment:
      - VIRTUAL_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
      - VIRTUAL_PORT=80
      - VIRTUAL_NETWORK=proxy
      - LETSENCRYPT_HOST=${SUBDOMAIN}.${ROOT_HOST},www.${SUBDOMAIN}.${ROOT_HOST}
      - LETSENCRYPT_EMAIL=admin@${ROOT_HOST}

My goal is to start app1-prod.my-project.dev, app1-next.my-project.dev, app2.my-project.dev, etc. and then re-use the same git repos and images on the server just by changing ROOT_HOST and some DEV flag in the .env files. Rather than testing the containers locally one by one I see a huge benefit in running the whole project together. If letsencrypt-nginx-proxy-companion could work in some _dummy cert mode_ on a local machine while I'm playing with https://*.my-project.dev, it would be awesome.

[kachkaev]

@JrCs could you please consider automating self-signed certificates as an extra dev mode option in the companion? What is your opinion on this?

[JrCs]

@kachkaev Sorry but this companion is only used to create signed certificate from letsencrypt, no more no less.
If you want to create self-sign certificates use another container that will use the openssl tools to create this certificates.

[kachkaev]

@JrCs that's a bit of a shame as such functionality would be useful in some dev workflows, but I understand your view. Keeping the list of features as small as possible is a great, not a bad thing!

I might do a small investigation later on how to make https available on a local machine while testing websites behind nginx reverse-proxy and docker-gen. The goal would be to exclude any manual work and minimise the difference between docker-compose.yml on the local machine and on the real servers. Drop your ideas here if you're in the same situation as I am!

[hadim]

That feature would be really helpful !!!

[mvigneron]

Hello @kachkaev,

I am in the same situation. I would like to have https available in developpement environnement. did you manage to do that ?

Thanks for your help

[buchdag]

docker exec your-le-companion-container \
    openssl req -x509 \
    -newkey rsa:4096 \
    -sha256 \
    -nodes \
    -days 365 \
    -subj '/CN=yourdomain.tld'  \  
    -keyout /etc/nginx/certs/yourdomain.tld.key \
    -out /etc/nginx/certs/yourdomain.tld.crt

docker exec nginx-proxy nginx -s reload

[wvengen]

Thanks for the hints!

I'm now using something like this in my dev setup to generate certificates for all my virtual hosts before I bring up the stack. This is run from a path that contains multiple directories each containing a docker-compose.yml file.

DOMAINS=`cat */docker-compose.yml | sed 's/,/\n/g; s/^.*LETSENCRYPT_HOST=\(.*\)/\1/p;d'`

for domain in ${DOMAINS}; do
  # https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion/issues/8#issuecomment-381893731
  openssl req -x509 \
    -newkey rsa:4096 \
    -sha256 \
    -nodes \
    -days 365 \
    -subj "/CN=${domain}"  \
    -keyout "./certs/${domain}.key" \
    -out "./certs/${domain}.crt"
 done

Of course this could break easily, and doesn't allow putting .env variables in LETSENCRYPT_HOST (I'm using a sed-workaround for that). Solving this fully would either need a custom docker-gen setup (replicating some logic from the current project), or needs direct support (which doesn't seem in-scope for the project right now).