DietPi-LetsEncrypt| certbot failed to renew https cert

[minnux]

Creating a bug report/issue
Required Information

DietPi version | 6.22
Distro version | stretch
Kernel version | Linux DietPi 4.4.154-1124-rockchip-ayufan-ged3ce4d15ec1 #1 SMP Mon Oct 22 20:59:41 UTC 2018 aarch64 GNU/Linux
SBC device | RockPro64 (aarch64)
Power supply used | (EG: 5V 1A RAVpower)
SDcard used | none

Additional Information (if applicable)

Software title | Nextcloud 15.05
updated
Can this issue be replicated on a fresh installation of DietPi? | not sure

l

Steps to reproduce

run dietpi-letsencrypt
Expected behaviour

should have renewed automatically
Actual behaviour

errors out when trying to renew manually
Extra details

I haven't had any problems with my https cert renewing automatically until now. I have recently upgraded my internet service and received a new modem and router, so maybe that's the issue? Please advise as to what logs you would like me to post. I noticed the problem when trying to log in to my nextcloud account and firefox stopped the connection.

[MichaIng]

@minnux
Many thanks for your report.

Please paste the output of:

systemctl status certbot
certbot renew

[minnux]

systemctl status certbot

● certbot.service - Certbot
   Loaded: loaded (/lib/systemd/system/certbot.service; static; vendor preset: enabled)
  Drop-In: /etc/systemd/system/certbot.service.d
           └─dietpi-lighttpd.conf
   Active: failed (Result: exit-code) since Mon 2019-04-01 12:03:30 CDT; 1h 12min ago
     Docs: file:///usr/share/doc/python-certbot-doc/html/index.html
           https://letsencrypt.readthedocs.io/en/latest/
  Process: 11564 ExecStart=/usr/bin/certbot -q renew (code=exited, status=1/FAILURE)
 Main PID: 11564 (code=exited, status=1/FAILURE)

Apr 01 12:03:25 DietPi systemd[1]: Starting Certbot...
Apr 01 12:03:30 DietPi certbot[11564]: Attempting to renew cert (marla.asuscomm.com) from /etc/letsencrypt/renewal/marla.asuscomm.com.
conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
Apr 01 12:03:30 DietPi certbot[11564]: All renewal attempts failed. The following certs could not be renewed:
Apr 01 12:03:30 DietPi certbot[11564]:   /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem (failure)
Apr 01 12:03:30 DietPi certbot[11564]: 1 renew failure(s), 0 parse failure(s)
Apr 01 12:03:30 DietPi systemd[1]: certbot.service: Main process exited, code=exited, status=1/FAILURE
Apr 01 12:03:30 DietPi systemd[1]: Failed to start Certbot.
Apr 01 12:03:30 DietPi systemd[1]: certbot.service: Unit entered failed state.
Apr 01 12:03:30 DietPi systemd[1]: certbot.service: Failed with result 'exit-code'.

certbot renew

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/marla.asuscomm.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for marla.asuscomm.com
Cleaning up challenges
Attempting to renew cert (marla.asuscomm.com) from /etc/letsencrypt/renewal/marla.asuscomm.com.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

[minnux]

One more thing. I explained before that I have a new router, my DDNS service is through ASUS as they provide a free address. I've always used this DDNS service but today looking at it I noticed that they now employ Let's Encrypt for an https cert for your personal address. See screen shot attached. Could this be interfering in some way?

[MichaIng]

@minnux
Confusing. When accessing your domain, do you reach the router then or the dedicated DietPi server behind the router (NAT)?

The router seems to have successfully issued a certificate for your domain. But this is only possible if the router does not forward port 80/443 to the DietPi server. So your server should be not reachable (and of course can then not renew a certificate), only the router instead (?).

Actually this is what it states: "Webui SSL Certificate". So this certificate is indeed only for accessing the router itself via SSL. Most unnecessary since the router should anyway never be reachable directly from www. Do you have a chance to disable global router Webui access (so it is only accessible from the local LAN) and then you can disable the HTTPS certificate there as well.

But another thing:
Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
I mean the router issue pretty clearly breaks renewal (and DietPi access from outside of LAN). But this part of the error message sounds like there might be another issue. Which authentication method do your use? The certbot drop-in config looks like you're using Lighttpd webserver. Then you need to use webroot authentication. Please try:
certbot renew --webroot -w /var/www

[minnux]

Ok I'm almost positive it's the router issuing a cert for the same domain as my nextcloud install that's preventing me from renewing my nextcloud install cert. I do have the webui turned off on the router but I had it issue a cert for the router basically for no reason.

The server is behind the router and is accessed through https. The router is only accessible within my network or vpn. I turned off lets encrypt certification on my router but more than likely I will have to wait for that current router cert to expire before I can renew the one on my server. Here is the output I received after certbot renew --webroot -w /var/www

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/marla.asuscomm.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for marla.asuscomm.com
Using the webroot path /var/www for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (marla.asuscomm.com) from /etc/letsencrypt/renewal/marla.asuscomm.com.conf produced an unexpected error: Failed authorization procedure. marla.asuscomm.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://marla.asuscomm.com/.well-known/acme-challenge/iw8OVcPxR_gCcEWgsyeOO4wukLG3hDoWRVu42JWM1eQ: Timeout during connect (likely firewall problem). Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: marla.asuscomm.com
   Type:   connection
   Detail: Fetching
   http://marla.asuscomm.com/.well-known/acme-challenge/iw8OVcPxR_gCcEWgsyeOO4wukLG3hDoWRVu42JWM1eQ:
   Timeout during connect (likely firewall problem)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

[MichaIng]

@minnux
I am pretty sure that port forwarding for port 80 and 443 got removed or is missing since your web UI was accessible via browser. Please check that. It also fit's to the error message:

The server could not connect to the client to verify the domain

[minnux]

That's so weird because I've never had to forward port 80 within my router before to access nextcloud remotely but I went ahead and did so per your suggestion and viola, I can now connect and it works. 443 was already forwarded so I didn't have to do anything there. Thanks for helping me get this sorted.

On a side note I noticed that the next iteration of Nextcloud (16) will be php 7.1 only. Will dietpi be updating its php from 7.0 to 7.1?

[MichaIng]

@minnux

and viola

That is great 🙂.

I've never had to forward port 80

Hmm, perhaps some UPnP functionality allowed the device to enable port forwarding on the router automatically. This is possible, if enabled on router and some software on the device supports it. But generally I would consider it as security risk. No client should be able to open any ports on the router without your knowledge IMO 😉.

On a side note I noticed that the next iteration of Nextcloud (16) will be php 7.1 only. Will dietpi be updating its php from 7.0 to 7.1?

Ui important hint! Raises priority of https://github.com/MichaIng/DietPi/issues/2367.
Hum no minor change. This would mean we need to upgrade all existing installs to PHP7.3 or PHP7.2. Not sure which is best. PHP7.3 will last longer but might not yet be supported by all software we offer. Would require a large testing session...
We cannot leave existing installs on PHP7.0 since then some software installs would work, some not anymore and our usual tests would either not include PHP7.0 or need to be doubled (which we cannot effort).

---

Quick question:

• Did you renew certificate now with certbot renew --webroot -w /var/www, respectively does /etc/letsencrypt/renewal/marla.asuscomm.com.conf show webroot as authentication module?
• Just because indeed with dietpi-letsencrypt we chose standalone when Lighttpd is installed but that is actually wrong because needs the webserver to be down. If the webserver is up, webroot method should definitely work and allow auto renewal without stopping the webserver.

[minnux]

I actually renewed via the let's encrypt tool within dietpi-launcher. the conf shows

# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/marla.asuscomm.com
cert = /etc/letsencrypt/live/marla.asuscomm.com/cert.pem
privkey = /etc/letsencrypt/live/marla.asuscomm.com/privkey.pem
chain = /etc/letsencrypt/live/marla.asuscomm.com/chain.pem
fullchain = /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem

[MichaIng]

@minnux
Is there no part below? E.g. in my case:

# Options used in the renewal process
[renewalparams]
authenticator = apache
installer = apache
rsa_key_size = 4096
account = XXXX
server = https://acme-v02.api.letsencrypt.org/directory

[minnux]

oh yes sorry:

# Options used in the renewal process
[renewalparams]
authenticator = standalone
rsa_key_size = 4096
account = xxxx
server = https://acme-v02.api.letsencrypt.org/directory

[MichaIng]

@minnux
Okay still standalone authenticator. Might I ask you to try to refresh again with webroot authentication? I personally use Apache so can't test this with Lighttpd on my home server. Start Lighttpd webserver if it is not yet running:

systemctl start lighttpd

Then:

certbot renew --force-renewal --webroot -w /var/www
cat /etc/letsencrypt/archive/marla.asuscomm.com/privkey.pem /etc/letsencrypt/archive/marla.asuscomm.com/cert.pem > /etc/letsencrypt/archive/marla.asuscomm.com/combined.pem

• The second command is some Lighttpd special since it requires cert and key in one file. DietPi-LetsEncrypt adds this task to the CertBot auto renewal service as well.

This would also assure that the auto renewal task of CertBot succeeds in renewing itself ~30 days before the cert expires next time, even if Lighttpd is running, blocking port 80 for Certbot standalone authenticator.

[minnux]

This is what I get after the 2nd command:

cat: /etc/letsencrypt/archive/marla.asuscomm.com/privkey.pem: No such file or directory
cat: /etc/letsencrypt/archive/marla.asuscomm.com/cert.pem: No such file or directory

The contents of /etc/letsencrypt/archive/marla.asuscomm.com are:

cert1.pem  cert3.pem   chain2.pem  combined.pem    fullchain2.pem  privkey1.pem  privkey3.pem
cert2.pem  chain1.pem  chain3.pem  fullchain1.pem  fullchain3.pem  privkey2.pem

[MichaIng]

@minnux
Whoopsie, should be:

cat /etc/letsencrypt/live/marla.asuscomm.com/privkey.pem /etc/letsencrypt/live/marla.asuscomm.com/cert.pem > /etc/letsencrypt/live/marla.asuscomm.com/combined.pem

And the certbot command succeeded? cat /etc/letsencrypt/renewal/marla.asuscomm.com.conf now shows authenticator = webroot?

[minnux]

Ok that worked. Here's the conf:

# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/marla.asuscomm.com
cert = /etc/letsencrypt/live/marla.asuscomm.com/cert.pem
privkey = /etc/letsencrypt/live/marla.asuscomm.com/privkey.pem
chain = /etc/letsencrypt/live/marla.asuscomm.com/chain.pem
fullchain = /etc/letsencrypt/live/marla.asuscomm.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
rsa_key_size = 4096
account = xxxx
server = https://acme-v02.api.letsencrypt.org/directory
webroot_path = /var/www,
[[webroot_map]]
marla.asuscomm.com = /var/www

So I should not use the dietpi-launcher let's encrypt tool then since I am using lighttpd?

[MichaIng]

@minnux
Many thanks for testing. So I can update/enhance our script the use this method.

EDIT: PR up: https://github.com/MichaIng/DietPi/pull/2688

So I should not use the dietpi-launcher let's encrypt tool then since I am using lighttpd?

You can still safely use it. When a certificate already exists, it only runs certbot renew, so does not change any authentication method. This is only relevant when running it the first time.
But now you definitely never need to run it anymore manually since this is already done by the CertBot renewal timer.

I guess in most cases even the standalone method was working. The timer tries to renew the certificate two times a day when it's about the expire in less then 30 days. I guess it's high likely that one of the attempts succeeds due to some webserver downtime 😄.

[minnux]

Thanks for your help and glad I could contribute in some small way.

[MichaIng]

I mark this as closed. Feel free to reopen if required.