Dietpi: Certbot + NoIP: no connection via dns-domain - (nextcloud works)

[butterbrot21]

Hey,
unfortunately i can't connect to the web-gui/software via NoIP domain.
I have tried with Lighttpd and Nginx. In both cases I could only connect to nextcloud (with dns-domain and ssl). For all others there is an error. The required ports are open.
I only have a free Account on NoIP, but i read here on GitHub it works without premium for some time.

Required Information:

• DietPi version | 6.11
• Distro version | stretch
• Kernel version | Linux DietPi 4.14.32+
• SBC device | Odroid HC2
• Power supply used | 5V 2A
• SDcard used | SanDisk ultra

Additional Information:

• Software title | Certbot, NoIp, Koel, Emby
• Fresh install
• Can this issue be replicated on a fresh installation of DietPi? -> i think yes
• dietpi-bugreport ID = 57adfb6c-3277-4068-8242-10c965406199

Steps to reproduce:

1. Fresh install:
2. Software: NoIp, Cerbot, Emby, Nextcloud, Lighttpd
3. Setup Router (Port Forwarding for 80,443,8000,8096...)
4. Setup NoIp -> sucess
5. Install Software ->sucess
6. Setup Certbot (dietpi-letsencrypt) hsts+redirect=on
7. check urls with ip:port -> works
8. check https://domain.net/nextcloud --> works

Expected behaviour:

Connect to the installed sofware via NoIP/domain
ex: https://domain.net:8000, https://domain.net:8096 ....

Actual behaviour:

Can´t conntect via NoIp-dns to installed sofware.
Only Nextcloud works with domain/dns (with ssl)

-Browser Error:
This website cannot provide a secure connection
ERR_SSL_PROTOCOL_ERROR

Extra details:

-On the fist run I tried Nginx and then I tried it with Lighttpd.

-On both i was able to conect to https://domain.net/nextcloud,
but to the other installed software it failld to conect.

-With Nginx there was a 404-Nginx error, with Lighttpd the dafault
(This website cannot provide a secure connection-ERR_SSL_PROTOCOL_ERROR)

-I tried it with Certbot with hsts+redirect=on and both on off.

I hope you can help me, I really love dietpi

[MichaIng]

@butterbrot21
Thank you for your report.

Actually this behaviour is somehow expected. CertBot/dietpi-letsencrypt by default applies the certificates and SSL configuration to the webservers only, Lighttpd in your case. Connecting to this webserver through the ports it listens to (80+443) therefore works + SSL redirection.

Koel and Emby bring and run their own embedded webservers, which can be seen as they listen to another port than default 80/443. These embedded webservers need to be configured separately, some software titles allow this via their web interface or manually, some bring their own SSL by default, some simply do not support it. This is totally up to the developers of these.

One exception is Minio, which brings it's own webserver as well, where dietpi-letsencrypt with quite some coding effort enables SSL an applies the letsencrypt certificates. But for all the others it is either impossible or simply the effort is too high 😉.

What should work instead, is accessing the web UIs via non-HTTPS http://domain.net:8000 / http://domain.net:8096, as domain => IP translation should be done by router?

[butterbrot21]

Ah ok I understand, that makes sense.
Thanks a lot for the quick detailed answer! SSL for nextcloud/pydio is completely enough.

For Koel etc. it would be nice to connect just with http.
I have a AVM FritzBox 7360 wth Port Forwarding (8000/TCP)
For know im not able to connect with http://domain.net:8000. Can this happen becouse
of the hsts+redirect on with cerbot?

One last question^^ IP translation is just to open the right port forwaord it on the router (ex. 8000 for koel)? Than just http://xxx:8000 instead of https?

Maybee i should go for a fresh install, than setup the router with dns/noip and Port Forwarding 80 +443 for nextcloud and 80000 for koel. I did a lot unisntall/reinstall

[butterbrot21]

After a few try, I see that http//example.net:8000 automatically convert to https, resulting in the 404 Not Found with an active letsEncrypt Certificat

Does certbot work in between or do I have to set more in the router like port forwarding e.g. 8000 to 8000 to the ip from my odroid?

I have one more question about nextcloud, I hope it's ok? :)
Is nextcloud protected by default with failtoban or similar or should I change the config /etc/fail2ban/jail.conf with:

[nextcloud]

enabled  = true
port     = 443           -->can/should i use 443+80 or 443, 80?             
filter   = nextcloud
logpath  = /var/log/auth.log
maxretry = 3

[MichaIng]

@butterbrot21
Hmm, CertBot itself does not interact with the connection at all. It is just the webservers that forces redirection. In your case /etc/lighttpd/conf-enabled/redirect.conf is responsible to make Lighttpd redirect all traffic to HTTPS. But this as said does only affect connections Lighttpd listens to, thus 80 and 443, not 8000.

In the router of course you need to forward incoming port 8000 requests to Odroid port 8000 and incoming 8096 to Odroid 8096 etc.

If then http://example.net:8000 is forwarded to https://example.net:8000, it must be caused by the embedded webserver.
🤔 Ah wait, you enabled HSTS. Hehe now I know where this is from:

• HSTS makes the browser remember that it should access this domain always via HTTPS, in this case for 1 year: https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-letsencrypt#L203
• So in your case, since the browser got HSTS header from Lighttpd, it directly attempts to access your domain via HTTPS, no matter which port.

I never thought about that, but this makes HSTS break other non-HTTPS webservers on the same domain!
In your case then I would try to make Koel and Emby use HTTPS as well, as I don't know if removing browser cache/cookies etc. includes HSTS info, actually I doubt, since this would make it useless in many cases where users auto purge their browser data on exit etc.

@Fourdee
Just to remember:

• Within dietpi-letsencrypt we should pop a warning that ALL web services, available on the same machine, that shall be available via domain, need to be available via HTTPS then, otherwise HSTS breaks access, and this for by default a long long time 😄.

[butterbrot21]

thanks for the info and help! 1 Year, thats a long time :)

I am currently reinstalling, completely fresh.
I'm going to leave redirect and HSTS off this time and in addition clear my browser cache/cookis.

A tool for self-signed certificates would be cool^^ Probably not worth it, because as you already said most of the software takes care of it or it has no real sense.

I'll report when everything's reset

[MichaIng]

@butterbrot21
Yeah, as said, might be that the browser you used to access to the server still has it's HSTS bits set, or possibly they are even applied to the OS, not sure how this is handled. In case try another browser and/or access by mobile phone or something to be really sure.
Checking Debian:

root@VM-Stretch:~# cat ~/.wget-hsts
# HSTS 1.0 Known Hosts database for GNU Wget.
# Edit at your own risk.
# <hostname>    <port>  <incl. subdomains>      <created>       <max-age>
raw.githubusercontent.com       0       0       1526831083      31536000
deb.debian.org  0       0       1526832956      15552000
install.pi-hole.net     0       1       1527895763      31536000
github.com      0       1       1526831156      31536000
codeload.github.com     0       0       1526831156      31536000

• wget as it's own HSTS list on user basis, this is how it looks like. Okay easy to reset here at least.
• I think then it should be also possible to reset this on other systems without larger effort, but a hint on selecting HSTS still makes sense 👍.

Self-signed certificate or LetsEncrypt certificate should not make any difference here. The result of both is just to have the certificate files. If/How web services use them or not, is up to their configuration.
I also thought about integrating into dietpi-letsencrypt the option to create self-signed certificates for e.g. testing, before calling CertBot. But really, why not directly go with the real ones. You can at any time throw those away or get new ones. Okay if I remember right there is a limit of how often one client/account/domain can retrieve new certs per week. But creating self-signed certificates finally is just a wasted step from my point of view, besides one experiences issues with CertBot 😉. But with self-signed the clients (browsers) on the other hand will show warnings and some even deny access, because the cert is not trusted.

[butterbrot21]

yap you're right^^
With chrome(used with HSTS) it does not work. Here it is still redirected to https :)

With edge/firefox it works.

Now I was really looking forward to the evening but nonono,
plex makes trouble this time xD

I can connect to plex an login works. When i tried to chose a directory an endless waiting loop pops up.

I use a Wd-Red 2TB, which is also formatted and mounted via drive-manager with ext4.
The UserData i changed to the drive.

By the way, the new Drive-Manager is genious!

[MichaIng]

@butterbrot21
I am running some test installations anyway currently. Will add plex media server for the next and see if I can replicate.

[butterbrot21]

Sorry!I can sleep in peace 👍.
After a reboot and browser change everything works. No problem with Plex!

[MichaIng]

Information/warning added to dietpi-letsencrypt: https://github.com/Fourdee/DietPi/pull/2002

I will mark this as closed, further discussion in case within the PR.