nginx-proxy-manager 🚀 - Internal error on SSL certificates

I founded this is docker inspect looks like something is blocking curl wich should not be the case

[
    {
        "Id": "5396c4e1207e3486ccb714d7cf9f2027f8be83c35fcbc5c0393d7cb3120a4609",
        "Created": "2019-07-26T12:05:10.838857676Z",
        "Path": "/init",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 15761,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-07-29T07:06:18.91961627Z",
            "FinishedAt": "2019-07-29T07:05:44.533099275Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 23,
                "Log": [
                    {
                        "Start": "2019-07-29T07:11:08.475218801Z",
                        "End": "2019-07-29T07:11:08.698302026Z",
                        "ExitCode": 1,
                        "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                    },
                    {
                        "Start": "2019-07-29T07:11:23.708785289Z",
                        "End": "2019-07-29T07:11:23.931405635Z",
                        "ExitCode": 1,
                        "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                    },
                    {
                        "Start": "2019-07-29T07:11:38.941957592Z",
                        "End": "2019-07-29T07:11:39.161505674Z",
                        "ExitCode": 1,
                        "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                    },
                    {
                        "Start": "2019-07-29T07:11:54.171975308Z",
                        "End": "2019-07-29T07:11:54.424361521Z",
                        "ExitCode": 1,
                        "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                    },

IIPoliII on 29 Jul 2019

When trying to ping my own ip on port 9876 (docker proxy i get this)

( founded there https://github.com/EOSIO/eosio-project-demux-example/issues/25)

IIPoliII on 29 Jul 2019

I finnaly went back to another snapshot i think the issue was that i auto run the container but if i am right it already does it ?
https://askubuntu.com/questions/620930/how-do-i-autostart-docker-container-at-system-reboot
I let this issue open in case someone has the same issue and Jc is willing to give a look at it c:

IIPoliII on 29 Jul 2019

Okay i said nothing it did it again and i founded what is the issue. So basiclly create a new proxy host "Request a new SSL certificate". Then confirm, once you have done it remodify the host and add some more domaine, subdomain and go back in ssl certificates and "Request a new SSL certificate" save it and there it is the internal error.

IIPoliII on 29 Jul 2019

After that it's impossible to use let's encrypt ever again in an ways

IIPoliII on 29 Jul 2019

I ran into this issue just today. I proceeded under the assumption that it was a bug.

While reviewing the docker logs I noticed it was looking for "npm-4" and "npm-5", but it couldn't find them. Eventually I came around to deleting the error SSL and remaking the proxy host, requesting the SSL again.

When you view docker logs what do they contain?

s4b3rt0oth on 7 Aug 2019

I've got the same issue.

There is definitely a bug with Let's Encrypt certs.

Currently my Enginx Proxy Manager works only on port 81 (so I've got access to panel). Whatever I try to change - I've got internal error (but after web page refresh changes looks like saved). Unfortunately all my web traffic is dead now regarding to this issue :(

Wadera on 22 Aug 2019

I've managed to make it running by simple copying any other cert on place where script looking for and everything start's working:

Wadera on 22 Aug 2019

I've managed to make it running by simple copying any other cert on place where script looking for and everything start's working:

Can you explain me what you did exactly and it's working now ? (Also i am a noob in docker so if you could provide the command to access the docker shell with it would be really nice from you c: !)

IIPoliII on 23 Aug 2019

I've used Portainer (Docker GUI), but you can do it running bellowed command on your node server:

To list all docker containers:

$ docker ps
CONTAINER ID  IMAGE    COMMAND  CREATED      STATUS      PORTS  NAMES
3xamp13id  my_image          X hours ago  Up X hours         my_container

To enter into container:

$ docker exec -it 3xamp13id bash

Then inside just go into /etc/letsencrypt/live directories and copy any other certificate on place of missing one.

You can also do it on volume level as letsencrypt directories should be keep outside dicker container.

On default config:

    volumes:
      # Make sure this config.json file exists as per instructions above:
      - ./config.json:/app/config/production.json
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

Wadera on 23 Aug 2019

I see intressting thanks ! i hope @jc21 see this so maybe he can foudn a fix for it

IIPoliII on 23 Aug 2019

👍3

Yup, just got the same issue.

Has anyone found a fix yet that doesn't involve messing with the container itself? I'm not at all familiar with Docker, I'm happy that I got it running somehow haha :)

Let's ping @jc21 again to see if it helps ;)

EpicLPer on 15 Oct 2019

Same here. Running on docker as well.

bpjobin on 23 Oct 2019

same, but I have problems with NAT Loopback (because of my router). Could this be an issue?

Shurov on 11 Nov 2019

I've didn't check my logs deeply when it occur, but it can be a case (I'm using it on home configuration and behind of home router as well).

Wadera on 11 Nov 2019

Same issues as above. Don't know Docker - friend setup for me. Worked forever but had to re-image OS drive (All Virtual stuff through HyperV and none of this was even on OS drive). After that nothing works. I try to delete all hosts and SSL certs and just get "Internal Error" for everything. Hoping somebody posts some things to try / command to fix

theDwiz01 on 16 Nov 2019

I think I found the issue. When the URL you gave it isn't reachable it'll act as if everything worked instead of telling you what went wrong. I accidentally gave it an URL that hadn't been configured properly yet.

EpicLPer on 23 Nov 2019

👍3

I started having this same issue yesterday after updating to 2.1.0. Rolling back to 2.0.14 seems to have fixed it.

lachlan334 on 21 Feb 2020

@lachlan334 it would be nice to know why it failed, the docker logs would have the info. FWIW I'm using 2.1.0 in production and not having these problems.

jc21 on 21 Feb 2020

Guys, I also have unhealthy status of npm-container, but it's works... what will be the recommendations?

SAOPP on 21 Feb 2020

@lachlan334 it would be nice to know why it failed, the docker logs would have the info. FWIW I'm using 2.1.0 in production and not having these problems.

The output I'm getting from the health check is % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (7) Failed to connect to localhost port 9876: Connection refused

Logs are as follows:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.,
[s6-init] ensuring user provided files have correct perms...exited 0.,
[fix-attrs.d] applying ownership & permissions fixes...,
[fix-attrs.d] done.,
[cont-init.d] executing container initialization scripts...,
[cont-init.d] done.,
[services.d] starting services,
[services.d] done.,
[2/21/2020] [2:53:12 AM] [Migrate  ] › ℹ  info      Current database version: 20190227065017,
[2/21/2020] [2:53:12 AM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...,
[2/21/2020] [2:53:12 AM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json,
[2/21/2020] [2:53:15 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4,
[2/21/2020] [2:53:15 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6,
[2/21/2020] [2:53:15 AM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized,
[2/21/2020] [2:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [2:53:15 AM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized,
[2/21/2020] [2:53:15 AM] [Global   ] › ℹ  info      Backend PID 206 listening on port 3000 ...,
[2/21/2020] [2:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [2:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
`QueryBuilder#allowEager` method is deprecated. You should use `allowGraph` instead. `allowEager` method will be removed in 3.0,
`QueryBuilder#eager` method is deprecated. You should use the `withGraphFetched` method instead. `eager` method will be removed in 3.0,
QueryBuilder#omit is deprecated. This method will be removed in version 3.0,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [3:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [3:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [3:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
Connection Error: Error: Packets out of order. Got: 1 Expected: 16,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [4:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [4:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [4:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [5:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [5:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [5:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [6:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [6:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [6:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [7:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [7:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [7:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [8:53:15 AM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...,
[2/21/2020] [8:53:15 AM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json,
[2/21/2020] [8:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [8:53:15 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4,
[2/21/2020] [8:53:15 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6,
[2/21/2020] [8:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [8:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [8:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [9:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [9:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [9:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,
[2/21/2020] [10:53:15 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...,
[2/21/2020] [10:53:16 AM] [Nginx    ] › ℹ  info      Reloading Nginx,
[2/21/2020] [10:53:16 AM] [SSL      ] › ℹ  info      Renew Complete,
Connection Error: Error: read ECONNRESET,

lachlan334 on 21 Feb 2020

@lachlan334 hey buddy I have the same.. but npm is working fine.

Last output | % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (7) Failed to connect to localhost port 9876: Connection refused
-- | --

SAOPP on 21 Feb 2020

If it means anything, I've been getting this error with version 2.0.14 running the docker image jlesage/nginx-proxy-manager

GregHilston on 21 Feb 2020

👎1 👍1

@GregHilston Yep that means a lot. I don't maintain jlesage's images. We're up to 2.1.1 now and port 9876 isn't used internally anymore.

@lachlan334 Looks like you were using that 2.1.1 image, but it doesn't make sense why the Healthcheck is still using a setting from a previous docker image. The correct healthcheck will eventually run a curl command: curl --silent http://127.0.0.1:81/api/ from within the NPM container.

Perhaps try pulling down the stack entirely and putting it back up, assuming your have directory mounted volumes: docker-compose down && docker-compose up -d

jc21 on 24 Feb 2020

I'm running into a similar problem. Requesting a new SSL certificate generates an Internal Error.
Inspecting the docker container shows it's unhealthy and it's spitting some errors:
`
[

{

    "Id": "9d2d526acb3512e60ca4db020b9ef86b93e859735267d224223ca40f5054b3d4",
    "Created": "2020-02-21T02:30:55.239750232Z",
    "Path": "/init",
    "Args": [],
    "State": {
        "Status": "running",
        "Running": true,
        "Paused": false,
        "Restarting": false,
        "OOMKilled": false,
        "Dead": false,
        "Pid": 3473,
        "ExitCode": 0,
        "Error": "",
        "StartedAt": "2020-02-21T02:30:58.116917696Z",
        "FinishedAt": "0001-01-01T00:00:00Z",
        "Health": {
            "Status": "unhealthy",
            "FailingStreak": 20095,
            "Log": [
                {
                    "Start": "2020-02-24T16:56:39.096936098+01:00",
                    "End": "2020-02-24T16:56:39.294612731+01:00",
                    "ExitCode": 1,
                    "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                },
                {
                    "Start": "2020-02-24T16:56:54.409165629+01:00",
                    "End": "2020-02-24T16:56:54.726601709+01:00",
                    "ExitCode": 1,
                    "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                },
                {
                    "Start": "2020-02-24T16:57:09.810871898+01:00",
                    "End": "2020-02-24T16:57:10.019143483+01:00",
                    "ExitCode": 1,
                    "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                },
                {
                    "Start": "2020-02-24T16:57:25.125134859+01:00",
                    "End": "2020-02-24T16:57:25.335763382+01:00",
                    "ExitCode": 1,
                    "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                },
                {
                    "Start": "2020-02-24T16:57:40.435902211+01:00",
                    "End": "2020-02-24T16:57:40.643842955+01:00",
                    "ExitCode": 1,
                    "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 9876: Connection refused\n"
                }
            ]
        }

`
Here I saw the connection refused on 9876, not knowing what's causing it.
I'm using jc21's image, running version 2.1.1.

I don't mean to hijack, but perhaps this is the same problem as OP's running into.

EDIT: Code formatting

0ShinyNinetales0 on 24 Feb 2020

Here I saw the connection refused on 9876, not knowing what's causing it.
I'm using jc21's image, running version 2.1.1.

Same here ! Using jc21's image, version 2.1.1, on docker bridge.

EDIT: deleting the container and creating a new one from scratch did fix it though.

WhistleMaster on 26 Feb 2020

Not sure if it's a docker bug or expected behaviour but it feels like upgrading a docker image within an already-created container, doesn't update the healthcheck. There were quite a few changes between 2.0.* and 2.1.* of NPM so I recommend bringing down the stack entirely and recreating it.

Using existing data and letsencrypt data mounts will be fine though.

jc21 on 26 Feb 2020

That said, the healthcheck shouldn't affect the ability for a certificate to be renewed.

jc21 on 26 Feb 2020

Thank you for the reply jc21.
I don't know about the other users, but in my case this is a clean container / image of version 2.1.1.
I have recreated it once with re-using the data and let's encrypt folder. But the obtaining of SSL certificates never worked.

I eventually gave up on it and used my synology nas to retreive certificates and manually importing them into Nginx, which worked fine.

But it would be nice to find out why the internal error occurs and why the log is spamming the connection refusal on port 9876.

Is there anything I can submit that could help you give insight on the situation?

Kind regards.

0ShinyNinetales0 on 27 Feb 2020

I'm getting the same error and unhealthy status in Docker. I have had the error message pop in the past but at the moment I can still create new certs. I tried deleting the old certs and recreating them but always have the same unhealthy container :(

Outputs from my container below if it helps

Docker:

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (7) Failed to connect to localhost port 9876: Connection refused

Logs:

Traceback (most recent call last):,
File "/usr/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 64, in _reconstitute,
renewal_candidate = storage.RenewableCert(full_path, config),
File "/usr/lib/python3.8/site-packages/certbot/_internal/storage.py", line 465, in __init__,
self._check_symlinks(),
File "/usr/lib/python3.8/site-packages/certbot/_internal/storage.py", line 531, in _check_symlinks,
raise errors.CertStorageError(,
certbot.errors.CertStorageError: expected /etc/letsencrypt/live/npm-2/cert.pem to be a symlink,
Renewal configuration file /etc/letsencrypt/renewal/npm-2.conf is broken. Skipping.,
,
... [Repeats up to npm-7] ...
,
at ChildProcess.exithandler (child_process.js:295:12),
at ChildProcess.emit (events.js:210:5),
at maybeClose (internal/child_process.js:1028:16),
at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5),

shanecasey on 27 Feb 2020

There is a github-develop docker image that has removed 9876 from the Dockerfile EXPOSE. This was the last trace of this port in the entire codebase and although the 2.1.* images should not be using this port whatsoever, for some reasons that apply to specific people it might make a difference.

So, use the github-develop image tag, make sure the previous container is entirely removed before creating a new one. I'll mention the steps here for anyone who doesn't know:

docker rm -f name_of_nginx_proxy_manager_container
or
docker-compose down

If there is any trace of this port 9876 in your healthchecks after that, then I'll literally flip my desk and rage quit this project.

healthy

jc21 on 27 Feb 2020

❤4 😄1

Removing the container and setting up a new one using the github-develop image solved the internal error when requesting SSL certificates for me.
I can see port 9876 is no longer used by this image as well.

I just managed to get Let's Encrypt certificates for all my proxy hosts directly trough Nginx, what a blessing!

Currently I'm unable to SSH to my device (only allowed from local network and I'm at the office right now), so when I am back home I will check the docker container health status. Will get back to you!

0ShinyNinetales0 on 28 Feb 2020

Can confirm!

Used github-develop image and the container becomes healthy again with port 9876 no longer used anymore! I've tried it on 3 different hosts and containers are no longer unhealthy.

There is a github-develop docker image that has removed 9876 from the Dockerfile EXPOSE. This was the last trace of this port in the entire codebase and although the 2.1.* images should not be using this port whatsoever, for some reasons that apply to specific people it might make a difference.

So, use the github-develop image tag, make sure the previous container is entirely removed before creating a new one. I'll mention the steps here for anyone who doesn't know:

docker rm -f name_of_nginx_proxy_manager_container
or
docker-compose down

If there is any trace of this port 9876 in your healthchecks after that, then I'll literally flip my desk and rage quit this project.

AnonJervis on 28 Feb 2020

I just got home and checked trough SSH what the docker container's health status is. It's looking perfectly fine!

Health": {
"Status": "healthy",
"FailingStreak": 0,

No need to flip the desk and rage quit the project @jc21 ;)

0ShinyNinetales0 on 28 Feb 2020

I nuked my container and rebuilt with github-develop and mine is happy and healthy now too. Thanks!

shanecasey on 28 Feb 2020

Can also confirm that using the github-develop image fixed the issue.

lachlan334 on 29 Feb 2020

Yep, after removing the container and using the github-docker image, the certs where updated automatically.

brantje on 3 Mar 2020

Ok sweet I'll merge and re-release today. Phew :)

jc21 on 6 Mar 2020

👍1

So guys, what I need to do? Use the develop of the container? 'Coz I have unhealthy still. Thanks in advance.

SAOPP on 10 Mar 2020

@SAOPP unhealthy typically means the container took more than 5 seconds to return an OK status. If you're able to create new SSL certificates, you should be good to go. The unhealthy status isn't a big deal.

nldoty on 10 Mar 2020

Yes I understood it, just interesting, and yes I can create a new certs, but I see guys talked about not possible auto renewal certs or I misunderstood?

SAOPP on 10 Mar 2020

Please forgive my being uninformed on Docker, but when will this get pushed out to places like Dockerhub? I'm using an Unraid environment.
Is there a way I can upload and use my own certificate in the meantime, such as a Cloudflare certificate?

Foxglove-X on 15 Mar 2020

👍1

@Foxglove-X I believe it was already pushed about 4 days ago.
Check the tags under the Docker Hub page.
https://hub.docker.com/r/jc21/nginx-proxy-manager/tags

nldoty on 15 Mar 2020

Ah I see! Mine was pulled from jlesage/nginx-proxy-manager/ which is 5 months out of date.
That must be some other Docker image that's not as well maintained. I should probably use this one instead. Thank you.

Foxglove-X on 16 Mar 2020

jlesage will have to update his own docker images each time I release mine. I don't have an unraid setup myself so I don't know what changes are required to make it work

jc21 on 16 Mar 2020

Hello,
I just installed the latest version of npm.

docker-compose.yml

version: "3"
services:
  app:
    image: jc21/nginx-proxy-manager:latest
    restart: always
    ports:
      - 80:80
      - 81:81
      - 443:443
    volumes:
      - ./config.json:/app/config/production.json
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    depends_on:
      - db
    environment:
    # if you want pretty colors in your docker logs:
    - FORCE_COLOR=1
  db:
    image: mariadb:latest
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: "npm"
      MYSQL_DATABASE: "npm"
      MYSQL_USER: "npm"
      MYSQL_PASSWORD: "npm"
    volumes:
      - ./data/mysql:/var/lib/mysql

config.json

{
  "database": {
    "engine": "mysql",
    "host": "db",
    "name": "npm",
    "user": "npm",
    "password": "npm",
    "port": 3306
  }
}

I tried to add new SSL certificates and i go this issue when try to generate it :

I have

Thank for all

rto54 on 7 Apr 2020

@rto54 without the logs from the docker containers themselves, there's not a whole lot anyone can debug. Can you provide them?

nldoty on 7 Apr 2020

@nldoty, You can help me for this.
I am a beginner with the docker :-(
Thank

rto54 on 7 Apr 2020

[4/7/2020] [6:55:06 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [6:55:07 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot certonly --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-2" --agree-tos --email "[email protected]" --preferred-challenges "dns,http" --webroot --domains "unifi.ophiuchus.fr" , Saving debug log to /var/log/letsencrypt/letsencrypt.log, Plugins selected: Authenticator webroot, Installer None, Obtaining a new certificate, Performing the following challenges:, http-01 challenge for unifi.ophiuchus.fr, Using the webroot path /data/letsencrypt-acme-challenge for all unmatched domains., Waiting for verification..., Challenge failed for domain unifi.ophiuchus.fr, http-01 challenge for unifi.ophiuchus.fr, Cleaning up challenges, Some challenges have failed., , [4/7/2020] [7:03:09 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:03:24 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:03:32 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:03:42 PM] [SSL ] › ℹ info Renewing Let'sEncrypt certificates for Cert #2: unifi.ophiuchus.fr, [4/7/2020] [7:03:43 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot renew --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-2" --preferred-challenges "dns,http" --disable-hook-validation , Saving debug log to /var/log/letsencrypt/letsencrypt.log, No certificate found with name npm-2 (expected /etc/letsencrypt/renewal/npm-2.conf)., , [4/7/2020] [7:03:46 PM] [SSL ] › ℹ info Renewing Let'sEncrypt certificates for Cert #1: jeedom.ophiuchus.fr, [4/7/2020] [7:03:48 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot renew --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-1" --preferred-challenges "dns,http" --disable-hook-validation , Saving debug log to /var/log/letsencrypt/letsencrypt.log, No certificate found with name npm-1 (expected /etc/letsencrypt/renewal/npm-1.conf)., , [4/7/2020] [7:05:41 PM] [SSL ] › ℹ info Renewing Let'sEncrypt certificates for Cert #1: jeedom.ophiuchus.fr, [4/7/2020] [7:05:43 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot renew --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-1" --preferred-challenges "dns,http" --disable-hook-validation , Saving debug log to /var/log/letsencrypt/letsencrypt.log, No certificate found with name npm-1 (expected /etc/letsencrypt/renewal/npm-1.conf)., , Connection Error: Error: read ECONNRESET, [4/7/2020] [7:22:47 PM] [SSL ] › ℹ info Renewing Let'sEncrypt certificates for Cert #1: jeedom.ophiuchus.fr, [4/7/2020] [7:22:48 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot renew --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-1" --preferred-challenges "dns,http" --disable-hook-validation , Saving debug log to /var/log/letsencrypt/letsencrypt.log, No certificate found with name npm-1 (expected /etc/letsencrypt/renewal/npm-1.conf)., , [4/7/2020] [7:23:19 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:23:23 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:23:53 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:24:08 PM] [SSL ] › ℹ info Renewing Let'sEncrypt certificates for Cert #1: jeedom.ophiuchus.fr, [4/7/2020] [7:24:09 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot renew --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-1" --preferred-challenges "dns,http" --disable-hook-validation , Saving debug log to /var/log/letsencrypt/letsencrypt.log, No certificate found with name npm-1 (expected /etc/letsencrypt/renewal/npm-1.conf)., , [4/7/2020] [7:24:14 PM] [SSL ] › ℹ info Revoking Let'sEncrypt certificates for Cert #1: jeedom.ophiuchus.fr, [4/7/2020] [7:24:22 PM] [SSL ] › ℹ info Revoking Let'sEncrypt certificates for Cert #2: unifi.ophiuchus.fr, [4/7/2020] [7:25:17 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:25:51 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:25:51 PM] [SSL ] › ℹ info Requesting Let'sEncrypt certificates for Cert #3: unifi.ophiuchus.fr, [4/7/2020] [7:25:56 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:25:56 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot certonly --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-3" --agree-tos --email "[email protected]" --preferred-challenges "dns,http" --webroot --domains "unifi.ophiuchus.fr" , Saving debug log to /var/log/letsencrypt/letsencrypt.log, Plugins selected: Authenticator webroot, Installer None, Obtaining a new certificate, Performing the following challenges:, http-01 challenge for unifi.ophiuchus.fr, Using the webroot path /data/letsencrypt-acme-challenge for all unmatched domains., Waiting for verification..., Challenge failed for domain unifi.ophiuchus.fr, http-01 challenge for unifi.ophiuchus.fr, Cleaning up challenges, Some challenges have failed., , [4/7/2020] [7:26:25 PM] [Access ] › ℹ info Building Access file #1 for: ophiuchus, [4/7/2020] [7:26:25 PM] [Access ] › ℹ info Adding: rodolphe, [4/7/2020] [7:26:26 PM] [Access ] › ✔ success Built Access file #1 for: ophiuchus, [4/7/2020] [7:33:08 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:33:08 PM] [SSL ] › ℹ info Requesting Let'sEncrypt certificates for Cert #4: jeedom.ophiuchus.fr, [4/7/2020] [7:33:13 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:33:13 PM] [Express ] › ⚠ warning Command failed: /usr/bin/certbot certonly --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-4" --agree-tos --email "[email protected]" --preferred-challenges "dns,http" --webroot --domains "jeedom.ophiuchus.fr" , Saving debug log to /var/log/letsencrypt/letsencrypt.log, Plugins selected: Authenticator webroot, Installer None, Obtaining a new certificate, Performing the following challenges:, http-01 challenge for jeedom.ophiuchus.fr, Using the webroot path /data/letsencrypt-acme-challenge for all unmatched domains., Waiting for verification..., Challenge failed for domain jeedom.ophiuchus.fr, http-01 challenge for jeedom.ophiuchus.fr, Cleaning up challenges, Some challenges have failed., , [4/7/2020] [7:33:15 PM] [SSL ] › ℹ info Renewing SSL certs close to expiry..., [4/7/2020] [7:33:17 PM] [Nginx ] › ℹ info Reloading Nginx, [4/7/2020] [7:33:17 PM] [SSL ] › ℹ info Renew Complete, [4/7/2020] [7:33:17 PM] [SSL ] › ✖ error Certificate is not valid (Command failed: openssl x509 -in /etc/letsencrypt/live/npm-3/fullchain.pem -subject -noout, Can't open /etc/letsencrypt/live/npm-3/fullchain.pem for reading, No such file or directory, 140650706484552:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/live/npm-3/fullchain.pem','r'), 140650706484552:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:, unable to load certificate, ), [4/7/2020] [7:33:17 PM] [SSL ] › ✖ error Certificate is not valid (Command failed: openssl x509 -in /etc/letsencrypt/live/npm-4/fullchain.pem -subject -noout, Can't open /etc/letsencrypt/live/npm-4/fullchain.pem for reading, No such file or directory, 140360297569608:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('/etc/letsencrypt/live/npm-4/fullchain.pem','r'), 140360297569608:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:, unable to load certificate, ), [4/7/2020] [7:33:41 PM] [Nginx ] › ℹ info Reloading Nginx,

rto54 on 7 Apr 2020

Logs letsencrypt

[root@docker-70a1dec97cfe:/var/log/letsencrypt]# vi letsencrypt.log
2020-04-07 20:01:27,880:DEBUG:certbot._internal.main:certbot version: 1.0.0
2020-04-07 20:01:27,881:DEBUG:certbot._internal.main:Arguments: ['--non-interactive', '--config', '/etc/letsencrypt.ini', '--cert-name', 'npm-3', '--preferred-challenges', 'dns,http', '--disable-hook
2020-04-07 20:01:27,881:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-04-07 20:01:27,919:DEBUG:certbot._internal.log:Root logging level set at 20
2020-04-07 20:01:27,919:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-04-07 20:01:27,920:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==1.0.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3.8/site-packages/certbot/main.py", line 14, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 1350, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 1259, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 391, in handle_renewal_request
    conf_files = [storage.renewal_file_for_certname(config, config.certname)]
  File "/usr/lib/python3.8/site-packages/certbot/_internal/storage.py", line 52, in renewal_file_for_certname
    raise errors.CertStorageError("No certificate found with name {0} (expected "
certbot.errors.CertStorageError: No certificate found with name npm-3 (expected /etc/letsencrypt/renewal/npm-3.conf).

rto54 on 7 Apr 2020

I seem to have a very similar issue...
I had the issue prior to pulling the latest image too
docker logs nginx_proxy

[4/16/2020] [3:17:52 PM] [Migrate  ] › ℹ  info      Current database version: 20200410143839
[4/16/2020] [3:17:52 PM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[4/16/2020] [3:17:52 PM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[4/16/2020] [3:17:52 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4
[4/16/2020] [3:17:53 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6
[4/16/2020] [3:17:53 PM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[4/16/2020] [3:17:53 PM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...
[4/16/2020] [3:17:53 PM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[4/16/2020] [3:17:53 PM] [Global   ] › ℹ  info      Backend PID 203 listening on port 3000 ...
`QueryBuilder#allowEager` method is deprecated. You should use `allowGraph` instead. `allowEager` method will be removed in 3.0
`QueryBuilder#eager` method is deprecated. You should use the `withGraphFetched` method instead. `eager` method will be removed in 3.0
QueryBuilder#omit is deprecated. This method will be removed in version 3.0
[4/16/2020] [3:18:46 PM] [SSL      ] › ✖  error     Error: Command failed: /usr/bin/certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --preferred-challenges "dns,http" --disable-hook-validation
Challenge failed for domain cloud.domain.net
Challenge failed for domain www.domain.net
Attempting to renew cert (npm-3) from /etc/letsencrypt/renewal/npm-3.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/npm-3/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1028:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

container-start.sh

docker create \
  --name $name \
  -p 80:80 \
  -p 443:443 \
  -p 81:81 \
  -v /mnt/z.raid1/software.docker/A-dockerStartFiles/nginx-proxy/config.json:/app/config/production.json \
  -v /mnt/z.raid1/software.docker/nginx_proxy/data:/data \
  -v /mnt/z.raid1/software.docker/nginx_proxy/letsencrypt:/etc/letsencrypt \
  -e TZ=Europe/Helsinki \
  -e FORCE_COLOR=1 \
  --restart unless-stopped \
  --log-opt max-size=1m \
    jc21/nginx-proxy-manager:latest

docker image inspect jc21/nginx-proxy-manager:latest

[
    {
        "Id": "sha256:ed18ab76b851f62ab3672b5a35b19f43671a4380c4bb8d95b4209a08c6de36e5",
        "RepoTags": [
            "jc21/nginx-proxy-manager:latest"
        ],
        "RepoDigests": [
            "jc21/nginx-proxy-manager@sha256:940681f8439604c9a5a2b88d2036adbcdd23675908bf915806a3289fa8c6b226"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2020-04-15T06:02:31.803807951Z",

logs in container:
(/var/log/letsencrypt/letsencrypt.log)

2020-04-17 08:21:32,578:INFO:certbot._internal.renewal:Cert not yet due for renewal
2020-04-17 08:21:32,580:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2020-04-17 08:21:32,583:DEBUG:certbot._internal.cli:Var pref_challs=dns,http (set by user).
2020-04-17 08:21:32,583:DEBUG:certbot._internal.cli:Var authenticator=webroot (set by user).
2020-04-17 08:21:32,584:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2020-04-17 08:21:32,584:DEBUG:certbot._internal.cli:Var webroot_map={'webroot_path'} (set by user).
2020-04-17 08:21:32,584:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2020-04-17 08:21:32,609:INFO:certbot._internal.renewal:Cert not yet due for renewal
2020-04-17 08:21:32,612:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2020-04-17 08:21:32,612:ERROR:certbot._internal.renewal:All renewal attempts failed. The following certs could not be renewed:
2020-04-17 08:21:32,613:ERROR:certbot._internal.renewal:  /etc/letsencrypt/live/npm-3/fullchain.pem (failure)
2020-04-17 08:21:32,613:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==1.0.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3.8/site-packages/certbot/main.py", line 14, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 1350, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/main.py", line 1259, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3.8/site-packages/certbot/_internal/renewal.py", line 473, in handle_renewal_request
    raise errors.Error("{0} renew failure(s), {1} parse failure(s)".format(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

Let me know if there are any other information missing from above.

Nornode on 17 Apr 2020

Not sure if anyone else has run into this. I'm running the proxy container on VM inside corporate network and when I try and create the proxy in Nginx Proxy Manager, I too get the 'Internal Error' and in the log I can see that lets encrypt can't validate my domain. Under SSL Certificates there is an entry but its immediately expired. In the DNS Manager we use I have my domain mapped to the host IP of the machine where I'm running the proxy container in the Forward Lookup Zone. Am I correct in thinking letsencrypt just can't reach my domain to validate that 'I own it'?

2020-05-04 17:10:39,713:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/4362994539 HTTP/1.1" 200 625
2020-05-04 17:10:39,714:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Mon, 04 May 2020 17:11:21 GMT
Content-Type: application/json
Content-Length: 625
Connection: keep-alive
Boulder-Requester: 85028888
Cache-Control: public, max-age=0, no-cache
Link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
Replay-Nonce: 0002Z8epRT7DdbUiCHQ6gsOQuvNlarPnL7KRDGYsEg7qoCM
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"identifier": {
"type": "dns",
"value": ""
},
"status": "invalid",
"expires": "2020-05-11T17:11:19Z",
"challenges": [
{
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:dns",
"detail": "DNS problem: NXDOMAIN looking up A for - check that a DNS record exists for this domain",
"status": 400
},
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/4362994539/zHEy1w",
"token": "PCdm1kOKDDqyzF4LX6-gAt0omNYuV0iwRNBtQ0NLR5Q"
}
]
}

jmford775 on 4 May 2020

@jmford775 Yes, you are correct. Letsencrypt requires you to verify ownership of a domain before issuing a certificate. It does this by first looking for a valid domain response, then reaching out to the domain to verify the server is sitting there. For this to happen you will need to point the domain to the IP you plan on using and open port 80 on that IP and have it point to your proxy container.

From the sound of your post you plan on using this internally. Note that setting up internal DNS to point to your machine will not be sufficient. There must be a valid public IP address with matching DNS pointed to your container. Likewise, the certificate will be valid ONLY for that external address and will not work internally (unless you utilize the same naming scheme).

For example:
External Public IP: 1.1.1.x
Internal Public IP: 10.0.10.x
Public DNS: public.com
Internal DNS: internal.com

Your container sits on 10.0.10.y and it requests a certificate for container.internal.com. Letsencrypt does not know where your container is, because it is using an internal DNS schema. If you request container.public.com then letsencrypt will be able to see that you pointed container.public.com to your external IP (1.1.1.x) and will reach out to that IP (Port 80 [forwarded on firewall]) to verify that the container is actually sitting there.

s4b3rt0oth on 4 May 2020

@s4b3rt0oth I have the same problem as @jmford775 but mine is to connect from the outside. My port 80 and 443 are open and pointing to NPM server but it still shows internal error. The exact same thing happened to me where once trying to add SSL cert, it shows "Internal Error" and will show the cert expired same date and time when I tried adding.

AnonJervis on 5 May 2020

👍1

Same here. Running the github-develop build.

Sent from my iPhone

On May 5, 2020, at 2:38 PM, AnonJervis notifications@github.com wrote:

@s4b3rt0oth I have the same problem as @jmford775 but mine is to connect from the outside. My port 80 and 443 are open and pointing to NPM server but it still shows internal error. The exact same thing happened to me where once trying to add SSL cert, it shows "Internal Error" and will show the cert expired same date and time when I tried adding.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

bpjobin on 5 May 2020

@s4b3rt0oth Yeah I realized it was going to be a no-go being inside our corporate network and thanks for the reply. I decided since we are already behind super strict firewalls that I would just run a no-auth registry and joxit ui

jmford775 on 6 May 2020

Same thing here. Using the Nginx Proxy Manager Add On for Home Assistant

lucashend on 8 Sep 2020

👍2

Nginx-proxy-manager: Internal error on SSL certificates

Most helpful comment

All 57 comments

Related issues