Certbot: Trying to get Certbot to install new cert with HSTS

[KeiroD]

Hi,

Some background:
So, I'm using InterWorx and I'm encountering this issue where ./certbot-auto --hsts doesn't work nor does ./certbot-auto certonly --standalone after temporarily stopping httpd on CentOS 7 x64. Also note that it's not proxied. This is on a Xen VPS. I've also confirmed that telnet to port 443 works for this server, so there should be no connection issues...

IW dev/support noted that HSTS might be causing Certbot-auto to fail on helmsgate.eidolonhost.com because eidolonhost.com is preloaded via the HSTS preload list (which I've now since requested removal so I can get this issue fixed...) but I was hoping there might be some answer where I can force through a LE cert to be installed.

Relevant logs:

[root@helmsgate letsencrypt]# cat letsencrypt.log
2016-07-08 00:32:50,790:DEBUG:certbot.main:Root logging level set at 30
2016-07-08 00:32:50,791:INFO:certbot.main:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2016-07-08 00:32:50,791:DEBUG:certbot.main:certbot version: 0.8.1
2016-07-08 00:32:50,791:DEBUG:certbot.main:Arguments: ['--hsts']
2016-07-08 00:32:50,791:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#webroot,PluginEntryPoint#null,PluginEntryPoint#manual,PluginEntryPoint#standalone)
2016-07-08 00:32:50,801:DEBUG:certbot.plugins.selection:Requested authenticator None and installer None
2016-07-08 00:32:54,292:DEBUG:certbot.plugins.selection:Multiple candidate plugins: * apache
Description: Apache Web Server - Alpha
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.configurator:ApacheConfigurator
Initialized: <certbot_apache.configurator.ApacheConfigurator object at 0x1fe7750>
Prep: True

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0x28da690>
Prep: True

* standalone
Description: Automatically use a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator
Initialized: <certbot.plugins.standalone.Authenticator object at 0x28da390>
Prep: True
2016-07-08 00:33:15,830:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.standalone.Authenticator object at 0x28da390> and installer None
2016-07-08 00:33:22,034:DEBUG:root:Sending GET request to https://acme-v01.api.letsencrypt.org/directory. args: (), kwargs: {}
2016-07-08 00:33:22,038:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-07-08 00:33:33,034:DEBUG:certbot.main:Exiting abnormally:
Traceback (most recent call last):
  File "/root/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
    sys.exit(main())
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 744, in main
    return config.func(config, plugins)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 550, in obtain_cert
    le_client = _init_le_client(config, auth, installer)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 360, in _init_le_client
    acc, acme = _determine_account(config)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/main.py", line 345, in _determine_account
    config, account_storage, tos_cb=_tos_cb)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/client.py", line 118, in register
    acme = acme_from_config_key(config, key)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/certbot/client.py", line 42, in acme_from_config_key
    return acme_client.Client(config.server, key=key, net=net)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 63, in __init__
    self.net.get(directory).json())
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 631, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/acme/client.py", line 613, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/root/.local/share/letsencrypt/lib/python2.7/site-packages/requests/adapters.py", line 437, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPSConnectionPool(host='acme-v01.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x3303e10>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Edit: Note that helmsgate.eidolonhost.com is now using a valid SSL certificate from Comodo. It'll at least allow me to get the SSL certificate replaced with Let's Encrypt in the future.

Edit2: It appears to be still persisting for a customer domain, in this case, dragonfox.net with the exact same error. The customer is using CloudFlare in this case, though so I don't know if that's relevant to the client being unable to install new SSL certs?

[bmw]

So the problem here isn't that the Let's Encrypt CA can't connect to your server, it's that Certbot can't connect to Let's Encrypt. The client is failing is where we first try to establish a connection with the CA.

What happens if you run curl https://acme-v01.api.letsencrypt.org/directory on the system where you're having this problem?

[KeiroD]

@bmw,

Thanks for pointing me the way to the problem.

Turns out that the server didn't know who acme-v01.api.letsencrypt.org was.

[root@helmsgate ~]# curl https://acme-v01.api.letsencrypt.org/directory
curl: (6) Could not resolve host: acme-v01.api.letsencrypt.org; Name or service not known
[root@helmsgate ~]# nano /etc/resolv.conf
[root@helmsgate ~]# curl https://acme-v01.api.letsencrypt.org/directory
{
  "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
  "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
  "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
  "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}[root@helmsgate ~]# ^C
[root@helmsgate ~]#

Edit: Sure enough, as soon as I add nameserver 8.8.8.8 to /etc/resolv.conf, it immediately picks up on Let's Encrypt's DNS settings and is now able to issue SSL certificates.

[bmw]

Glad I could help! Has this issue been resolved then?

[KeiroD]

Yup! I am now able to get the IW LE plugin to issue certs for customer domains. I've reproduced this behavior for several customer domains and in all cases, it's fixed.

I'll go ahead and close this issue. Thanks again for pointing the way. :)