Plugins: security/acme-client switching from staging to prod leads to account error

Created on 30 Sep 2019 · 14Comments · Source: opnsense/plugins

I tried with different accounts, even a new one, but with the same result.

What i noticed after some extra digging is that the error states the invalid account URL as "https://acme-staging-v02.api.letsencrypt.org/acme/acct/9468460\" which is clearly pointing at the LE staging environment although i changed my setting to use the production environment.
Is the script trying to get a real certificate from the staging environment??? I can't imagine, otherwise a lot of people would have the same problem.

I also noticed the following lines in the log:

_saved_account_key_hash='+D9eLQaohKgjpS+HZ4kp6KX3tdg15N+i+uCw4TJo3pE='
_saved_account_key_hash is not changed, skip register account.

It is always the same key hash, but i cannot find where it is coming from. I would expect that it changes when i use a new account, but it doesn't.

Steps to reproduce:

Set up the acme plugin with an account, validation method and certificate and use the staging environment to get a test certificate which works fine.
Then change in the settings tab the LE environment to 'Production Environment' and save and apply the new setting.
Then go to the certificates tab and re-issue the same certificate. That is when the error occurs.

Kind regards,
Jack.

_Originally posted by @jekare in https://github.com/opnsense/plugins/issues/1473#issuecomment-526837247_

bug

Source

proofy

Most helpful comment

which is clearly pointing at the LE staging environment although i changed my setting to use the production environment

I think this can be solved. Currently we'll create a new directory for every LE account:
https://github.com/opnsense/plugins/blob/a878826c05ff470ab2a5001cf9f26b153a1f7bcf/security/acme-client/src/opnsense/scripts/OPNsense/AcmeClient/certhelper.php#L347
It should be pretty easy to modify this to not only use the internal Account ID but also include the LE environment in the path. This should allow to use the "same" account for both the LE staging and production environments.

However, this will provide a challenge for all existing configurations, because we have to migrate all accounts to the new (staging or production) path. I'll try to figure this out.

fraenki on 10 Oct 2019

❤2

All 14 comments

I can confirm this issue with the same steps to reproduce.
OPNsense 19.7.4_1-amd64
A crash report came with this issue. I forwarded it.

luismompohanden on 1 Oct 2019

Same

miodzicho on 2 Oct 2019

Same.

This can be avoided by creating dedicated accounts for STAGING and PROD and to make sure to change those in each and every certificate when changing the global PROD/STAGING setting.

jpawlowski on 2 Oct 2019

Not in my case, even changing account doesn't help :

"detail": "KeyID header contained an | invalid account URL: \"https://acme-staging-v02.api.letsencrypt.org/acme/acct/11242486\"", -- | --

miodzicho on 2 Oct 2019

You need to start from scratch by cleaning up the artifacts first, otherwise the account mixup will persist.

jpawlowski on 2 Oct 2019

Got the exact same error as cited in the bug report. However I used jpawlowski's suggestion.

Switching to a new LE account fixed the problem. But needed to run (re-) issue twice. It seems the first (re-) issue just registered the new LE account and stopped there.

The output below came from acme.sh with a _--debug 2_ flag. This is recent pull request (#1531 debug change is buried but it's there) not yet integrated into OPNsense. So, yes, I'm running a self-patched system. This level of detail may be different on your system (YMMV).

[Mon Oct  7 01:19:45 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:45 CEST 2019] Use length 4096
[Mon Oct  7 01:19:45 CEST 2019] Using RSA: 4096
[Mon Oct  7 01:19:46 CEST 2019] Create account key ok.
[Mon Oct  7 01:19:46 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] Using config home:/var/etc/acme-client/home
[Mon Oct  7 01:19:46 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] _init api for server: https://acme-v02.api.letsencrypt.org/directory
[Mon Oct  7 01:19:46 CEST 2019] GET
[Mon Oct  7 01:19:46 CEST 2019] url='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] timeout=
[Mon Oct  7 01:19:46 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.uqWfxFQl  -g '
[Mon Oct  7 01:19:46 CEST 2019] ret='0'
[Mon Oct  7 01:19:46 CEST 2019] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_AUTHZ
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:46 CEST 2019] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
[Mon Oct  7 01:19:46 CEST 2019] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Mon Oct  7 01:19:46 CEST 2019] ACME_VERSION='2'
[Mon Oct  7 01:19:46 CEST 2019] RSA key
[Mon Oct  7 01:19:47 CEST 2019] Registering account
[Mon Oct  7 01:19:47 CEST 2019] url='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:47 CEST 2019] payload='{"contact": ["mailto: [email protected]"], "termsOfServiceAgreed": true}'
[Mon Oct  7 01:19:47 CEST 2019] HEAD
[Mon Oct  7 01:19:47 CEST 2019] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Mon Oct  7 01:19:47 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.9YOKeRnv  -g '
[Mon Oct  7 01:19:47 CEST 2019] _ret='0'
[Mon Oct  7 01:19:47 CEST 2019] POST
[Mon Oct  7 01:19:47 CEST 2019] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:47 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.9YOKeRnv  -g '
[Mon Oct  7 01:19:48 CEST 2019] _ret='0'
[Mon Oct  7 01:19:48 CEST 2019] code='201'
[Mon Oct  7 01:19:48 CEST 2019] Registered
[Mon Oct  7 01:19:48 CEST 2019] _accUri='https://acme-v02.api.letsencrypt.org/acme/acct/68693193'
[Mon Oct  7 01:19:48 CEST 2019] Calc CA_KEY_HASH='ClklQy8f34s/PpiT5z69ca3B1nK5jviaS/H8cNmVYuA='
[Mon Oct  7 01:19:48 CEST 2019] ACCOUNT_THUMBPRINT='Re7J53urvbg2FKxtLgyoJGi-aRvqDMqZZElkMM-sHgo'

Second try using the same LE account as registered above. Did nothing more than hit “Forcefully (re-) issue the selected certificate?” button a second time. Cert issued. Success!

But perhaps there is a bug, that is, it should issue a cert the first time through?

billgertz on 7 Oct 2019

I did as well, but unfortunatelly getting this :

[Mon Oct 7 10:29:31 CEST 2019] response='{ "type": "urn:ietf:params:acme:error:malformed", "detail": "Unable to update challenge :: authorization must be pending", "status": 400

miodzicho on 7 Oct 2019

That’s just the regular limiter when you have tried too often and there are pending validations. Wait for them to time out, it should usually take 1-2 hours.

jpawlowski on 7 Oct 2019

So. I cannot get this working at all...Created all fresh.

Last Acme Status : validation failed

Log:
[Tue Oct 8 09:07:53 CEST 2019] original='{
"type": "http-01",
"status": "pending",

miodzicho on 8 Oct 2019

@miodzicho I think that's an unrelated problem. Please open a new issue and provide more details about your Let's Encrypt plugin and certificate configuration.

fraenki on 10 Oct 2019

which is clearly pointing at the LE staging environment although i changed my setting to use the production environment

However, this will provide a challenge for all existing configurations, because we have to migrate all accounts to the new (staging or production) path. I'll try to figure this out.

fraenki on 10 Oct 2019

❤2

@jpawlowski I've created a fix in https://github.com/fraenki/plugins/commit/5866358adcd73220f556a39c53a6cf7f3a034887, would you do a quick review?

fraenki on 11 Oct 2019

Patch 5866358 fixed the issue for me, I had restored my config.xml previously and was having trouble getting my existing certificate to renew even after rekeying etc. I was switching around to the staging environment etc. After deleting/recreating everything again I was still having issues. Applied patch, renewed, and it seems to be working now. I was getting the same invalid account URL message from the staging server URL.