Acme.sh: Bash 4.4.12: "warning: command substitution: ignored null byte in input"

[incognitoms]

With newer versions of bash (GNU bash, version 4.4.12(1) on Debian Stretch in my case) a cron run throws a warning:

acme.sh: line 1819: warning: command substitution: ignored null byte in input

The warning seems to be harmless as far as I know, but results in a unnecessary (somewhat nasty) cron job notification.

[Neilpang]

I could not reproduce it in my bash 4.4 machine.
please give me more clue.

[incognitoms]

The following example shows the shell command acme.sh was invoked with and the resulting console log. The actual used domain name was replaced with „mydomain.com“. acme.sh version is 2.7.5

acme.sh --issue -d mydomain.com --webroot /var/run/acme

[Sun Nov 19 20:50:04 CET 2017] Single domain='mydomain.com'
[Sun Nov 19 20:50:04 CET 2017] Getting domain auth token for each domain
[Sun Nov 19 20:50:04 CET 2017] Getting webroot for domain='mydomain.com'
[Sun Nov 19 20:50:04 CET 2017] Getting new-authz for domain='mydomain.com'
[Sun Nov 19 20:50:06 CET 2017] The new-authz request is ok.
[Sun Nov 19 20:50:06 CET 2017] Verifying:mydomain.com
[Sun Nov 19 20:50:09 CET 2017] Success
[Sun Nov 19 20:50:09 CET 2017] Verify finished, start to sign.
/root/.acme.sh/acme.sh: line 1819: warning: command substitution: ignored null byte in input
[Sun Nov 19 20:50:10 CET 2017] Cert success.
-----BEGIN CERTIFICATE-----

[ovizii]

Got (apparently) the same issue today, will subscribe and see if there is any progress on this issue or any more questions arise from the author of acme.sh

[EugenMayer]

Got the same here today, shell is zsh

zsh 5.3.1 (x86_64-debian-linux-gnu)
on debian:stretch

I am using docker debian:stretch so i should be easy for you to reproduce.

but i guess due to the shebang, bash is more interesting:
bash --version GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)

exact warning

[Fri Dec 15 10:57:06 UTC 2017] Verify finished, start to sign.
/usr/local/bin/acme.sh: line 1818: warning: command substitution: ignored null byte in input
[Fri Dec 15 10:57:07 UTC 2017] Cert success.

The certificate has been retrieved without issues, exit status 0

[dugwood]

Had this error on Debian Stretch too, for many weeks.

As the error happens here:
https://github.com/Neilpang/acme.sh/blob/master/acme.sh#L1818
but there's no issue here:
https://github.com/Neilpang/acme.sh/blob/master/acme.sh#L1805
I'd suspect that $response isn't the issue, but | _normalizeJson is (and it's the one creating the null byte).

Perhaps:
https://github.com/Neilpang/acme.sh/blob/master/acme.sh#L1373
tr may be creating NULL in replacement for \r\n (I've read other Linux programs create NULL when delimiters are empty, so maybe tr is doing the same with the delete flag).

Or perhaps just change the tr -d "\r\n" with tr -d "\r\n\0".

[indigoxela]

It looks like the warning is a new feature introduced in bash version 4.4.?.
(http://git.savannah.gnu.org/cgit/bash.git/tree/subst.c)

Which means, the null byte probably was always there.

dugwood's suggestion looks good to me (tr -d "\r\n\0").

[VojtechMyslivec]

It happens to me as well. line: 1837 and only during certificate renew.

Running Debian Stretch (up-to-date 9.3), bash version:

GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)

[stevebovy]

The following fix works ;) { the problem is in "_dbase64" }

_body="$(echo $_body | _dbase64 | tr -d '\0')"

[kobuki]

It happens to me of Debian 9/Stretch too. The additional null-biting tr appears to solve this. Pretty annoying with multiple renewals a night.

[Neilpang]

@kobuki @stevebovy
fixed, please try again.

export  BRANCH=dev
acme.sh --upgrade

[stevebovy]

I tried the new dev branch ; and I am getting a new error
[Thu Mar 8 20:03:20 PST 2018] Sign failed:

See output below;
/mnt/Optware/acmesh/acme.sh --issue --test --local-address 75.83.231.181
[Thu Mar 8 20:03:09 PST 2018] Using stage ACME_DIRECTORY: https://acme-staging.api.letsencrypt.org/directory
[Thu Mar 8 20:03:11 PST 2018] Standalone mode.
[Thu Mar 8 20:03:12 PST 2018] Single domain='home.miscbitbag.org'
[Thu Mar 8 20:03:13 PST 2018] Getting domain auth token for each domain
[Thu Mar 8 20:03:13 PST 2018] Getting webroot for domain='home.miscbitbag.org'
[Thu Mar 8 20:03:13 PST 2018] Getting new-authz for domain='home.miscbitbag.org'
[Thu Mar 8 20:03:15 PST 2018] The new-authz request is ok.
[Thu Mar 8 20:03:16 PST 2018] home.miscbitbag.org is already verified, skip http-01.
[Thu Mar 8 20:03:16 PST 2018] Verify finished, start to sign.

/mnt/Optware/acmesh/acme.sh: line 4105: warning: command substitution: ignored null byte in input
[Thu Mar 8 20:03:20 PST 2018] Sign failed:
[Thu Mar 8 20:03:20 PST 2018] Please check log file for more details: /root/.acme.sh/acme.sh.log

Because of the new error (above) the null-byte problem has now popped up in a new location
/mnt/Optware/acmesh/acme.sh: line 4105: warning: command substitution: ignored null byte in input

( As I mentioned before the null-byte problem is in "_dbase64" )
I wonder if this problem could also impact "_base64"
I would like to recommend that you fix "_dbase64" as follows

Usage: multiline

_dbase64() {
if [ "$1" ]; then
${ACME_OPENSSL_BIN:-openssl} base64 -d -A | tr -d '\0'
else
${ACME_OPENSSL_BIN:-openssl} base64 -d | tr -d '\0'
fi
}

NOTE: I do not understand what is causing the NEW-ERROR (above) ; the previous dev-branch version
does not have the (sign failed) problem

[indigoxela]

I can confirm, that the change in commit e1db5db8ac4604c123fbbfe3c87615b25657820c solves the warning problem (Debian stretch, bash 4.4.12).

I did not check out the dev branch though, but added the tiny change manually in version v2.7.7.
Then I manually changed the Le_NextRenewTime setting in a domain config, so the next cron job would start a renew.

The renewal (webroot mode) worked fine. No more warnings.

The error stevebovy determined, might come from another change in dev branch, unrelated to this issue.

[Neilpang]

@stevebovy your issue is fixed now. please try again.

We can not fix _dbase64() as you suggested.

_dbase64 is designed to convert any base64 string to its binary string. So, it should output any bytes in it.
For example, we use _dbase64 and _base64 to convert DER to PEM. if we lost \0 in _dbase64, we would get a wrong cert.

This issue is about bash 4.4, we just are not allowed to assign a string containing \0 to a bash variable.

So, the fix is simple, we just need to trim the \0 byte before we assign it to a bash variable.

[stevebovy]

Thanks

From: neil [mailto:[email protected]]
Sent: Friday, March 09, 2018 4:21 AM
To: Neilpang/acme.sh [e-mail]
Cc: stevebovy [e-mail]; Mention [e-mail]
Subject: Re: [Neilpang/acme.sh] Bash 4.4.12: "warning: command substitution: ignored null byte in input" (#1105)

@stevebovy https://github.com/stevebovy your issue is fixed now. please try again.

We can not fix _dbase64() as you suggested.

_dbase64 is designed to convert any base64 string to its binary string. So, it should output any bytes in it.
For example, we use _dbase64 and _base64 to convert DER to PEM. if we lost \0 in _dbase64, we would get a wrong cert.

This issue is about bash 4.4, we just are not allowed to assign a string containing \0 to a bash variable.

So, the fix is simple, we just need to trim the \0 byte before we assign it to a bash variable.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/Neilpang/acme.sh/issues/1105#issuecomment-371798162 , or mute the thread https://github.com/notifications/unsubscribe-auth/ALUA-6Xdm9VPn9UswqS3s6T0k51tiIEAks5tcnO2gaJpZM4QZLaM . https://github.com/notifications/beacon/ALUA-988oFIunvqRVXn0NoTGu_wKal75ks5tcnO2gaJpZM4QZLaM.gif