Official-images: Curl and wget in Alpine 3.5 fail to verify certificates

[yosifkit]

Creating a tracking issue from https://github.com/docker-library/official-images/pull/2756#issuecomment-288226764, which is mostly copied below. Also related to https://github.com/docker-library/tomcat/issues/65.

The issue at hand is that for the url below, curl fails to download the file because it is unable to verify the ca certificate chain (curl or wget on https://google.com works fine).

Non-busybox wget does work in some situations, but not curl :confused:

| downloader | ssl lib | works? |
|--------------|:--------:|:------:|
| busybox wget | openssl | yes |
| busybox wget | libressl | yes |
| apk add wget | openssl | no |
| apk add wget | libressl | yes |
| apk add curl | openssl | no |
| apk add curl | libressl | no |

(downloading this url: https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip)

cc @ncopa, have you heard of any similar problems with curl/wget and *ssl in alpine 3.5? (I can file an issue if you like.)

Test output of busybox wget, regular wget, and curl with openssl:

$ docker run -it --rm alpine:3.5 sh
/ # apk add --no-cache curl openssl
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/7) Installing ca-certificates (20161130-r0)
(2/7) Installing libssh2 (1.7.0-r2)
(3/7) Installing libcurl (7.52.1-r2)
(4/7) Installing curl (7.52.1-r2)
(5/7) Installing libcrypto1.0 (1.0.2k-r0)
(6/7) Installing libssl1.0 (1.0.2k-r0)
(7/7) Installing openssl (1.0.2k-r0)
Executing busybox-1.25.1-r0.trigger
Executing ca-certificates-20161130-r0.trigger
OK: 9 MiB in 18 packages
/ # wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
Connecting to sonarsource.bintray.com (108.168.243.150:443)
Connecting to akamai.bintray.com (23.206.199.89:443)
sonarqube-6.3.zip    100% |**********************************************************|   111M  0:00:00 ETA
/ # apk add --no-cache wget
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/1) Installing wget (1.18-r1)
Executing busybox-1.25.1-r0.trigger
OK: 9 MiB in 19 packages
/ # wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
--2017-03-21 21:47:45--  https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
Resolving sonarsource.bintray.com... 108.168.243.150
Connecting to sonarsource.bintray.com|108.168.243.150|:443... connected.
HTTP request sent, awaiting response... 302 
Location: https://akamai.bintray.com/fd/fdb4ba71e684bb02d86e84e191c2b82d0d669e2665a350c50bcd21e197a18459?__gda__=exp=1490133586~hmac=665182b4142055bcdfe7a794f63274c3c0de1c9eb0fc5a674fa75a9456518173&response-content-disposition=attachment%3Bfilename%3D%22sonarqube-6.3.zip%22&response-content-type=application%2Fzip&requestInfo=U2FsdGVkX1_GmrBiqWqDTYzLxzld6_Xt6OCvh3lcV6b_YIaHvEXI6cOm0luHZgFPJr2VCJLsPImy1ifkE2DnEYp33wIrHcyImPfebIL_8Z6kQnagxQw4BWSrJh32P8KabEZBcqpEnPKIYXpD5uiTYA [following]
--2017-03-21 21:47:46--  https://akamai.bintray.com/fd/fdb4ba71e684bb02d86e84e191c2b82d0d669e2665a350c50bcd21e197a18459?__gda__=exp=1490133586~hmac=665182b4142055bcdfe7a794f63274c3c0de1c9eb0fc5a674fa75a9456518173&response-content-disposition=attachment%3Bfilename%3D%22sonarqube-6.3.zip%22&response-content-type=application%2Fzip&requestInfo=U2FsdGVkX1_GmrBiqWqDTYzLxzld6_Xt6OCvh3lcV6b_YIaHvEXI6cOm0luHZgFPJr2VCJLsPImy1ifkE2DnEYp33wIrHcyImPfebIL_8Z6kQnagxQw4BWSrJh32P8KabEZBcqpEnPKIYXpD5uiTYA
Resolving akamai.bintray.com... 23.206.199.89
Connecting to akamai.bintray.com|23.206.199.89|:443... connected.
ERROR: cannot verify akamai.bintray.com's certificate, issued by 'CN=Verizon Akamai SureServer CA G14-SHA2,OU=Cybertrust,O=Verizon Enterprise Solutions,L=Amsterdam,C=NL':
  Unable to locally verify the issuer's authority.
To connect to akamai.bintray.com insecurely, use `--no-check-certificate'.
/ # curl -o sonar.zip -fSL https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

[stonehz]

this is a workaround that gets the versions from 3.4..
curl and git works

echo 'http://dl-cdn.alpinelinux.org/alpine/v3.4/main' >> /etc/apk/repositories
apk update

apk del curl
apk del git
apk del libcurl
apk del libssh2

apk add libssh2==1.7.0-r0 --no-cache
apk add libcurl==7.52.1-r1 --no-cache 
apk add curl==7.52.1-r1 --no-cache 
apk add git==2.8.3-r0 --no-cache 

[sgtsquiggs]

I can reproduce this, except instead of complaining about certificates, the processes are simply killed:

[~] # docker run -it --rm alpine:3.5 sh
Unable to find image 'alpine:3.5' locally
3.5: Pulling from library/alpine

627beaf3eaaf: Pull complete
Digest: sha256:58e1a1bb75db1b5a24a462dd5e2915277ea06438c3f105138f97eb53149673c4
Status: Downloaded newer image for alpine:3.5
/ # apk add --no-cache curl openssl
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/7) Installing ca-certificates (20161130-r1)
(2/7) Installing libssh2 (1.7.0-r2)
(3/7) Installing libcurl (7.52.1-r2)
(4/7) Installing curl (7.52.1-r2)
(5/7) Installing libcrypto1.0 (1.0.2k-r0)
(6/7) Installing libssl1.0 (1.0.2k-r0)
(7/7) Installing openssl (1.0.2k-r0)
Executing busybox-1.25.1-r0.trigger
Executing ca-certificates-20161130-r1.trigger
OK: 9 MiB in 18 packages
/ # wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
Connecting to sonarsource.bintray.com (75.126.118.188:443)
Connecting to akamai.bintray.com (104.107.25.166:443)
sonarqube-6.3.zip    100% |***********************************************************************************************************************************************************|   111M  0:00:00 ETA
/ # apk add --no-cache wget
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/1) Installing wget (1.18-r2)
Executing busybox-1.25.1-r0.trigger
OK: 9 MiB in 19 packages
/ # wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
--2017-04-07 23:40:18--  https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
Killed
/ # curl -o sonar.zip -fSL https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.3.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Killed
/ #

[tianon]

Given that this is really a bug in Alpine 3.5 itself, and not something actionable for the official images, I'm going to close. If someone's still interested in finding a resolution to this, I'm sure the Alpine folks would appreciate an issue filed at https://bugs.alpinelinux.org detailing what's happening so they can attempt to reproduce it and see whether a fix is something they can reasonably push out for 3.5 (since it's in maintenance/security mode now).

[ritiek]

I had a similar problem with Python.

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

This answer helped me.

What we need is:

RUN apk --no-cache add ca-certificates

[giuliobraini]

Thank you, @ritiek
Your proposed solution worked for me also:

RUN apk --no-cache add ca-certificates

[drenze]

I am experiencing this same issue, and

RUN apk --no-cache add ca-certificates

does NOT work for me.

[ysangkok]

You might need to update libssl if curl/wget is dynamically linked. At least I had that problem with Ubuntu on Travis CI.