Kong: Fail to parse SSL certificate to kong-0.10.0

[flyabc]

Summary

I have a question on the SSL certificate authentication that I don't how to add
a correct certificate to kong. And I don't know how to configuration kong with https request.

Steps To Reproduce

1. I create a self-signed certificate to use SSL Through 'Creating an SSL certificate'
   in https://getkong.org/plugins/dynamic-ssl/.
   And got two files, the certificate will be stored in a file named server.crt, while the key is at server.key.

2. Next through the following steps to add a certificate with kong:
   curl -i -X POST http://localhost:8001/certificates \
   -d "cert=@usr/ssl/server.crt" \
   -d "key=@usr/ssl/server.key" \
   -d "snis=test.com"

The response:
HTTP/1.1 201 Created
Date: Wed, 05 Apr 2017 08:08:52 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: false
Server: kong/0.10.0

{"cert":"@usr\/ssl\/server.crt","id":"69146aa5-f934-4e43-8d85-b961de1d59d0","snis":["test.com"],"created_at":1491379733000,"key":"@usr\/ssl\/server.key"}

1. Then add a API:
   curl -i -X POST http://localhost:8001/apis \
   -d "name=test-api" \
   -d "upstream_url=http://xx.xx.xx.xx:8080/webIfep/contact/toinsert" \
   -d "hosts=test.com"

4.
Request to this API:
curl -i https://localhost:8443/ \
-H "Host:test.com"

Response:
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

5,Then Add parameters '-k' in the curl command,I got the content of this API:
curl -i -k -v https://localhost:8443 -H 'host:test.com'

Response:

• Rebuilt URL to: https://localhost:8443/
• Trying 127.0.0.1...
• Connected to localhost (127.0.0.1) port 8443 (#0)
• found 173 certificates in /etc/ssl/certs/ca-certificates.crt-----------------Not the certificate I specified
• found 697 certificates in /etc/ssl/certs------------------------------------Not the certificate I specified
• ALPN, offering http/1.1
• SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
• server certificate verification SKIPPED
• server certificate status verification SKIPPED
• common name: localhost (matched)
• server certificate expiration date OK
• server certificate activation date OK
• certificate public key: RSA
• certificate version: #1
• subject: C=US,ST=California,L=San Francisco,O=Kong,OU=IT Department,CN=localhost
• start date: Mon, 27 Mar 2017 01:40:55 GMT
• expire date: Wed, 26 Apr 2017 01:40:55 GMT
• issuer: C=US,ST=California,L=San Francisco,O=Kong,OU=IT Department,CN=localhost
• compression: NULL
• ALPN, server accepted to use http/1.1

GET / HTTP/1.1
host:test.com
User-Agent: curl/7.47.0
Accept: /

< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Wed, 05 Apr 2017 09:15:16 GMT
Date: Wed, 05 Apr 2017 09:15:16 GMT
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
< Connection: keep-alive
Connection: keep-alive
< Server: Apache-Coyote/1.1
Server: Apache-Coyote/1.1
< Set-Cookie: JSESSIONID=A632F52C28E06372F31BE323AE1402FC; Path=/webIfep; HttpOnly
Set-Cookie: JSESSIONID=A632F52C28E06372F31BE323AE1402FC; Path=/webIfep; HttpOnly
< Content-Language: en-US
Content-Language: en-US

Then I ask for an API that does not have an associated ssl certificate,I found that regardless of whether the certificate is associated with the API, it can be accessed by the command: curl -k -i localhost...

The above is the problem I encountered,and I try to change the order of step 2:
curl -i -X POST http://localhost:8001/certificates \
-d "cert=@/usr/ssl/server.crt" \
-d "key=@/usr/ssl/server.key" \
-d "snis=test.com"
or
curl -i -X POST http://localhost:8001/certificates \
-d "cert=/usr/ssl/server.crt" \
-d "key=/usr/ssl/server.key" \
-d "snis=test.com" .
This does not working too.

When I change step 2 the command is as follows:
curl -i -X POST http://localhost:8001/certificates \
-f "cert=@/usr/ssl/server.crt" \
-f "key=@/usr/ssl/server.key" \
-f "snis=test.com"
this respond:
curl: (26) couldn't open file "usr/ssl/server.crt"

I did not modify the contents of kong.conf, such as:ssl, ssl_cert, ssl_cert_key, are the default.

I do not know what to do, can someone help me, thank you!

• Kong version ($ kong version)0.10.0

[subnetmarco]

If you are using 0.10.x, instead of using the plugin, can you follow this guide: https://getkong.org/docs/0.10.x/proxy/#configuring-ssl-for-an-api

[thibaultcha]

Hi,

According to your curl command, you are setting the following certificate: @usr/ssl/server.crt and the following key: @usr/ssl/server.key to this API. Use the proper curl options to send the content of those files (the multipart-enabling -F option).

[flyabc]

@thefosk Thank you, you are right . I add ssl certificates via https://getkong.org/docs/0.10.x/proxy/#configuring-ssl-for-an-api . But I still have the same problem I do not know if my custom certificate has no effect.

[flyabc]

@thibaultcha Thanks! I re-edited the order through what you said:
curl -i -X POST localhost:8001/certificates
-F 'cert=@/usr/ssl/server.crt'
-F 'key=@/usr/ssl/server.key'
-F 'snis=test.com'

HTTP/1.1 100 Continue
HTTP/1.1 201 Created

and I checked the certificate in the database, There are keys and certificates that begin with the beginning and end of the string.
Then I visit this API:curl -i https://localhost:8443 -H 'host:test.com'
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html

Next ,I changed the order:curl -i -k https://localhost:8443 -H 'host:test.com'

• found 173 certificates in /etc/ssl/certs/ca-certificates.crt
• found 697 certificates in /etc/ssl/certs
• SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
• server certificate verification SKIPPED
• server certificate status verification SKIPPED
• common name: localhost (matched)
• server certificate expiration date OK
• server certificate activation date OK
• certificate public key: RSA
• certificate version: #1
• subject: C=US,ST=California,L=San Francisco,O=Kong,OU=IT Department,CN=localhost
• start date: Mon, 27 Mar 2017 01:40:55 GMT
• expire date: Wed, 26 Apr 2017 01:40:55 GMT
• issuer: C=US,ST=California,L=San Francisco,O=Kong,OU=IT Department,CN=localhost
• compression: NULL
• ALPN, server accepted to use http/1.1
  Although I got the contents of this API, but I do not know if my custom certificate is working.

[elapse2039]

I am seeing the same above response & behaviour in Kong 0.10.1 (latest Docker Container) using a signed cert from Network Solutions. To check that I had the bundle correct I have successfully tested the cert chain and server key work on a fresh Ubuntu16.04 based NGINX installation configured for SSL. These are the same two files I am using in the Kong certificates/ API parameters. Any help or clarification in the documentation would be very welcomed.

[flyabc]

@elapse2039 Excuse me, can you elaborate on the details? Do you have the same problem? How did you solve it?

[elapse2039]

@flyabc I believe I see the same symptoms as you but I am not using a self-signed cert.

[elapse2039]

Here are some details on what I'm trying to do (and maybe incorrectly)... I have obscured some information and tried to summarise what I'm trying to setup.

Notes:
a) Replaced some fields in this explaination with XXXXXXX for privacy.
b) My goal is to have one primary domain (showcase.XXXXXXXX.net) with Kong Gateway installed and my intention is to have it routing requests to different servers based on the URI.
c) All calls to the Kong Gateway must be over SSL. (For now I've enabled HTTP and HTTPS for testing)
d) The requests from Kong Gateway will be sent to the different servers will be HTTP-based.
e) All URIs will use the same SSL Cert for HTTPS and this is created for showcase.XXXXXXX.net. However, I do not want to apply site-wide SSL cert to all Kong Gateway API calls.

At some point in the future I may want to have different HTTPS certs for different URIs on the showcase.XXXXXXX.net domain.
Also, possibly have HTTPS upstream connections.

Steps I've tried so far:

1) Create CSR and KEY

> openssl req -new -newkey rsa:2048 -nodes -keyout nginx_host_server.key -out nginx_host_server.csr

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Vermont
Locality Name (eg, city) []:Burlington
Organization Name (eg, company) [Internet Widgits Pty Ltd]:XXXXXXX
Organizational Unit Name (eg, section) []:ATG
Common Name (e.g. server FQDN or YOUR name) []:showcase.XXXXXXX.net
Email Address []:[email protected]

Produced two files...
nginx_host_server.csr
nginx_host_server.key

2) Submit CSR to Naming Service... get a crt back
3) The bundled cert file or pem file: showcase.XXXXXXX.net.chain.crt
4) Verified this chain on a fresh Ubuntu16.04 NGINX installation.

5) Installed Ubuntu16.04 Docker Containers for Postgres and Kong.
6) Verified Kong works on HTTP with a few test APIs.

Followed this guide:
https://getkong.org/docs/0.10.x/proxy/#configuring-ssl-for-an-api

Kong is answering public calls on ports 80 and 443 at showcase.XXXXXXX.net and Admin is on 127.0.0.1:8001

7) Then add the certs to Kong:

curl -i -X POST http://localhost:8001/certificates \
-d "cert=@/home/tom/Downloads/showcase.XXXXXXX.net.chain.crt" \
-d "key=@/home/tom/Downloads/nginx_host_server.key" \
-d "snis=showcase.XXXXXXX.net"

HTTP/1.1 201 Created
Date: Sat, 08 Apr 2017 00:41:41 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: false
Server: kong/0.10.1

{"cert":"@\/home\/tom\/Downloads\/showcase.XXXXXXX.net.chain.crt","id":"8d15408c-31cd-42d8-aca6-aab6ae1ae111","snis":["showcase.XXXXXXX.net"],"created_at":1491612101000,"key":"@\/home\/tom\/Downloads\/nginx_host_server.key"}

8) Then add a test API which should connect using HTTP to a server behind the Kong API Gateway on a local subnet. Adding the 'hosts' field to allow Kong to recognise any associated certificate as per the "configuring-ssl-for-an-api. I have not enforced https-only in this test API so I can see if Kong gateway works e2e with plain HTTP.

curl -i -X POST \
--url http://localhost:8001/apis/ \
--data 'name=showcase55' \
--data 'uris=/55' \
--data 'hosts=showcase.XXXXXXX.net' \
--data 'upstream_url=http://192.168.1.139'

HTTP/1.1 201 Created
Date: Sat, 08 Apr 2017 14:20:59 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: false
Server: kong/0.10.1

{"http_if_terminated":true,"id":"b2451f91-fe1f-4cf4-bfc1-a38416a001ed","retries":5,"preserve_host":false,"created_at":1491661259000,"upstream_connect_timeout":60000,"upstream_url":"http:\/\/192.168.1.139","upstream_read_timeout":60000,"upstream_send_timeout":60000,"https_only":false,"strip_uri":true,"uris":["\/55"],"name":"showcase55","hosts":["showcase.XXXXXXX.net"]}

9) Test HTTP local subnet call to showcase55 server:

curl http://192.168.1.139/
Showcase55

10) Test external HTTP call:

I expected to see these messages...

curl http://showcase.XXXXXXX.net/55
Showcase55

curl http://showcase.XXXXXXX.net/
{"message":"no API found with those values"}

12) Test external call:

curl https://showcase.XXXXXXX.net/55
curl: (35) gnutls_handshake() failed: Internal error

curl https://showcase.XXXXXXX.net/
curl: (35) gnutls_handshake() failed: Internal error

Chrome gives a "This site can't proide a secure connection ERR_SSL_PROTOCOL_ERROR" message.

[elapse2039]

This afternoon I tried to set the same cert and key for Kong HTTPS connection.

If the same cert is set as above for showcase55 api I see the similar error messages above. However, if I delete this dynamic ssl cert for showcase55 then the response changes to "this site is insecure" message.

I assume, because Kong is using an automatically generated self-signed cert and not the one I have configured?

To update the Kong Docker configuration file I have mounted a host directory containing the following:
~/bin/docker/kong/etc_kong $ ls
kong.conf.default nginx_host_server.key showcase.XXXXXXX.net.chain.crt

sudo docker run -d --name kong \
--link kong-database:kong-database \
-v /home/tom/bin/docker/kong/etc_kong/:/etc/kong/ \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-p 80:8000 \
-p 443:8443 \
-p 127.0.0.1:8001:8001 \
kong

Logging into the Kong container I can see this is visible/available.
This is my Kong configuration file SSL section I've changed from the original kong.conf.default file.

ssl = on
ssl_cert = /etc/kong/certs/showcase.XXXXXXX.net.chain.crt
ssl_cert_key = /etc/kong/certs/nginx_host_server.key

To clarify my basic NGINX test, here is the NGINX default server settings for SSL using the same files:
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl on;
ssl_certificate /home/tom/bin/docker/kong/etc_kong/showcase.XXXXXXX.net.chain.crt;
ssl_certificate_key /home/tom/bin/docker/kong/etc_kong/nginx_host_server.key;
server_name showcase.XXXXXXX.net;

There is no problem using Chrome or curl to connect to this NGINX site over HTTPS.

[flyabc]

@elapse2039 Thanks for your help, I solved the problem with your help

[elapse2039]

@flyabc care to share how you solved it as I still have a problem? My post was not a solution but the steps I've taken without success. @thibaultcha do you want me to open an Issue of my own?

[flyabc]

@elapse2039 Hello
I compared the other posts ，Then do the following steps：
1，The prerequisite is that you have already uploaded the custom certificate correctly with the following command：curl -i -X POST http://localhost:8001/certificates \
-F "cert=@/usr/ssl/server.crt" \
-F "key=@/usr/ssl/server.key" \
-F "snis=test.com"
2 , Modify the hosts file：sudo gedit /etc/hosts
Add a row of data：127.0.0.1 test.com
3 , Try the order:
curl -k -v https:test.com：8443/test
--------"test" is my API‘s attritbute ；“test.com” is my API's hosts attribute
But I have a problem：I can not access this api with https：//test.com:8443/test or http:test.com:8000/test though brower！

[elapse2039]

So your problem is not fixed? I see you are using -k which essentially tells curl to ignore the security cert from my understanding. Can you re-open this Issue if you still have a problem as I do?

[flyabc]

@elapse2039 The custom certificate itself is not certified by the authority, I think we visit the custom certificate protection API is to ignore the system to verify the certificate, so I use the "-k" command, like we Some exceptions need to be added when accessing certain https in the browser
I do not know if my understanding is correct, but I can see that the custom certificate works。If you have other way ,can you tell me please?Thank you

[elapse2039]

Arh, I forgot you are using self-signed cert. You could instruct curl to use this or add it to your client's CA store for testing purposes.

I spent some more time surfing Kong Issues and I found one more SSL-related Issue on Kong github (https://github.com/Mashape/kong/issues/1880). This gave me the idea of opening up the Kong Docker image further and looking at what Kong is actually configured to use. Although I want to use dynamic SSL cert feature I have no idea how to trouble shoot that plugin. Instead, I decided to settle for HTTPS access to Kong using a single signed cert for the whole domain. Noting that setting the kong.default.conf SSL settings and certs (for NGINX section) makes no difference I found the actual internal nginx-kong.conf file reading through Issue 1880.

Note: I'm using Kong Docker because I cannot get native Kong install in Ubuntu 16.04 to connect to the database on startup.

sudo docker run -d --name kong \
--link kong-database:kong-database \
-v /home/tom/bin/docker/kong/etc_kong/:/etc/kong/ \
-v /home/tom/bin/docker/kong/usr_local_kong/:/usr/local/kong/ \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-p 80:8000 \
-p 443:8443 \
-p 127.0.0.1:8001:8001 \
kong

In this mounted directory you can see that nginx-kong.conf is using auto-generated certs (as suspected reading other Issues).

/// EXTRACT of nginx-kong.conf

server {
server_name kong;
listen 0.0.0.0:8000;
error_page 404 408 411 412 413 414 417 /kong_error_handler;
error_page 500 502 503 504 /kong_error_handler;

access_log logs/access.log;

listen 0.0.0.0:8443 ssl;
ssl_certificate /usr/local/kong/ssl/kong-default.crt;
ssl_certificate_key /usr/local/kong/ssl/kong-default.key;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_certificate_by_lua_block {
    kong.ssl_certificate()
}

/// EXTRACT

Since this directory is now mounted on the Docker host I copied over the signed-certs for my domain and restarted Kong. Now, Kong starts using what it thinks are auto-generated but in fact are my signed cert details.

Now, I can access Kong over HTTPS without a certificate error. I dont think this is how you are supposed/expected to set up site-wide HTTPS access for Kong but it now works. I also re-added the "55" API and it now routes as expected over an HTTPS connection:

curl -i https://showcase.XXXXXXX.net/55/
HTTP/1.1 200 OK
Date: Thu, 13 Apr 2017 14:21:02 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 37
Connection: keep-alive
Server: nginx/1.10.0 (Ubuntu)
Last-Modified: Fri, 07 Apr 2017 23:42:36 GMT
ETag: "58e823ec-25"
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type
Accept-Ranges: bytes
X-Kong-Upstream-Latency: 2
X-Kong-Proxy-Latency: 0
Via: kong/0.10.1

Showcase55

or

wget https://showcase.XXXXXX.net/55/ -O out.txt
--2017-04-13 10:21:37-- https://showcase.XXXXXX.net/55/
Resolving showcase.XXXXXX.net (showcase.XXXXXX.net)... XXX.XXX.XXX.XXX
Connecting to showcase.XXXXXX.net (showcase.XXXXXX.net)|XXX.XXX.XXX.XXX|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 37 [text/html]
Saving to: ‘out.txt’

out.txt 100%[======================================================>] 37 --.-KB/s in 0s

more out.txt
Showcase55

I think this Issue should _remain open_ as my steps are a workaround to site-wide HTTPS support and not dynamic SSL. I would like to know how both should be configured (as per Kong documentation) and not have to do this hack which I may not fully appreciate any side-effects.

[flyabc]

I understand you idea，you changed the configuration of the document path,rather than through the order to dynamically add the right?I have done this before，This is indeed the certificate is fixed, if we want to add a different host name of the certificate can not use this method. Use the ”-k“ command only temporary, the use of the certificate in the production environment is certified by the authority

[ghost]

I had the same problem as the OP. The Kong docs at https://getkong.org/docs/0.10.x/proxy/#configuring-ssl-for-an-api
are to blame for me. Using the multi-form upload instead of -d solved my problems.

[p0pr0ck5]

@TheDolo thanks for the report. This is fixed in https://github.com/Mashape/getkong.org/pull/391.

@elapse2039 as of 0.10, the dynamic SSL plugin is replaced by SSL handling via core, as noted above. As with the original author, it looks like you were originally adding the certificate object incorrectly via curl (-d instead of -F). Since that documentation is being resolved as noted above, I'm going to go ahead and close this out.