Kong: Custom plugin prevents Kong 1.0 from starting

Created on 27 Sep 2018  路  7Comments  路  Source: Kong/kong

Summary

Custom plugins that load with 0.14 do not load with 1.0 and prevent Kong from starting.
Plugins:

Steps To Reproduce

  1. Build kong:1.0.0rc1 Dockerfile locally from https://github.com/Kong/docker-kong/blob/master/alpine/Dockerfile
  2. Built kong-wshirey Dockerfile locally using kong:1.0.0rc1as base with custom plugins
FROM kong:1.0.0rc1
RUN apk update && apk upgrade && \
    apk add --no-cache bash git openssh && \
    luarocks install https://raw.githubusercontent.com/wshirey/kong-plugin-jwt-claims-validate/master/kong-plugin-jwt-claims-validate-1.0-1.rockspec && \
    luarocks install https://raw.githubusercontent.com/wshirey/kong-plugin-file-log-extended/master/kong-plugin-file-log-extended-1.0-1.rockspec
ENV KONG_PLUGINS=bundled,kong-plugin-jwt-claims-validate,kong-plugin-file-log-extended
  1. Run postgres DB
docker run -d --name kong-database \
              --network=kong-net \
              -p 5432:5432 \
              -e "POSTGRES_USER=kong" \
              -e "POSTGRES_DB=kong" \
              postgres:9.6
  1. Run migrations 
docker run --rm \
    --network=kong-net \
    -e "KONG_DATABASE=postgres" \
    -e "KONG_PG_HOST=kong-database" \
    kong-wshirey kong migrations bootstrap

Output

bootstrapping database...
migrating core on database 'kong'...
core migrated up to: 000_base (executed)
core migrated up to: 001_14_to_15 (executed)
migrating rate-limiting on database 'kong'...
rate-limiting migrated up to: 000_base_rate_limiting (executed)
migrating hmac-auth on database 'kong'...
hmac-auth migrated up to: 000_base_hmac_auth (executed)
migrating oauth2 on database 'kong'...
oauth2 migrated up to: 000_base_oauth2 (executed)
oauth2 migrated up to: 001_14_to_15 (executed)
migrating jwt on database 'kong'...
jwt migrated up to: 000_base_jwt (executed)
migrating basic-auth on database 'kong'...
basic-auth migrated up to: 000_base_basic_auth (executed)
migrating key-auth on database 'kong'...
key-auth migrated up to: 000_base_key_auth (executed)
migrating acl on database 'kong'...
acl migrated up to: 000_base_acl (executed)
acl migrated up to: 001_14_to_15 (executed)
migrating response-ratelimiting on database 'kong'...
response-ratelimiting migrated up to: 000_base_response_rate_limiting (executed)
12 migrations processed
12 executed
database is up-to-date
  1. Start Kong
docker run -d --name kong \
    --network=kong-net \
    -e "KONG_DATABASE=postgres" \
    -e "KONG_PG_HOST=kong-database" \
    -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
    -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
    -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
    -p 8000:8000 \
    -p 8443:8443 \
    -p 8001:8001 \
    -p 8444:8444 \
    kong-wshirey

Output

2018/09/27 20:55:46 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/db/dao/plugins.lua:94: attempt to index field 'schema' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:94: in function 'convert_legacy_schema'
/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:240: in function 'load_plugin_schemas'
/usr/local/share/lua/5.1/kong/init.lua:226: in function 'init'
init_by_lua:3: in main chunk
nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/db/dao/plugins.lua:94: attempt to index field 'schema' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:94: in function 'convert_legacy_schema'
/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:240: in function 'load_plugin_schemas'
/usr/local/share/lua/5.1/kong/init.lua:226: in function 'init'
init_by_lua:3: in main chunk

Additional Details & Logs

  • Kong version 1.0.0rc1
  • Operating System - Docker alpine
tasbug
Source
picture wshirey
👍1

Most helpful comment

Hi @wshirey,

Thank you for testing 1.0rc1, and thank you for reporting your observations! We really appreciate it. There is more work that we need to do to ensure that custom plugins can be run in 1.0, and we will make sure rc2 can do so :) In the meantime, it is reports like this one that help us making sure it will, so thank you again. We will make sure to fix this issue by then.

In the meantime, would you mind pursuing your testing of 1.0rc1 without using this plugin for now?

picture thibaultcha on 28 Sep 2018
👍2

All 7 comments

Hi @wshirey,

Thank you for testing 1.0rc1, and thank you for reporting your observations! We really appreciate it. There is more work that we need to do to ensure that custom plugins can be run in 1.0, and we will make sure rc2 can do so :) In the meantime, it is reports like this one that help us making sure it will, so thank you again. We will make sure to fix this issue by then.

In the meantime, would you mind pursuing your testing of 1.0rc1 without using this plugin for now?

thibaultcha picture thibaultcha on 28 Sep 2018
👍2

Also pinging @hishamhm here

thibaultcha picture thibaultcha on 28 Sep 2018

Hi @wshirey, thanks for reporting!

The crash itself is an easy fix, PR in #3813 is attached to this thread.

The more interesting question is about the schema changes in 1.0. Kong 1.0 uses a new schema library that we have been gradually rolling out to the internal Kong entities and have now extended to plugins: it has a number of benefits, including support for more precise type definitions, which means less custom Lua code for handwritten type-checking in plugin schema definitions.

Since this is a major change, to ease the transition we added an automatic converter which does a best-effort translation of the old schema format to the new one. It is best-effort since the translation is not fully lossless, for two reasons: first, because some of the definitions of the old schema were very loose, in particular the "table" type that we're hitting here; second, the check functions in the old schema library were overly powerful (you could do a _lot_ of stuff with it, including things a schema validation operation shouldn't really be doing), so the custom check functions aren't auto-converted.

In many cases, the new validation attributes (or the new type definitions) of the new schema library can replace the custom check functions. For this, we added support for a new_type field in the old schema format, where a new-format type definition can be added, and then this takes precedence over the auto-conversion. You can see some examples of that in action in the schemas of Kong plugins bundled with 1.0.0rc1.

However, a complication in the case of kong-plugin-jwt-claims-validate is that it is _relying_ on the looseness of the old schema format, by accepting what is essentially a map of "string" keys to "booleans or numbers or integers"... which is something that the new schema library does not support in general.

Having said all that, could you give this alternate schema.lua a try?

local cjson = require "cjson"

local valid_types = {
  ["string"]  = true, 
  ["boolean"] = true,
  ["number"]  = true
}

local function is_scalar(v)
  local type = type(v)
  return valid_types[type] or type == cjson.null
end

local function claim_check(value, conf)
  for k,v in pairs(value) do
    if not is_scalar(v) then
      return false, "'claims."..k.."' is not one following types: [boolean, string, number]"
    end
  end
  return true, nil
end

return {
  no_consumer = true,
  fields = {
    uri_param_names = {type = "array", default = {"jwt"}},
    claims = {
      type = "table",
      func = claim_check,
      new_type = {
        type = "map",
        keys = { type = "string" },
        values = { type = "skip", custom_validator = is_scalar },
      }
    }
  }
}

(PS: there's a small bug in your original schema.lua: the return true, nil statement is inside the for loop, so claim_check only checks at most one claim of the table.)

hishamhm picture hishamhm on 28 Sep 2018

I'll give this a shot in the next day or so. Thanks for the feedback on the plugin!

wshirey picture wshirey on 29 Sep 2018

~@hishamhm Your suggestion worked, not only in loading the plugin, but I verified that the plugin is working correctly. Thanks for the help!~

~I'll let you decide whether to keep this PR open.~

Sorry I was still targeting 0.14 when I tested. Having to figure out how to get a Dockerfile running with your changes. Will update shortly.

wshirey picture wshirey on 1 Oct 2018

@hishamhm I was able to build a Docker container from the source with your changes. Kong at least starts, but when I try to create a jwt-claims-validate plugin, Kong responds with error response. I updated the plugin as you suggested and the latest rockspec can be installed via

luarocks install https://raw.githubusercontent.com/daxko/kong-plugin-jwt-claims-validate/kong-1.0.0rc1-compatibility/kong-plugin-jwt-claims-validate-1.0-1.rockspec

Request

curl -X POST \
  http://localhost:8001/plugins \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "jwt-claims-validate",
    "config": {
        "uri_param_names": "jwt",
        "claims": {
            "username": "wshirey"
        }
    }
}'

Response

HTTP/1.1 500 Internal Server Error
Date: Mon, 01 Oct 2018 15:16:55 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/1.0.0rc1
Content-Length: 43
{"message":"An unexpected error occurred"}

Error logs

2018/10/01 15:14:48 [error] 32#0: *2286 [lua] init.lua:143: handle_error(): /usr/local/share/lua/5.1/lapis/application.lua:397: /usr/local/share/lua/5.1/kong/db/schema/init.lua:1181: attempt to index local 'field_schema' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/kong/db/schema/init.lua: in function 'adjust_field_for_context'
/usr/local/share/lua/5.1/kong/db/schema/init.lua:1258: in function 'adjust_field_for_context'
/usr/local/share/lua/5.1/kong/db/schema/init.lua:1258: in function 'process_auto_fields'
/usr/local/share/lua/5.1/kong/db/dao/init.lua:683: in function 'insert'
/usr/local/share/lua/5.1/kong/api/endpoints.lua:235: in function </usr/local/share/lua/5.1/kong/api/endpoints.lua:217>
2018-10-01T15:14:48.564263500Z 
stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/lapis/application.lua:397: in function 'handler'
/usr/local/share/lua/5.1/lapis/application.lua:130: in function 'resolve'
/usr/local/share/lua/5.1/lapis/application.lua:161: in function </usr/local/share/lua/5.1/lapis/application.lua:159>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch'
/usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api'
content_by_lua(nginx-kong.conf:175):2: in function <content_by_lua(nginx-kong.conf:175):1>, client: 172.23.0.1, server: kong_admin, request: "POST /routes/routes_status/plugins HTTP/1.1", host: "localhost:8001"
wshirey picture wshirey on 1 Oct 2018

1.0rc2 has just been released and should fix this!

https://discuss.konghq.com/t/kong-1-0rc2-available-for-testing/2142

Thanks for the report @wshirey, feel free to reopen if the issue occurs again.

thibaultcha picture thibaultcha on 23 Oct 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

cospotato picture cospotato  路  3Comments
hos7ein picture hos7ein  路  3Comments
hbagdi picture hbagdi  路  3Comments
hbagdi picture hbagdi  路  3Comments
z843248880 picture z843248880  路  3Comments