Kong: Kong not support ARM platform

[timusketeers]

when we compile Kong source code on ARM platform and try to start and run it, it failed, it can not start successfully. at first, we find it is a problem for openresty-1.13.6.1/2, after we contract with the author of Openresty, he offered me a latest version openresty-1.15.8.1rc.tar.gz(https://openresty.org/download/openresty-1.15.8.1rc0.tar.gz). it really works. But when I try integrate Kong upon this version of Openresty, it failed. At current, we find that when postgres table contains the type of array, when execute SQL statement query, there is a "Error: bad light userdata pointer" error occur.
Also, the discuss about Openresty can see here: https://github.com/openresty/lua-nginx-module/issues/1152

[thibaultcha]

@zllovezj Hi,

Yes, I was personally involved with the development and the release of 1.15.8.1rc0/rc1. BTW, you can use rc1 now (instead of rc0). I also saw the comment you posted there.

We have not yet bumped the underlying OpenResty version in Kong yet, and do not officially support 1.15.8.1 (both rc0 and rc1). We'll look at it shortly, but we are focused on our own 1.1.0 release for the time being.

[subnetmarco]

@zllovezj may I ask where are you deploying Kong that requires ARM support? Are you deploying it on a specific cloud, or else?

[timusketeers]

@subnetmarco it's on a specific cloud

[svenwal]

I have a working Kong 1.0.3 on my Raspberry Pi based Kubernetes with my own build Docker image. I could share my Dockerfiles (which shows all the compilation steps) after cleaning it up if this would help you. Using OpenResty 1.13.6 with Postgres

[svenwal]

See https://github.com/svenwal/kong-arm for my ARM based Docker image.

In the provided Dockerfile you can view all the steps it has taken me to compile Kong on ARM (including dependencies).

Note: This is not a Kong Inc. supported image but at least "works for me on my Raspberry Pi 3"

[timusketeers]

@svenwal Thanks for your kindness, I am afraid it will give little help for us. Thank you!

[thibaultcha]

@zllovezj BTW, the alert error logs you saw were due to the new _G write guard we introduced to discourage the use of global variables due to the underlying changes from the ARM64 work. Some of Kong’s dependencies may be setting globals, and we’ll review the warnings accordingly.

Again, please allow for some time for us to test it, and maybe you can follow our incoming patches on your side. Expect a 1.2.0 release with 1.15.8.1, most likely next month.

[timusketeers]

@thibaultcha Thanks for your message.

[p0pr0ck5]

Going to close this, as there is no pending action and a number of Kong folks have answered in detail. @timusketeers thanks for bringing this up! We're looking forward to supporting ARM deployments soon! BTW keep an eye on https://github.com/Kong/kong/pull/4382 :D

[p0pr0ck5]

Whups, I had two different responses going in different tabs and closed this when it shouldn't been been closed! Re-opening this just so everything's clear on where ARM support in Kong distributions stands.

As Thibault mentioned, do be on the lookout for the next release of Kong which should contain OpenResty 1.15.8.1, supporting ARM architectures, hopefully next month!

[timusketeers]

@p0pr0ck5 , Hi, Is there any further progress or plan of the deadline about the issue "Kong to support ARM platform"? thanks

[hutchic]

@timusketeers you can keep track of progress in the following places:
kong/kong-build-tools: ability to compile current Kong w armv7hf (which is capable of running on arm64)
kong/kong: upgrading the openresty version Kong uses to one which can be compiled / ran on arm64

For a more immediate solution the following docker image might work https://github.com/svenwal/kong-arm

[timusketeers]

@hutchic , thanks for your message. I will try it.

[luxianjun]

@timusketeers Is kong currently supported on ARM64? I am experiencing the same problem.

[thibaultcha]

@hutchic’s above comment is still up-to-date as of today.

[lag-linaro]

Do you have a timeline for when Kong will be officially supported on AArch64?

[thibaultcha]

@lag-linaro We do not have an official timeline yet. Kong 1.3 will ship with OpenResty 1.15.8.1, but some of Kong's dependencies may require further work to be compatible on ARM64. We will eventually get there, but maybe not by 1.3, given ARM64 support is not a high priority for us at the moment.

[wjcaitu]

@thibaultcha I also use openresty 1.15.8.1 to integrated with kong 0.11.x version. And nginx part is ok. But when I call the admin api, the log comes:
2019/07/31 20:47:18 [error] 50#0: *9945 [lua] init.lua:83: handle_error(): bad light userdata pointer stack traceback: [C]: in function 'parse' /usr/local/share/lua/5.1/lapis/router.lua:344: in function 'build_route' /usr/local/share/lua/5.1/lapis/router.lua:312: in function 'build' /usr/local/share/lua/5.1/lapis/router.lua:433: in function 'resolve' /usr/local/share/lua/5.1/lapis/application.lua:161: in function </usr/local/share/lua/5.1/lapis/application.lua:159> [C]: in function 'xpcall' 172.26.0.105 - admin [31/Jul/2019:20:47:18 +0800] "GET /apis HTTP/1.1" 500 54 "-" "curl/7.29.0" /usr/local/share/lua/5.1/lapis/application.lua:159: in function 'dispatch' /usr/local/share/lua/5.1/lapis/nginx.lua:215: in function 'serve_admin_api' content_by_lua(nginx-kong.conf:165):2: in main chunk, client: 172.26.0.105, server: kong_admin, request: "GET /apis HTTP/1.1", host: "hw-105:8008"

It seems kong-lapis doesn't support arm64?
In another issue, https://github.com/openresty/openresty/issues/479
agentzh said

You just need to put the following to the http {} block in your nginx.conf:

init_by_lua_block {
require "resty.core"
}
I tried, but still failed. Does anyone can help with it?

[carnei-ro]

@thibaultcha according to CHANGELOG.md, Kong 1.3.0 rc1 is compatible with ARM64. Could Kong publish an ARM64 docker image to dockerhub? I just saw that there are just x86_64 images there.

[hutchic]

@wjcaitu might be best to post your issue on Kong Nation for assistance as we try to keep github issues for bugs / feature requests

@carnei-ro it's being actively worked on

[vielmetti]

@timusketeers - @daurnimator has updated the luaossl library to address at least one of the Lua lightuserdata issues you are experiencing.

If there are other libraries that are triggering this condition we can work back upstream to see about getting those issues identified.

[vcunat]

Or you could kill all of them by a single luajit-wide workaround like proposed in https://github.com/Kong/openresty-patches/pull/47

[lag-linaro]

@carnei-ro it's being actively worked on

@hutchic Could you provide a little more information please? I don't see any open PRs:

https://github.com/docker-library/official-images/pulls?utf8=%E2%9C%93&q=is%3Apr+kong+

[hutchic]

The link vcunat shared is where most of the work is isolated at this time https://github.com/Kong/openresty-patches/pull/47

We're able to build / release arm64 but some of our dependencies aren't compatible hence the lightuserdata issues folks are seeing when running Kong built for arm64

[lag-linaro]

Hopefully these issues are rectified and you guys can release support for AArch64 soon.

Thanks for the update Colin (@hutchic). I'll keep a close eye on this.

[carnei-ro]

The lightuserdata was merged. 1.3.0 has been released. Any updates in kong for arm64?

[Bevisy]

I have already made a arm64 image bevisy/kong:1.3.0_arm64. Now it's working well. Maybe you can try.
For more details, you can check repo.

[guanlan]

Xenial arm64 package has been released with 1.3, closing this.
Also, we are actively investigating the load failure under ARM64 via #5042, stay tuned @timusketeers @carnei-ro @Bevisy @lag-linaro

[lag-linaro]

Docker Hub still lacks support for arm64v8.

This Issue should be left open until Kong's library file in Docker's Official Images has been updated.

[hutchic]

The Docker Hub image will be available shortly. To build it locally it's available git://kong/docker-kong

[lag-linaro]

Thanks for the update Colin (@hutchic).

Out of interest, what does "shortly" mean? A day, week, month?

[hutchic]

Should be available once the docker team reviews, approves and merges https://github.com/docker-library/official-images/pull/6661

[lag-linaro]

Excellent, thank you Colin (@hutchic).

[kevinsingapore]

Do you have a timeline for when Kong will be officially supported on AArch64?

do you had running kong ok on aarch64?

[hutchic]

@kevinsingapore they've been available for a while now

ubuntu@34:~$ docker run -it --rm kong:ubuntu uname -a
Unable to find image 'kong:ubuntu' locally
ubuntu: Pulling from library/kong
Digest: sha256:c28243282c2c20235f77c9374a909f3288b3c8102f708c9d9971b64a2cf0e521
Status: Downloaded newer image for kong:ubuntu
Linux d2730bdcb773 4.15.0-1043-aws #45-Ubuntu SMP Mon Jun 24 14:08:49 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@34:~$ docker run -it --rm kong:ubuntu kong version
2.0.4

If there's a specific issue you're having let's open a new bug and leave this one as is 👍

[kevinsingapore]

@kevinsingapore they've been available for a while now

ubuntu@34:~$ docker run -it --rm kong:ubuntu uname -a
Unable to find image 'kong:ubuntu' locally
ubuntu: Pulling from library/kong
Digest: sha256:c28243282c2c20235f77c9374a909f3288b3c8102f708c9d9971b64a2cf0e521
Status: Downloaded newer image for kong:ubuntu
Linux d2730bdcb773 4.15.0-1043-aws #45-Ubuntu SMP Mon Jun 24 14:08:49 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@34:~$ docker run -it --rm kong:ubuntu kong version
2.0.4

If there's a specific issue you're having let's open a new bug and leave this one as is 👍

thank u!
got it.