Kong: Kong can't find nginx

[jason-riddle]

Bug:

Kong is unable to find nginx even though it it defined in my path.

Background:

My path looks like the following

/app/tools/opt/bin:/usr/local/bin:/usr/bin:/bin

and typing in nginx -v returns

nginx version: openresty/1.7.10.2

and the path to nginx is /app/tools/opt/bin/nginx

Potential Culprit?

My knowledge on Lua is limited, but it looks like the paths used to search for the nginx binary are defined here:

https://github.com/Mashape/kong/blob/6cc0d183deceb9c3008553ad3ae144fb2d532c40/kong/cli/utils/signal.lua#L55-L60

local NGINX_SEARCH_PATHS = {
  "/usr/local/openresty/nginx/sbin/",
  "/usr/local/opt/openresty/bin/",
  "/usr/local/bin/",
  "/usr/sbin/"
}

Where the logic to find the nginx binary is done here:

https://github.com/Mashape/kong/blob/6cc0d183deceb9c3008553ad3ae144fb2d532c40/kong/cli/utils/signal.lua#L64-L72

local function find_nginx()
  for i = 1, #NGINX_SEARCH_PATHS + 1 do
    local prefix = NGINX_SEARCH_PATHS[i] and NGINX_SEARCH_PATHS[i] or ""
    local to_check = prefix..NGINX_BIN
    if is_openresty(to_check) then
      return to_check
    end
  end
end

So it looks like only NGINX_SEARCH_PATHS is used to search for the binary and not $PATH? I ask because I see the following comment which makes it seem like $PATH is being searched?

https://github.com/Mashape/kong/blob/6cc0d183deceb9c3008553ad3ae144fb2d532c40/kong/cli/utils/signal.lua#L62

-- Try to find an `nginx` executable in defined paths, or in $PATH

[subnetmarco]

@jason-riddle I am looking into this. On a side note may I ask how did you install Kong?

[jason-riddle]

@thefosk I've been trying to get kong working on the heroku platform. In order to do this, I created a Dockerfile to compile openresty and luarocks from source. Heroku has a readonly filesystem which resulted in a lot of "hacks" in order to get this to work.

[subnetmarco]

@jason-riddle can you try again with the fix/nginx-path branch which should have fixed your problem (https://github.com/Mashape/kong/tree/fix/nginx-path) ?

[jason-riddle]

I don't believe it did? There's not much available in the form of logging, so I'm unable to confirm if anything else weird is going on.

~ $ kong start -c kong.yml
[INFO] Using configuration: kong.yml
[INFO] Kong version.......0.5.0
       Proxy HTTP port....8000
       Proxy HTTPS port...8443
       Admin API port.....8001
       dnsmasq port.......8053
       Database...........cassandra keepalive=60000 timeout=2000 contact_points=ec2-54-172-252-55.compute-1.amazonaws.com:33500 keyspace=kong
[INFO] Connecting to the database...
[ERR] Kong cannot find an 'nginx' executable.
Make sure it is in your $PATH or in one of the following directories:
/usr/local/openresty/nginx/sbin/
/usr/local/opt/openresty/bin/
/usr/local/bin/
/usr/sbin/

[subnetmarco]

@jason-riddle I have just updated the branch with more logging messages. Would you be able to retry again and paste the output?

[jason-riddle]

~ $ kong start -c kong.yml
[INFO] Using configuration: kong.yml
[INFO] Kong version.......0.5.0
       Proxy HTTP port....8000
       Proxy HTTPS port...8443
       Admin API port.....8001
       dnsmasq port.......8053
       Database...........cassandra keepalive=60000 timeout=2000 contact_points=ec2-54-172-252-55.compute-1.amazonaws.com:33500 keyspace=kong
[INFO] Connecting to the database...
[INFO] Auto-generating the default SSL certificate and key...
Checking in /usr/local/openresty/nginx/sbin/nginx
Checking in /usr/local/opt/openresty/bin/nginx
Checking in /usr/local/bin/nginx
Checking in /usr/sbin/nginx
[ERR] Kong cannot find an 'nginx' executable.
Make sure it is in your $PATH or in one of the following directories:
/usr/local/openresty/nginx/sbin/
/usr/local/opt/openresty/bin/
/usr/local/bin/
/usr/sbin/

[subnetmarco]

It seems like the PATH environment variable is empty. Can you run:

$ echo $PATH && kong start -c kong.yml

[jason-riddle]

~ $ echo $PATH && kong start -c kong.yml
/app/tools/opt/bin:/usr/local/bin:/usr/bin:/bin
[INFO] Using configuration: kong.yml
[INFO] Kong version.......0.5.0
       Proxy HTTP port....8000
       Proxy HTTPS port...8443
       Admin API port.....8001
       dnsmasq port.......8053
       Database...........cassandra keepalive=60000 timeout=2000 contact_points=ec2-54-172-252-55.compute-1.amazonaws.com:33500 keyspace=kong
[INFO] Connecting to the database...
Checking in /usr/local/openresty/nginx/sbin/nginx
Checking in /usr/local/opt/openresty/bin/nginx
Checking in /usr/local/bin/nginx
Checking in /usr/sbin/nginx
[ERR] Kong cannot find an 'nginx' executable.
Make sure it is in your $PATH or in one of the following directories:
/usr/local/openresty/nginx/sbin/
/usr/local/opt/openresty/bin/
/usr/local/bin/
/usr/sbin/

[subnetmarco]

This makes me very confused. I hope you don't mind pulling again the branch and try again. This time I will be printing exactly how Kong parses the PATH variable.

[jason-riddle]

More logs.

Just a small note, this isn't a huge issue as there are more important things to work on for kong. I just wanted to see if I could get this running on heroku. I'll probably follow up in a month or so.

~ $ kong start -c kong.yml
[INFO] Using configuration: kong.yml
[INFO] Kong version.......0.5.0
       Proxy HTTP port....8000
       Proxy HTTPS port...8443
       Admin API port.....8001
       dnsmasq port.......8053
       Database...........cassandra keepalive=60000 timeout=2000 contact_points=ec2-54-172-252-55.compute-1.amazonaws.com:33500 keyspace=kong
[INFO] Connecting to the database...
[INFO] Auto-generating the default SSL certificate and key...
PATH variable is: /app/tools/opt/bin:/usr/local/bin:/usr/bin:/bin
/app/tools/opt/bin/lua: /app/tools/opt/share/lua/5.1/kong/cli/utils/signal.lua:79: module 'inspect' not found:No LuaRocks module found for inspect
    no field package.preload['inspect']
    no file '/app/.luarocks/share/lua/5.1/inspect.lua'
    no file '/app/.luarocks/share/lua/5.1/inspect/init.lua'
    no file '/app/tools/opt/share/lua/5.1/inspect.lua'
    no file '/app/tools/opt/share/lua/5.1/inspect/init.lua'
    no file '/app/.luarocks/lib/lua/5.1/inspect.so'
    no file '/app/tools/opt/lib/lua/5.1/inspect.so'
stack traceback:
    [C]: in function 'require'
    /app/tools/opt/share/lua/5.1/kong/cli/utils/signal.lua:79: in function 'find_nginx'
    /app/tools/opt/share/lua/5.1/kong/cli/utils/signal.lua:269: in function 'send_signal'
    /app/tools/opt/share/lua/5.1/kong/cli/start.lua:23: in main chunk
    [C]: in function 'require'
    /app/tools/opt/lib/luarocks/rocks/kong/0.5.0-1/bin/kong:55: in main chunk
    [C]: at 0x00404e10

~ $ which nginx
/app/tools/opt/bin/nginx

[thibaultcha]

As explained in #612 and as you rightfully quoted the comment @jason-riddle, searching in the $PATH should already be done by the line I pointed out there. I don't believe this is an issue with the current code, could it be related to Heroku?

[subnetmarco]

could it be related to Heroku

It could be, trying to figure it out. Heroku was on a very old issue (#178) and on our roadmap, so happy to sort this out.

@jason-riddle how can I replicate this problem?

[jason-riddle]

(Rough) Steps to Create an App and Reproduce..

# Clone kong and checkout the fix/nginx-path
git clone kong
cd kong
git fetch upstream 'fix/nginx-path':'local/fix/nginx-path'
git checkout fix/nginx-path

# Download Procfile and app.json (for heroku), and a slightly modified kong.yml
wget https://raw.githubusercontent.com/jason-riddle/kong-dist-heroku/master/Procfile
wget https://raw.githubusercontent.com/jason-riddle/kong-dist-heroku/master/app.json
wget https://raw.githubusercontent.com/jason-riddle/kong-dist-heroku/master/kong.yml

git add -A
git commit -m "Ready for heroku deploy"

# Create the kong-heroku app, specify a custom build path, and push the app to heroku
APP="handsome-gorilla-$((RANDOM%9000+1000))"
heroku create "${APP}"
heroku config:add BUILDPACK_URL=https://github.com/jason-riddle/heroku-buildpack-openresty.git
git push heroku heroku:fix/nginx-path

# Run a bash shell on an instance
heroku run bash
Running bash on handsome-gorilla-3465... up, run.2927
~ $

You are now running a shell on heroku. Now just run kong..

~ $ kong start -c kong.yml

Note: I'm using a custom build pack, heroku-buildpack-openresty which downloads tools-latest.tar.gz which includes some binaries (openresty, luarocks, lua), libraries, header files, etc.

[thibaultcha]

Fixed, will be included in 0.5.2.

[jason-riddle]

Awesome! Looks good. :+1:

[subnetmarco]

Just released

[scottillogical]

I'm getting this issue on 0.5.4

[subnetmarco]

@scottschulthess

• What OS are you running?
• How did you install Kong?
• Can you run sudo find / -type f -name "nginx" and tell me the output of the command?

[joshuaswilcox]

I am as well:

OS X 10.11.3

Installed kong from source with

make install

also

-> which nginx
-> /usr/local/bin/nginx

and the find commands produces

/usr/local/Cellar/nginx/1.8.0/bin/nginx

[scottillogical]

Yeah I installed kong and nginx through homebrew.

➜  kong  which nginx
/usr/local/bin/nginx
➜  kong  which kong
/usr/local/bin/kong

[ojengwa]

I just ran into the same issue. I also installed both Nginx and kong via homebrew. My version of kong is 0.5.2 which I guess is the latest version as of the time of writing this. Has this issue been fixed?

[sonicaghi]

@ojengwa latest version is 0.7

[francisco-andrade]

I am having a similar problem running on mesos.

Running local goes ok, but on mesos I have the "Cant find nginx" error:

�[0m�[34m[INFO] �[0m�[0mKong 0.7.0
�[0m�[34m[INFO] �[0m�[0mUsing configuration: /etc/kong/kong.yml
�[0m�[34m[INFO] �[0m�[0mSetting working directory to /usr/local/kong
�[0m�[34m[INFO] �[0m�[0mdatabase...........cassandra auth=password= username= timeout=5000 data_centers= password= ssl=verify=false enabled=false contact_points=: replication_strategy=SimpleStrategy username= replication_factor=1 keyspace=kong
�[0m�[34m[INFO] �[0m�[0mdnsmasq............address=127.0.0.1:8053 dnsmasq=true port=8053
�[0m�[34m[INFO] �[0m�[0mserf ..............-profile=wan -rpc-addr=127.0.0.1:7373 -event-handler=member-join,member-leave,member-failed,member-update,member-reap,user:kong=/usr/local/kong/serf_event.sh -bind=0.0.0.0:7946 -advertise=:7946 -node=_0.0.0.0:7946 -log-level=err
�[0m�[34m[INFO] �[0m�[0mTrying to auto-join Kong nodes, please wait..
�[0m�[33m[WARN] �[0m�[0mCannot join :. If the node does not exist anymore it will be automatically purged.
�[0m�[34m[INFO] �[0m�[0mSuccessfully auto-joined :
�[0m�[34m[INFO] �[0m�[0mAuto-generating the default SSL certificate and key...
�[0m�[34m[INFO] �[0m�[0mLeaving cluster..
�[0m�[31m[ERR] �[0m�[0mCan't find nginx
�[0m�[31m[ERR] �[0m�[0mCould not start Kong

Could it be related to low free memory on the container?

[subnetmarco]

@chicoandrade how did you install Kong? Can you run the following command to check where nginx is?

$ find / -type f -name "nginx"

[francisco-andrade]

Hi @thefosk ,

Here is the output:

$ find / -type f -name "nginx"
/usr/local/openresty/nginx/sbin/nginx

I'm installing kong through dpkg:

wget https://github.com/Mashape/kong/releases/download/0.7.0/kong-0.7.0.trusty_all.deb
dpkg -i kong-0.7.0.trusty_all.deb

And running with

/usr/local/bin/kong start

[subnetmarco]

@chicoandrade what OS are you running?

[francisco-andrade]

@thefosk , kong is running on Debian GNU/Linux 8 (jessie)

[allisthere2love]

As we discovered investigating the issue with @thibaultCha, the problem that @chicoandrade is having occurs when the Ruxit agent is running in the host.

[allisthere2love]

Hi, just a new info in the ruxit/kong/nginx issue
The problem occurs when the ruxit agent is up-to-date, in version 1.8.9
Previous versions of the agent doesn't cause the issue

[subnetmarco]

Interesting. I wonder why a monitoring agent would interfere with the nginx executable lookup.

[cvrabie]

Anybody banging their head against the wall because openresty nginx is on the path, you should be aware that Kong crashes with the exact same message if it does find nginx but nginx fails to start. In my case, I was missing libpcre.so.0.

[Ansar1628]

Facing the same problem from past few hours.

I am running kong on mac os and i dont have ruxit installed in my machine

output of sudo find / -type f -name "nginx"

/usr/local/Cellar/nginx-full/1.10.1/bin/nginx
/usr/local/Cellar/ngx_openresty/1.9.15.1/bin/nginx

Can someone please help in resolving the problem

when I try to start nginx alone using sudo nginx in mac i get below error

_nginx: [error] [lua] kong.lua:161: init(): Startup error: /usr/local/share/lua/5.1/kong/tools/io.lua:66: bad argument #1 to 'open' (string expected, got nil)_

[subnetmarco]

@thibaultcha could this be a Homebrew problem?

[thibaultcha]

Can't tell without more informations such as the actual installation method used, the desired Kong version, the full trace for the error, etc...

[ragingbal]

'could not find OpenResty 'resty' executable'

• Ubuntu 14.04, kong-0.9.4.trusty_all.deb ,
• nginx version: openresty/1.9.15.1 .
• Resty executable located at /usr/local/openresty/bin/resty .

Has anybody had any luck with kong on Ubuntu Trusty? Would 16.04 work better?

Tail of log:

2016/11/03 07:36:00 [verbose] stopped services
Error:
/usr/local/share/lua/5.1/kong/cmd/start.lua:37: /usr/local/share/lua/5.1/kong/cmd/utils/log.lua:49: bad argument #4 to 'format' (no value)
stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/kong/cmd/start.lua:37: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:89: in function
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:89: in function
/usr/local/bin/kong:15: in function 'file_gen'
init_worker_by_lua:38: in function
[C]: in function 'pcall'
init_worker_by_lua:45: in function

[subnetmarco]

Make sure that the path to the "resty" command is in your PATH environment variable.

export PATH=$PATH:/usr/local/openresty/bin

Marco Palladino | CTO @ Mashape | mashape.com | [phone]

On Nov 3, 2016, at 12:38 AM, Balaji Bal [email protected] wrote:

'could not find OpenResty 'resty' executable'

Ubuntu 14.04, kong-0.9.4.trusty_all.deb ,
nginx version: openresty/1.9.15.1 .
Resty executable located at /usr/local/openresty/bin/resty .
Has anybody had any luck with kong on Ubuntu Trusty? Would 16.04 work better?

Tail of log:

2016/11/03 07:36:00 [verbose] stopped services
Error:
/usr/local/share/lua/5.1/kong/cmd/start.lua:37: /usr/local/share/lua/5.1/kong/cmd/utils/log.lua:49: bad argument #4 to 'format' (no value)
stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/kong/cmd/start.lua:37: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:89: in function
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:89: in function
/usr/local/bin/kong:15: in function 'file_gen'
init_worker_by_lua:38: in function
[C]: in function 'pcall'
init_worker_by_lua:45: in function

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[ehizogie]

Hello,
Tried updating the path with no luck. Any help would be really useful

[ragingbal]

Partial temporary fix, diagnosis and notes ( with the usual claim of - I do not get Lua ).

Setting the PATH does not fix the issue. ( in case of Ubuntu 14.0.4 / Kong 9.5/ 1.11.2.1 )

The issue does seems to come from the start up script throwing an incorrect error 'cannot find resty'. In my case the issue was that on launching resty a 'Location not set' warning is being thrown by Perl.

The fix was to set a Perl environment variable to squelch the warning 'export PERL_BADLANG=0'.

The error is probably difficult to reproduce because it 'probably' comes(in my case) from the Mac terminal SSH client and its passing the locale settings to the server.

[raymondtay]

I concur with you assessment that by setting the ENV variable export PERL_BADLANG=0 then i was able to start Kong via kong start.

If anyone is interested, i've attached the output

ubuntu@ip-10-0-0-165:~$ export PERL_BADLANG=0
ubuntu@ip-10-0-0-165:~$ kong start
ulimit is currently set to "1024". For better performance set it to at least "4096" using "ulimit -n"
migrating core for database kong
core migrated up to: 2015-01-12-175310_skeleton
core migrated up to: 2015-01-12-175310_init_schema
core migrated up to: 2015-11-23-817313_nodes
core migrated up to: 2016-02-29-142793_ttls
migrating hmac-auth for database kong
hmac-auth migrated up to: 2015-09-16-132400_init_hmacauth
migrating rate-limiting for database kong
rate-limiting migrated up to: 2015-08-03-132400_init_ratelimiting
rate-limiting migrated up to: 2016-07-25-471385_ratelimiting_policies
migrating acl for database kong
acl migrated up to: 2015-08-25-841841_init_acl
migrating ip-restriction for database kong
ip-restriction migrated up to: 2016-05-24-remove-cache
migrating jwt for database kong
jwt migrated up to: 2015-06-09-jwt-auth
jwt migrated up to: 2016-03-07-jwt-alg
migrating galileo for database kong
galileo migrated up to: 2016-04-15_galileo-import-mashape-analytics
migrating key-auth for database kong
key-auth migrated up to: 2015-07-31-172400_init_keyauth
migrating oauth2 for database kong
oauth2 migrated up to: 2015-08-03-132400_init_oauth2
oauth2 migrated up to: 2016-07-15-oauth2_code_credential_id
oauth2 migrated up to: 2016-12-22-283949_serialize_redirect_uri
migrating response-ratelimiting for database kong
response-ratelimiting migrated up to: 2015-08-03-132400_init_response_ratelimiting
response-ratelimiting migrated up to: 2016-08-04-321512_response-rate-limiting_policies
migrating basic-auth for database kong
basic-auth migrated up to: 2015-08-03-132400_init_basicauth
3 migrations ran
Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:22: Error connecting to Serf agent: dial tcp 127.0.0.1:7373: getsockopt: connection refused

Run with --v (verbose) or --vv (debug) for more details
ubuntu@ip-10-0-0-165:~$ kong start --vv
2017/01/16 08:08:36 [verbose] Kong: 0.9.7
2017/01/16 08:08:36 [debug] ngx_lua: 10006
2017/01/16 08:08:36 [debug] nginx: 1011002
2017/01/16 08:08:36 [debug] Lua: LuaJIT 2.1.0-beta2
2017/01/16 08:08:36 [debug] PRNG seed: 112266929713
2017/01/16 08:08:36 [verbose] no config file found at /etc/kong.conf
2017/01/16 08:08:36 [verbose] reading config file at /etc/kong/kong.conf
2017/01/16 08:08:36 [debug] admin_listen = "0.0.0.0:8001"
2017/01/16 08:08:36 [debug] anonymous_reports = true
2017/01/16 08:08:36 [debug] cassandra_consistency = "ONE"
2017/01/16 08:08:36 [debug] cassandra_contact_points = {"127.0.0.1"}
2017/01/16 08:08:36 [debug] cassandra_data_centers = {"dc1:2","dc2:3"}
2017/01/16 08:08:36 [debug] cassandra_keyspace = "kong"
2017/01/16 08:08:36 [debug] cassandra_port = 9042
2017/01/16 08:08:36 [debug] cassandra_repl_factor = 1
2017/01/16 08:08:36 [debug] cassandra_repl_strategy = "SimpleStrategy"
2017/01/16 08:08:36 [debug] cassandra_ssl = false
2017/01/16 08:08:36 [debug] cassandra_ssl_verify = false
2017/01/16 08:08:36 [debug] cassandra_timeout = 5000
2017/01/16 08:08:36 [debug] cassandra_username = "kong"
2017/01/16 08:08:36 [debug] cluster_listen = "0.0.0.0:7946"
2017/01/16 08:08:36 [debug] cluster_listen_rpc = "127.0.0.1:7373"
2017/01/16 08:08:36 [debug] cluster_profile = "wan"
2017/01/16 08:08:36 [debug] cluster_ttl_on_failure = 3600
2017/01/16 08:08:36 [debug] custom_plugins = {}
2017/01/16 08:08:36 [debug] database = "postgres"
2017/01/16 08:08:36 [debug] dnsmasq = true
2017/01/16 08:08:36 [debug] dnsmasq_port = 8053
2017/01/16 08:08:36 [debug] log_level = "notice"
2017/01/16 08:08:36 [debug] lua_code_cache = "on"
2017/01/16 08:08:36 [debug] lua_package_cpath = ""
2017/01/16 08:08:36 [debug] lua_package_path = "?/init.lua;./kong/?.lua"
2017/01/16 08:08:36 [debug] lua_ssl_verify_depth = 1
2017/01/16 08:08:36 [debug] mem_cache_size = "128m"
2017/01/16 08:08:36 [debug] nginx_daemon = "on"
2017/01/16 08:08:36 [debug] nginx_optimizations = true
2017/01/16 08:08:36 [debug] nginx_worker_processes = "auto"
2017/01/16 08:08:36 [debug] pg_database = "kong"
2017/01/16 08:08:36 [debug] pg_host = "127.0.0.1"
2017/01/16 08:08:36 [debug] pg_password = "**"
2017/01/16 08:08:36 [debug] pg_port = 5432
2017/01/16 08:08:36 [debug] pg_ssl = false
2017/01/16 08:08:36 [debug] pg_ssl_verify = false
2017/01/16 08:08:36 [debug] pg_user = "kong"
2017/01/16 08:08:36 [debug] prefix = "/usr/local/kong/"
2017/01/16 08:08:36 [debug] proxy_listen = "0.0.0.0:8000"
2017/01/16 08:08:36 [debug] proxy_listen_ssl = "0.0.0.0:8443"
2017/01/16 08:08:36 [debug] serf_path = "serf"
2017/01/16 08:08:36 [debug] ssl = true
2017/01/16 08:08:36 [verbose] prefix in use: /usr/local/kong
2017/01/16 08:08:36 [verbose] preparing nginx prefix directory at /usr/local/kong
2017/01/16 08:08:36 [verbose] saving serf identifier to /usr/local/kong/serf/serf.id
2017/01/16 08:08:36 [debug] searching for OpenResty 'resty' executable
2017/01/16 08:08:36 [debug] /usr/local/openresty/bin/resty -V: 'nginx version: openresty/1.11.2.1'
2017/01/16 08:08:36 [debug] found OpenResty 'resty' executable at /usr/local/openresty/bin/resty
2017/01/16 08:08:36 [verbose] saving serf shell script handler to /usr/local/kong/serf/serf_event.sh
2017/01/16 08:08:36 [verbose] SSL enabled, no custom certificate set: using default certificate
2017/01/16 08:08:36 [verbose] default SSL certificate found at /usr/local/kong/ssl/kong-default.crt
2017/01/16 08:08:36 [warn] ulimit is currently set to "1024". For better performance set it to at least "4096" using "ulimit -n"
2017/01/16 08:08:36 [verbose] running datastore migrations
2017/01/16 08:08:36 [verbose] migrations up to date
2017/01/16 08:08:36 [verbose] dnsmasq not running, deleting /usr/local/kong/pids/dnsmasq.pid
2017/01/16 08:08:36 [debug] searching for 'dnsmasq' executable
2017/01/16 08:08:36 [debug] /usr/local/sbin/dnsmasq -v: ''
2017/01/16 08:08:36 [debug] 'dnsmasq' executable not found at /usr/local/sbin/dnsmasq
2017/01/16 08:08:36 [debug] /usr/local/bin/dnsmasq -v: ''
2017/01/16 08:08:36 [debug] 'dnsmasq' executable not found at /usr/local/bin/dnsmasq
2017/01/16 08:08:36 [debug] /usr/sbin/dnsmasq -v: 'Dnsmasq version 2.75 Copyright (c) 2000-2015 Simon Kelley'
2017/01/16 08:08:36 [debug] found 'dnsmasq' executable at /usr/sbin/dnsmasq
2017/01/16 08:08:36 [debug] starting dnsmasq: /usr/sbin/dnsmasq -p 8053 --pid-file=/usr/local/kong/pids/dnsmasq.pid -N -o --listen-address=127.0.0.1
2017/01/16 08:08:36 [debug] sending signal to pid at: /usr/local/kong/pids/serf.pid
2017/01/16 08:08:36 [debug] kill -0 cat /usr/local/kong/pids/serf.pid >/dev/null 2>&1
2017/01/16 08:08:36 [verbose] serf agent not running, deleting /usr/local/kong/pids/serf.pid
2017/01/16 08:08:36 [debug] checking 'serf' executable from 'serf_path' config setting
2017/01/16 08:08:36 [debug] serf version: 'Serf v0.7.0'
2017/01/16 08:08:36 [debug] starting serf agent: nohup serf agent -profile 'wan' -rpc-addr '127.0.0.1:7373' -event-handler 'member-join,member-leave,member-failed,member-update,member-reap,user:kong=/usr/local/kong/serf/serf_event.sh' -bind '0.0.0.0:7946' -node 'ip-10-0-0-165_0.0.0.0:7946_8f412c3f631343149895505258ac49b9' -log-level 'err' > /usr/local/kong/logs/serf.log 2>&1 & echo $! > /usr/local/kong/pids/serf.pid
2017/01/16 08:08:36 [verbose] waiting for serf agent to be running
2017/01/16 08:08:36 [debug] sending signal to pid at: /usr/local/kong/pids/serf.pid
2017/01/16 08:08:36 [debug] kill -0 cat /usr/local/kong/pids/serf.pid >/dev/null 2>&1
2017/01/16 08:08:36 [verbose] serf agent started
2017/01/16 08:08:36 [verbose] auto-joining serf cluster
2017/01/16 08:08:36 [verbose] no other nodes found in the cluster
2017/01/16 08:08:36 [verbose] registering serf node in datastore
2017/01/16 08:08:36 [verbose] cluster joined and node registered in datastore
2017/01/16 08:08:36 [debug] searching for OpenResty 'nginx' executable
2017/01/16 08:08:36 [debug] /usr/local/openresty/nginx/sbin/nginx -v: 'nginx version: openresty/1.11.2.1'
2017/01/16 08:08:36 [debug] found OpenResty 'nginx' executable at /usr/local/openresty/nginx/sbin/nginx
2017/01/16 08:08:36 [debug] starting nginx: /usr/local/openresty/nginx/sbin/nginx -p /usr/local/kong -c nginx.conf
2017/01/16 08:08:36 [debug] nginx started
2017/01/16 08:08:36 [info] Kong started

[paulk8s]

Partial temporary fix, diagnosis and notes ( with the usual claim of - I do not get Lua ).

Setting the PATH does not fix the issue. ( in case of Ubuntu 14.0.4 / Kong 9.5/ 1.11.2.1 )

The issue does seems to come from the start up script throwing an incorrect error 'cannot find resty'. In my case the issue was that on launching resty a 'Location not set' warning is being thrown by Perl.

The fix was to set a Perl environment variable to squelch the warning 'export PERL_BADLANG=0'.

The error is probably difficult to reproduce because it 'probably' comes(in my case) from the Mac terminal SSH client and its passing the locale settings to the server.

export PERL_BADLANG=0 worked for me. Thanks