st2api and st2auth always binds to 127.0.0.1

[saahn]

I've set the host param in st2.conf for auth and api to 0.0.0.0, but the config is ignored and the gunicorn process binds to 127.0.0.1:

[root@base-stackstorm2-138-onint ~]# ps -ef | grep gunicorn
st2      20711     1  9 22:50 ?        00:00:00 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 127.0.0.1:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30
st2      20723     1  9 22:50 ?        00:00:00 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2stream/gunicorn_config.py -k eventlet -b 127.0.0.1:9102 --workers 1 --threads 10 --graceful-timeout 10 --timeout 30
st2      20735     1  9 22:50 ?        00:00:00 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2auth/gunicorn_config.py -k eventlet -b 127.0.0.1:9100 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30
...

I've tried modifying the hard-coded local addresses in both https://github.com/StackStorm/st2/blob/master/st2api/st2api/gunicorn_config.py#L29 and https://github.com/StackStorm/st2/blob/master/st2auth/st2auth/gunicorn_config.py#L29

as well as the init scripts /etc/init.d/st2api, /etc/init.d/st2auth, which was hard-coding the bind address in the DAEMON_ARGS variable: DAEMON_ARGS="-k eventlet -b 127.0.0.1:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30 --pid ${PIDFILE}".

However, the auth and api endpoints are still binding to 127.0.0.1 after restarting the st2 processes.

I'm using st2 v1.4, and my environment was set up by following the documentation.

My st2.conf looks like:

[exporter]
logging = /etc/st2/logging.exporter.conf

[garbagecollector]
logging = /etc/st2/logging.garbagecollector.conf

[auth]
host = 0.0.0.0
port = 9100
use_ssl = False
debug = True
enable = True
logging = /etc/st2/logging.auth.conf
mode = standalone

# Note: Settings bellow are only used in "standalone" mode
backend = flat_file
backend_kwargs = {"file_path": "/etc/st2/.htpasswd"}

# Base URL to the API endpoint excluding the version (e.g. http://myhost.net:9101/)
api_url = 

[system]
base_path = /opt/stackstorm

[syslog]
host = 127.0.0.1
port = 514
facility = local7
protocol = udp

[log]
excludes = requests,paramiko
redirect_stderr = False
mask_secrets = True

[system_user]
user = stanley
ssh_key_file = /home/stanley/.ssh/stanley_rsa

[messaging]
url = amqp://guest:[email protected]:5672/

[ssh_runner]
remote_dir = /tmp
use_paramiko_ssh_runner = True

Perhaps I am missing other hard-coded overrides?

[Kami]

Which distribution you are running?

If you are running Ubuntu, you also need to edit upstart job files, since upstart is used on Ubuntu (IIRC, those files are in /etc/init/*.conf).

It's also worth noting that we explicitly bind those services to localhost since they are running behind a proxy (nginx) and aren't directly exposed to the internet.

[Kami]

@manasdk @lakshmi-kannan We discussed this before but we need to decide what to do with those listen directives in the config when using packages and gunicorn.

One option is to utilize those values instead of the CLI arguments which are passed to gunicorn process (then we need to be careful we ship a correct default config with the packages). Another one is to simply document this behavior and let user know they need to edit service manager files if they want to change listen ip and / or port.

In any case, it's something which caused quite a lot of confusion already so we need to come up with a solution soon.

[saahn]

@Kami I'm using Ubuntu 14.04.
Thanks, updating the upstart job files did the trick.

[lakshmi-kannan]

Yeah, I think we should make config work with gunicorn. It might get nasty because of ordering issues. Anyways, that seems like the ideal solution.

[hujunya]

I am using v1.4 on CentOS7.1.
When I run mistral-v2 workflow, the mistral-server log shows that Failed to establish a new connection to the server ip and 9101 port. Because the st2api bind to 127.0.0.1:9101 so it refused the connection to (server ip):9101.
How to config the st2api to bind to 0.0.0.0:9101 ?
How to config the st2api to run with nginx ?

st2      24435  0.0  0.4 245244 26728 ?        Ss   14:55   0:01 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2stream/gunicorn_config.py -k eventlet -b 127.0.0.1:9102 --workers 1 --threads 10 --graceful-timeout 10 --timeout 30
st2      24656  0.1  0.6 286236 39780 ?        S    14:55   0:02 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2stream/gunicorn_config.py -k eventlet -b 127.0.0.1:9102 --workers 1 --threads 10 --graceful-timeout 10 --timeout 30
st2      28183  0.0  0.4 245244 26732 ?        Ss   15:12   0:01 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 127.0.0.1:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30
st2      28195  0.1  0.7 304472 47736 ?        S    15:12   0:01 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 127.0.0.1:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30

[manasdk]

@tenyearslater The right way to do this is to update the URL for mistral to communicate with StackStorm API. In /etc/st2/st2.conf add -

[mistral]
api_url = https://${EXTERNAL_IP}/api

After updating the StackStorm config file restart StackStorm services by sudo st2ctl restart

[estee-tew]

https://stackstorm.slack.com/archives/community/p1492824598789091

[emptywee]

Not sure why it doesn't work for the author, but updating DAEMON_ARGS via /etc/sysconfig/st2api worked very well for me:

 # ps aux | grep st2api
st2      18075 25.8  0.1 232588 28156 ?        S    09:57   0:01 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 0.0.0.0:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30 --pid /var/run/st2/st2api.pid
st2      18110 45.0  0.2 284364 45472 ?        D    09:57   0:00 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/gunicorn_pecan /opt/stackstorm/st2/lib/python2.7/site-packages/st2api/gunicorn_config.py -k eventlet -b 0.0.0.0:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30 --pid /var/run/st2/st2api.pid
root     18121  0.0  0.0 103312   904 pts/3    S+   09:57   0:00 grep st2api
09:57:14 j0 root@lvcops101:~
# netstat -anp | grep 9101
tcp        0      0 0.0.0.0:9101                0.0.0.0:*                   LISTEN      18075/python

# cat /etc/sysconfig/st2api
DAEMON_ARGS="-k eventlet -b 0.0.0.0:9101 --workers 1 --threads 1 --graceful-timeout 10 --timeout 30 --pid ${PIDFILE}"

This way I don't have to update the original init.d script, which can be updated during upgrade. I am on CentOS 6.8