Supervisor: Supervisorctl tail fails on logs with ANSI codes

[Ismael]

logfile:
^[[31m^[[7memergency^[[0m: Master DOWN!

error: , not well-formed (invalid token): line 51, column 39: file: /usr/lib/python2.6/xmlrpclib.py line: 601

[mnaberez]

We should try to fix this but in the meantime, you can set strip_ansi=true in the [supervisord] section as a workaround.

_Edit: Fixed the section name to be [supervisord]. It originally said [program:x]. Please see the configuration guide for all settings._

[despairblue]

strip_ansy=true does not work. It still outputs

error: <class 'xml.parsers.expat.ExpatError'>, not well-formed (invalid token): line 25, column 0: file: /usr/lib/python2.7/xmlrpclib.py line: 557

Using tail -f on the other hand works, but is rather hard to integrate with hubot (to be able to check on processes via company chat). So having a way to fix the tail command would be nice. Any way I could help with that?

[mnaberez]

strip_ansy=true does not work.

Try strip_ansi=true instead.

[despairblue]

My bad. I misspelled it in my answer but not in the config file. strip_ansi=true does not work.

[mnaberez]

strip_ansi=true does not work.

Note that strip_ansi should be set in the [supervisord] section. I noticed my comment above originally said [program:x] section and have corrected it. Please see the configuration guide.

Here's an emit_escapes.py script to test it:

import sys
while True:
    sys.stdout.write('\x1b[31m\x1b[7memergency\x1b[0m: Master DOWN!\n')
    sys.stdout.flush()

Minimal config file:

[supervisord]
logfile = /tmp/supervisord.log
strip_ansi = true

[inet_http_server]
port = 127.0.0.1:9001

[supervisorctl]
serverurl = http://127.0.0.1:9001

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[program:emit_escapes]
command = python /path/to/emit_escapes.py
stdout_logfile = /tmp/foo.log

Running supervisord in the foreground with this config file:

$ supervisord -n -c sample.conf

Tailing the output with supervisorctl:

$ supervisorctl -c sample.conf tail emit_escapes
emergency: Master DOWN!
emergency: Master DOWN!
emergency: Master DOWN!

If that works for you but tailing your program does not, it might be that your program is emitting an escape sequence that is not filtered. Inspect the logfile for your program and note any escape sequences you find.

You can check a particular sequence like this:

$ python
>>> from supervisor.dispatchers import stripEscapes
>>> stripEscapes('\x1b[31m\x1b[7memergency\x1b[0m: Master DOWN!')
'emergency: Master DOWN!'

[despairblue]

That works! Thank you. Should have read the configuration guide as well :)

[vijayviji]

strip_ansi is not working for me. Here's the supervisor config file -> http://pastebin.com/QFzpg5Jg

I'm using ubuntu 14.04 in VM. The app folder is shared b/w host and the VM. Host is Windows 10.

Error:

vagrant@PeersKart:/app$ supervisorctl -c ./config/supervisord.ini node RUNNING pid 2382, uptime 0:00:14 supervisor> tail node error: <class 'xml.parsers.expat.ExpatError'>, not well-formed (invalid token): line 5, column 29: file: /usr/lib/python2.7/xmlrpclib.py line: 558 supervisor> shutdown Really shut the remote supervisord process down y/N? y Shut down supervisor>

[arrowd]

I'm running supervisord 3.3.4 and still see this problem with my program. strip_ansi = true didn't help. Any ideas how to debug/workaround this?

[timotheecour]

problem still there with 4.1.0; note:

• tail -f works fine with or without ansi codes (even without strip_ansi = true)
• tail -100 fails with ansi codes (with or without strip_ansi = true)

partial workaround

look for childlogdir in /etc/supervisor/supervisord.conf and find process you want, eg:
childlogdir=/Users/timothee/temp/supervisor/childlogs
find logfile (with random suffix) for process, eg:

tail -f /Users/timothee/temp/supervisor/childlogs/temp2_cmd_echo_forever_color-stderr---supervisor-iuwl1alh.log

where to fix

the bug appears here:
$homebrew_D/Cellar/supervisor/4.1.0/libexec/lib/python3.7/site-packages/supervisor/xmlrpc.py

        # on 2.x, the Expat parser doesn't like Unicode which actually
        # contains non-ASCII characters
        data = data.encode('ascii', 'xmlcharrefreplace')
        p, u = self.getparser()
        p.feed(data) # triggers bug

note: file can be edited directly and supervisorctl will pickup those modifications right away

[ruudk]

Having the same issue, any news on this?

[JorgenSolli]

Also facing this issue.

error: <class 'xml.parsers.expat.ExpatError'>, not well-formed (invalid token): line 5, column 15: file: /usr/lib/python2.7/xmlrpclib.py line: 558

[aendeavor]

@mnaberez What's the status here? We are seeing this in docker-mailserver/docker-mailserver#1768 and we would need to make logs colorless which is an unacceptable solution just because parsing of an escape code failed.

This issue is open for 8 years and hasn't been addressed. Why was #1306 just closed off without help?

[mnaberez]

What's the status here?

This issue is still open.

Why was #1306 just closed off without help?

Please see the explanation of why that could not be merged in https://github.com/Supervisor/supervisor/pull/1306#issuecomment-767698680.