Proxysql: show full processlist hostgroup: -1

[leagueofsoups]

Hi. My php-fpm freez after update to proxysql-2.0.3-1.x86_64.
CentOS Linux release 7.4.1708 (Core)
kernel 3.10.0-693.11.1.el7.x86_64
strace php-fpm process:

strace output:
connect(8, {sa_family=AF_INET, sin_port=htons(3306), sin_addr=inet_addr("")}, 16) = -1 EINPROGRESS (Operation now in progress)
poll([{fd=8, events=POLLIN|POLLOUT|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=8, revents=POLLOUT}])
getsockopt(8, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
fcntl(8, F_SETFL, O_RDWR) = 0
setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(8, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=8, revents=POLLIN}])
recvfrom(8, "J\0\0\0", 4, MSG_DONTWAIT, NULL, NULL) = 4
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=8, revents=POLLIN}])
recvfrom(8, "\n5.7.207\0mysql_native_password\0", 78, MSG_DONTWAIT, NULL, NULL) = 74
sendto(8, "Wmysql_native_password\0", 91, MSG_DONTWAIT, NULL, 0) = 91
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=8, revents=POLLIN}])
recvfrom(8, ",\0\0\2", 4, MSG_DONTWAIT, NULL, NULL) = 4
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=8, revents=POLLIN}])
recvfrom(8, "\376mysql_native_password\0000\0", 78, MSG_DONTWAIT, NULL, NULL) = 44
sendto(8, "", 24, MSG_DONTWAIT, NULL, 0) = 24
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=8, revents=POLLIN}])
recvfrom(8, "\7\0\0\4\0\0\0\0\0\0\0", 34, MSG_DONTWAIT, NULL, NULL) = 11
sendto(8, "\5\0\0\0\2", 9, MSG_DONTWAIT, NULL, 0) = 9 <-- use I think
poll([{fd=8, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928

fd 8 is connection to proxysql, I examine show processlist on proxysql, and I have many connections with HG -1, all it connection have status Sleep. My user have default schema and default hg, no problem with query rules, but something is going wrong, and hg go to -1, php-fpm freez(timeout problem). Help with debug proxysql, please.

After downgrade to proxysql-1.4.12-1.2.el7.x86_64 application working fine.

[renecannao]

when current_hostgroup (or hostgroup in stats_mysql_processlist) is -1 , it means that the session is yet not initialized, or the last request was retrieved from the query cache.
A hang in use db may make sense (not initialized yet), but that shouldn't hang at all.
Do you have a reproducible test case?

[leagueofsoups]

Problem reproduce only on one project on production. I tryed log all query by problem user, but get this bug https://github.com/sysown/proxysql/issues/1332
When patch code, like last comment -> segfault .. I do not have test cases now.

[leagueofsoups]

All problem connection look like this. From side proxysql "show full processlist":
5 4326 USERNAME information_schema 10.0.3.30 50484 -1 NULL NULL NULL NULL Sleep 29377 NULL NULL NULL
I have TCP ACK without TCP payload. On normal session I have "Response OK", and MySQL protocol encapsulation in TCP.

Problem reproduce on proxysql 2.0+(today on 2.0.6)

[renecannao]

To further debugging this issue, we need a reproducible test case.
Thanks

[cfrietschy]

We have the same problem. Unfortunately, the error is not reproducible. To get a traffic dump and stacktrace of the error I have to record everything and check the output of "select * from stats_mysql_processlist where hostgroup = "-1" and cli_host = "192.168.0.66";" for which socket the error occurred.

strace output:
setsockopt(726, SOL_TCP, TCP_NODELAY, [1], 4) = 0 ioctl(726, FIONBIO, [1]) = 0 sendto(726, "J\0\0\0\n5.7.26\0\242 \6\0H\1\23\20KDU%\0/\362!\2\0\217\200\25\0\0\0\0\0\0\0\0\0\0.8X\31g\taj:qt\5\0mysql_native_password\0", 78, MSG_NOSIGNAL, NULL, 0) = 78 recvfrom(726, "V\0\0\1", 4, 0, NULL, NULL) = 4 recvfrom(726, "\205\242\n\0\0\0\0\300!\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0dashboard\0\24I\225\30Q\336\360\235\257\236DDk\355\31\346\304\274\276\257\10\0mysql_native_password\0", 86, 0, NULL, NULL) = 86 sendto(726, ",\0\0\2\376mysql_native_password\0H\1\23\20KDU%.8X\31g\taj:qt\5\0", 48, MSG_NOSIGNAL, NULL, 0) = 48 recvfrom(726, "\24\0\0\3I\225\30Q\336\360\235\257\236DDk\355\31\346\304\274\276\257\10", 32768, 0, NULL, NULL) = 24 sendto(726, "\7\0\0\4\0\0\0\0\0\0\0", 11, MSG_NOSIGNAL, NULL, 0) = 11 recvfrom(726, "\3\0\0\0\33\1\0", 32768, 0, NULL, NULL) = 7

ProxySQL no longer responds after receiving the client's SET command.
In Wireshark I only see a TCP ACK without a payload, which is sent to the client.

proxysQL version 2.0.6

[cfrietschy]

After we installed version 2.0.7 the error seems to be gone.