Authmereloaded: authme is doing MySQL on server thread

[Govindass]

Before reporting an issue make sure you are running the latest build of the plugin and checked for duplicate issues!

What behaviour is observed:

What happened?
huge lag every join

What behaviour is expected:

What did you expect?
I expect authme to do MySQL connections on separate thread, yet it does it in main while useAsyncTasks is true

Steps/models to reproduce:

The actions that cause the issue

Plugin list:

This can be found by running /pl

Environment description

Standalone server/Bungeecord network, SQLite/MySql, ...
bungeecord, mysql

AuthMe build number:

This can be found by running /authme version
2385

Error Log:

Pastebin/Hastebin/Gist link of the error log or stacktrace (if any)

Configuration:

Pastebin/Hastebin/Gist link of your config.yml file (remember to delete any sensitive data)

[games647]

It should already do that. Do you have any stacktraces or errors that highlight this behavior?

[Govindass]

It should already do that. Do you have any stacktraces or errors that highlight this behavior?

here's one:
https://pastebin.com/raw/pQ2bBbDf
(the connecting IP is my mysql server)

it continued freezing main thread for 100s~ here

this is combined with your FastLogin, this is a premium player

[Govindass]

if I do /plugman unload AuthMe, the join lag is gone

[games647]

Well you can see at the log that's not AuthMe running on the server thread. It's

[21:07:59] [Paper Watchdog Thread/ERROR]: com.kirelcodes.miniaturepets.pets.Pet.(Pet.java:153)

If you mean this message:

[21:07:58] [AuthMeMYSQLPool connection adder/INFO]: [NetworkInterceptor] Intercepted outgoing connection to host '51.83.99.210'

That's normal, but there is no information that this is on the main thread.

EDIT: You can follow the stacktrace.

1. Skript running an effect task.
2. JavaScript user defined action is performed.
3. Bukkit command is executed
4. MinaturePets handles the command and performs database action.

However AuthMe could have trigger those actions after the user logged in, because AuthMe releases the player.

[Govindass]

Well you can see at the log that's not AuthMe running on the server thread. It's

[21:07:59] [Paper Watchdog Thread/ERROR]: com.kirelcodes.miniaturepets.pets.Pet.(Pet.java:153)

If you mean this message:

[21:07:58] [AuthMeMYSQLPool connection adder/INFO]: [NetworkInterceptor] Intercepted outgoing connection to host '51.83.99.210'

That's normal, but there is no information that this is on the main thread.

EDIT: You can follow the stacktrace.

1. Skript running an effect task.

2. JavaScript user defined action is performed.

3. Bukkit command is executed

4. MinaturePets handles the command and performs database action.

However AuthMe could have trigger those actions after the user logged in, because AuthMe releases the player.

I also thought that it is MiniaturePets issue at first, but now having removed AuthMe from my server has removed the lag while I am still having MiniaturePets, why'd this happen? (btw I have Skript spawning a MiniaturePets pet on server join, which gets pet name from mysql, so authme and miniaturepets do things at same time)

[games647]

The error is about MinaturePets is waiting for incoming data from the SQL server. There you can see that MinaturePets is doing SQL operations on the main thread, that should be performed asynchronous. This could affect performance. (Your title)

However you can also see that the server thread doesn't continue or there multiple tasks queued that all execute this code that will hold at the same position. AuthMe freezes the player if not logged it. This could cause Skript to queue too many effects and therefore they all report the same stacktrace. Similar there could be a loop inside MiniaturePets

The server thread freeze could also happen if there is a deadlock on the SQL server, another connection is not allowed or the MySQL/MariaDB server is overloaded. AuthMe also establishes MySQL/MariaDB connection. So please check the CPU usage and MySQL/MariaDB server settings (not the plugin settings).

Furthermore you could also try one run without MinaturePets or Skript with AuthMe enabled and observe the log and timings report. Remember Correlation does not imply causation. It's also useful in this case to post the configuration of AuthMe and your Skript plugin to debug it.

In summary (in this order):

• Please test one run without MinaturePets
• Same without Skript but with the other two plugins
• Check the MySQL/MariaDB server settings

[Govindass]

The error is about MinaturePets is waiting for incoming data from the SQL server. There you can see that MinaturePets is doing SQL operations on the main thread, that should be performed asynchronous. This could affect performance. (Your title)

However you can also see that the server thread doesn't continue or there multiple tasks queued that all execute this code that will hold at the same position. AuthMe freezes the player if not logged it. This could cause Skript to queue too many effects and therefore they all report the same stacktrace. Similar there could be a loop inside MiniaturePets

The server thread freeze could also happen if there is a deadlock on the SQL server, another connection is not allowed or the MySQL/MariaDB server is overloaded. AuthMe also establishes MySQL/MariaDB connection. So please check the CPU usage and MySQL/MariaDB server settings (not the plugin settings).

Furthermore you could also try one run without MinaturePets or Skript with AuthMe enabled and observe the log and timings report. Remember Correlation does not imply causation. It's also useful in this case to post the configuration of AuthMe and your Skript plugin to debug it.

In summary (in this order):

* Please test one run without `MinaturePets`

* Same without `Skript` but with the other two plugins

* Check the MySQL/MariaDB server settings

you may be right, it may be both authme and miniaturepets doing stuff then, it seems like also my database had downtime during the stacktraces I sent, authme itself usually delays it for few seconds. I've moved it to separate auth server now, but somehow the few second delay due to authme is gone on it today, while it was here few seconds usually earlier, same as previously on main server. but there is a difference that MySQL is now hosted on same machine as the auth server

[Govindass]

I found in my logs some stacktraces from 11th day when 5~ of my players were sent to main server at once and caused huge lagspike which was AuthMe's fault, these stacktraces do contain AuthMe a little, they also contain sockets/mysql stuff a lot, which should be coming from AuthMe
https://pastebin.com/raw/Rp3iyjzP

sorry that my previous stacktrace happened to be not informative enough - it was MiniaturePets fault here, but AuthMe is definitely also having issues

[games647]

Thanks that's what we need

[games647]

Could you please post your config. I'm trying to reproduce it.

[Govindass]

Could you please post your config. I'm trying to reproduce it.

here it is: https://pastebin.com/raw/en7uhbL5

I've removed mysql password, name, addresses
I've also removed email SMTP address/password

I am using TLS encryption for both, mysql part uses self-signed certificate, whereas email uses let's encrypt tls certificate

[games647]

Okay found it. Caching is required here. What is your motivation for disabling it?

[Govindass]

it says: "should be disabled on bungeecord environments" due to which I had disabled it, I've enabled it in my auth server yesterday.

[games647]

That's looks like a big design flaw in our system. I'll open another ticket for discussion. Looks like you found something critical. Thank you.

For now you could disable inventory protection. This should reduce the amount of calls drastically in case the player is not logged or registered and something changed in the inventory.

[Govindass]

wow I didn't even notice the ability to disable inventory protection, thank you!

[LoXR]

Might add aswell, noticed there is lag upon some players joining, and noticed there is a nice lagspike when new players join my Network. Re-occuring players dont seem to cause too much of a a lagspike but when a brand new player joins, there is a noticeable lagspike.

[LoXR]

Aswell will follow up on this, for some reason at times, players (Premium Auth) are needing to /login, But can do commands and jump. Although with that /login its suppose to prevent all that.