Forgottenserver: [QUESTION] - serversave.lua

[newacc12]

Looking at the code on the file /data/globalevents/scripts/serversave.lua I can see the status is set to GAME_STATE_SHUTDOWN or GAME_STATE_CLOSED depending upon configuration.

In my server, due to being private and just having a few players, it takes less than a second. It's like you get kicked and then press enter and you're back. I'm thinking about removing the lines to change the game state and adding the saveServer(). However I don't know if there will be any side effects.

Can somebody clarify please?

Thanks

[ranisalt]

AFAIK if the game server changes to SHUTDOWN, it saves and closes the executable, meaning you need an external restarter to perform the startup operation after the server save. It's safer because the server stops running before saving so data is isolated. If it changes to CLOSED, it does saving live, just kicks and blocks player login while saving, and then reopens.

You shouldn't save the server with players online because they might change data while the saving operation is running, thus making it possible to have erratic state.

[newacc12]

@ranisalt Thanks for the input!

What you mean by change data and erratic state? Like if I'm hunting and the save runs it should save that exact second right? Are you saying that it could corrupt the data?

[pzaj2]

@newacc12

The saving operation is not atomic I suppose and "exact second" is very big time period in terms of making database updates, within that "exact second", there's plenty of things happening. For example, it's theoretically possible (I doubt it would happen though) to duplicate items that way (like one player is saved, give the item to another player, then the another player is saved - you have two items now in database). Just one example. When everyone is kicked out of the server before the save, it is to ensure that no player will make anything that may affect the game state change during the safe, so the data is in the exact state players were kicked ("exact time" here as opposed to "exact second") - I'm not sure about all the time-based things like HP/MP regeneration though, this would probably require confirmation from @ranisalt or any TFS developer.

[newacc12]

@pzaj2 Alright. Thanks for the explanation.

[K3V1N-BR]

Today I have a big database so I created this (.lua) below in globalevents to save the server every three hours, I can't lose the entire day if something crash. The problem is that now the database is increasing and the (save time) is increasing together, today it took 60 seconds and some players got kicked.

Does anyone know what I can do to optimize the code to save the server every X hours, maybe I need talk with some DBA to improve the queries or like the @ranisalt said, I shouldn't save the server with players online ?

[save.lua]
function onThink(interval)
saveServer()
return true
end

Debian 8.7
mysql Ver 14.14 Distrib 5.5.54, for debian-linux-gnu (x86_64) using readline 6.3
TFS 1.2 (Default)
Global Map

[diath]

@ranisalt

If it changes to CLOSED, it does saving live, just kicks and blocks player login while saving, and then reopens.

FYI this is unnecessary anyway because the saving function changes the state to MAINTAIN which itself prevents any logins.

[pzaj2]

@K3V1N-BR
With or without players online, I think the server save will still take approximately the same amount of time. You should read above comments and try to understand why it is highly discouraged to save server state with players online in the first place.

I also think you don't need to save game state that often, if you take a look what is saved by SaveServer() you would see that it's made up of two parts:

1. Looping over players and saving them.
2. Call to Map::save() - this one is saving house info (who owns houses, etc.) and house items.

Yes, you can lose some stuff in case of the crash, but you won't lose everything, because players are saved more often - when they logout, die, receive mail, etc. Unfortunately the only thing you could do is to optimize how data is persisted - queries, batch inserts, database schema.

The hard way would be introducing an intermediate layer, making data snapshot and saving from that. This implemented correctly would significantly limit the amount of time required (you only count time to make the snapshot) to perform the save, but also grow memory requirements significantly. I will not go into details of possible solutions here, because I think it's not the place for that to begin with. You should rather look for software developers to tell you more about that.

[K3V1N-BR]

Thank you for your attention @pzaj2 I will talk with some Dev's.

[ranisalt]

You won't get much from improving queries as TFS queries are very lightweight (in terms of database functions, it only performs trivial ones) and the worst issue is the amount of data, so you would only benefit from a complexity decrease which I think isn't possible.

[pzaj2]

@ranisalt
I may be wrong, but I think using prepared, parametrized and batched inserts would decrease save time. I'm not sure about MySQL though, but with MS SQL I've seen up to 80% improvement after switching from a single bulk insert (see depotItems) to prepared, parametrized batch inserts (concerning trivial insert / update operations). This would work especially for trivial queries, because the query would be prepared and reused and I think it can be assumed (though it should be probably also tested) that significant part of query time is used to parse and "understand" it, rather than perform the actual query. But like I said, I may be wrong on that.

Anyway, I don't think it's considered a bug and thus there's a bit of time to do more research on that matter. I will try to do more research (w/ tests of course) during next two weeks and see how this could be improved.

[ranisalt]

Parametrized inserts aren't faster, they are safer, but send data to the server twice (statement, then datas) instead of a single packed statement. They excel if you reuse the same statement, but you can just use multiple insert syntax and there's no other improvement.

Prepared statements could help, but hardly anything can be squeezed from TFS über simple queries. We only do basic insertion and selects, nothing fancy that pre-optimizing would be significant. There's no point optimizing a query that already takes a fraction of a millisecond.

Of course, a deeper research on query timing and alternatives is much appreciated. I hope you can prove me wrong and find ways to improve query times greatly 👍

[diath]

@ranisalt
A follow up comment on that part:

You shouldn't save the server with players online because they might change data while the saving operation is running, thus making it possible to have erratic state.

Don't the scheduler & dispatcher take care of that? The dispatcher executes one task at a time meaning that while the server is saving, players will not be able to perform any actions as the actions will be queued and wait until the save task finishes.