Trinitycore: [3.3.5] [crash] Corrupted respawnTime values in creature_respawn table ($60 bounty)

[eocrs]

This happen so far for the following creatures but not allways:

High Overlord Saurfang
Skybreaker Deckhand
The Skybreaker
Orgrim's Hammer Crew
Muradin Bronzebeard

creature_respawn table gets a weird really high corrupted value on respawnTime and server starts crashing whenever it tries to load that value.

Current behaviour:
Server crashes on this line and continues to crash whenever players log back in, for example in ICC.
srcservergameentitiescreaturecreature.cpp line 1663

Steps to reproduce the problem:
Go inside ICC Raid, kill a boss and check the value inserted in creature_respawn for respawnTime.
Probably a memory corruption, I don't know yet how to reproduce the crash.
Talking with the players, seems this occurs while starting Gunship in ICC

Branch(es): 3.3.5

TC rev. hash/commit:
Latest revision:
https://github.com/TrinityCore/TrinityCore/commit/d1146a4495d542702954702b3d6a11ddb0beb81a

Operating system: Win server 2012 R2

[ghost]

Please post the full output of the implemented command: .server debug

[ariel-]

And crashlog

[Devgasm]

This happened to me as well in Halls of Reflection, specifically related to transport maps it seems. I don't have the crash log at hand, but it can definitely be recreated easily at the last part of HoR.

[eocrs]

unknown_worldserver.exe_[15-3_22-44-20].txt

I'm skipping GIT Version but it's the one mentioned (d1146a4)

[eocrs]

I forgot to mention also the issue started after adding/updating fixes posted on 19 Feb to 13 March.
Also example of a few corrupted values in creature_respawn table:
Guid: 201255
respawnTime: 9223372036854775807
Guid: 201433
respawnTime: 9223372036854775807

[shenhuyong]

I guess 94b5d9bfa1195ad7c32cf51a665b55514dac0938

[eocrs]

@shenhuyong I reverted that fix, it still inserts huge values.
Nothing is changed, does normal respawn values for normal or elite mobs inside ICC but adds those huge values only when you kill a boss.

[topg-org]

Problem seems to be related also with 64 Byte OS, max value it can take:
m_respawnTime = std::numeric_limits::max();

Added a basic check in SaveRespawnTimeDB as bosses are saved as progress anyway so those values don't register anymore:

if (respawnTime > 1599999999)
        return;

The crash is still there, even without the huge numbers.
Even this didn't help, it made it worst, more frequent crashes:

    ASSERT(respawnTime != std::numeric_limits<time_t>::max() && respawnTime > GameTime::GetGameTime());
    // Just here for support of compatibility mode

Somehow I think is related to all temp spawns, like the ads in Gunship. The respawn value may be a result of something else getting corrupted.

[ariel-]

Please use command .server debug and copy the output here

[eocrs]

When should I run that command?
I ran in console but there's no "output" just some reload and brief info.

[ghost]

What looks like "reload" is in fact the information we are asking for.
Copy the entire output from that command and paste it here.

[eocrs]

https://image.prntscr.com/image/OyzBiI2HTZCbLTkyt2b8PA.png

[eocrs]

Anyway, even with no bugged values in DB the crash still exists and it's related to Gunship and Transport when the function Creature::LoadFromDB.
Why we get those bugged values in database is another issue. Not sure is related but it's worth taking a look at m_respawnTime = std::numeric_limits::max(); as on 64 byte the value should be huge or infinite for Bosses. If we save a boss respawn time in instance should be for a reason :)

[Gooyeth]

This report is important, stability is totally lost with this.

Crashlog: https://gist.github.com/Gooyeth/8e3e18a622f20987863635fbd2e2db83

[ariel-]

Why we get those bugged values in database is another issue. Not sure is related but it's worth taking a look at m_respawnTime = std::numeric_limits::max(); as on 64 byte the value should be huge or infinite for Bosses

The field is bigint which should save 64 bit integers

[eocrs]

As far as I know, unix timestamp fits fine in a simple INT(10) as long as it's UTF. Not sure if is really necessary to use a BIGINT(20) on 64 bit systems, it's the same unix time :)

[ghost]

We had a crash because of non-UTF characters in some SQL once.
Could someone use something other than UTF-8 and make it crash?

[diegobfl]

@Golrag @tkrokli @joschiwald @ariel-
Prioritize this error because it is impossible to ICC with the elite mobs and the Boss giving crash every moment.

[Gooyeth]

@Treeston watch this, please.

[Cyclone47]

Added a $30 bounty.

[ghost]

Link to the Bountysource page:
https://www.bountysource.com/issues/56184635-3-3-5-crash-due-corrupted-values-in-respawntime-in-creature_respawn-table

[Gooyeth]

Related #21519, is caused by 9fc6b13027d2d66a99b6bee49d3add23f2c0c8af and it is intentional.

[ghost]

It's hard to fix this problem for devs trinitycore?

[ghost]

Depends on when a developer has got time to troubleshoot the crash logs and if the developer finds out what kind of changes are needed to fix it. Remember that most of the team members and owners are real people with real lives too.

[Cyclone47]

I increased the bounty to $60. There must be someone who has time and wants to look into this?
https://www.bountysource.com/issues/56184635-3-3-5-crash-corrupted-respawntime-values-in-creature_respawn-table-30-bounty/backers

[krullgor]

the really high respawn time is intended for bosses: https://github.com/TrinityCore/TrinityCore/blob/3.3.5/src/server/game/Entities/Creature/Creature.cpp#L1964
the problem is transports forcing respawn of all static passengers when they are activated https://github.com/TrinityCore/TrinityCore/blob/3.3.5/src/server/game/Entities/Transport/Transport.cpp#L539

the passengers are currently in default spawn group so the assert triggers if they have respawn timer of previous gunship attempt

to fix the crash you can add all static passengers on transports to the legacy spawn group:

insert into spawn_group
-- groupid, spawntype, spawnid
select 1, 0, guid
from creature
where map in (
    -- 15 = motransports, data6 = mapId
    select distinct Data6
    from gameobject_template
    where type = 15 and Data6 != 0);

[Shauren]

This is a fine workaround but I need to fix gunship script to work with non-legacy spawn group

[Foereaper]

@Shauren The same crash also happens with other transport maps, like the gunship at the end of Halls of Reflection and (possibly?) all generic transport creatures like zeppelines and boats, I seem to remember having seen the latter in a crash dump but I could be wrong. HoR however is definitely causing the same crash.

[ghost]

Confirm
halls of reflection
i tested by gm mod and killed fast all sum creatures by lich but the last map lich killed jaina !

jaina cant open last object... ice berg

a

player can kill lich and server will to crash on halls of reflection

a

So player exit or login this instance server will crashed.

It's also a problem in halls of reflection!

will to closed and disable instance :disappointed:

[jackpoz]

@irgc try the sql at https://github.com/TrinityCore/TrinityCore/issues/21641#issuecomment-380243162 , it could be an acceptable workaround to push that sql script while waiting for a proper fix as @Shauren wrote

[Shauren]

Test after a5df77a9d6f3e48f2c4e8a61d60e3b5f2f0c326e

[Killyana]

We need feedback please

[Foereaper]

Will test it either tomorrow or on monday

[Aokromes]

No answer after 16d i suppose this is fixed already.