Protobuf: SharedDtor crash

[jswigart]

I have protobuf 2.6.1 in a static library that multiple dlls use.

It seems that if you allocate a protobuf object from one dll, string fields will be initialized with an 'empty string' value unique to that DLL, so that if code from the other dll causes the destruction of the proto object, the SharedDtor() comparisons against GetEmptyStringAlreadyInited() will be different than where it was allocated, causing the deletion of a string pointer that is referenced by all 'empty string' fields, and thus a crash when it tries to delete it again(since they all point to the GetEmptyStringAlreadyInited() in the dll that created it)

[hawkchch]

I also encountered the same problem. may I ask how to solve it

[xfxyjwf]

The use of globals in protobuf makes it hard to work well when linked into multiple dlls. You probably want to avoid exposing any protobuf data structures through DLL interfaces so every DLL uses its local version of symbols.

[hawkchch]

Yes, I am using local symbols to instead of the protolcol buffer symbols through DLL interfaces...it is not so convenient..

[KjellKod]

I have also encountered this problem.

[xuanvinh2005]

Don't know why this one was closed, but I have encountered same issue

[xfxyjwf]

Passing protos across DLL interfaces cannot be supported due to the use of global data structures in protobuf so I closed this issue as "wontfix".

[xuanvinh2005]

in my app, I have one dll (A) it loads another dll (B). I only use protobuf in (B). For somehow, it crashed in SharedDtor (not always) when it went out the function where I called protobuf stuff.

[AlexDovgan]

So, if i need use protobuf across diferent modules in my application what is recomended solution? Build as dll?

[sillykelvin]

I also encountered the same problem, I built the protos as a static library and linked against my app, it sometimes crashed in SharedDtor (not always), my app is single threaded, and the protobuf I used is v2.6.1, the stack:

*** Error in `./console': free(): invalid pointer: 0x0000000000aa39d0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f9f85c067e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7f9f85c0ee0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f9f85c1298c]
./console(_ZN38MONITOR_CMD_REQ_RuntimeConfig_Property10SharedDtorEv+0x39)[0x667a59]
./console(_ZN38MONITOR_CMD_REQ_RuntimeConfig_PropertyD1Ev+0x2a)[0x6679c6]
./console(_ZN38MONITOR_CMD_REQ_RuntimeConfig_PropertyD0Ev+0x18)[0x667a12]
./console(_ZN6google8protobuf8internal18GenericTypeHandlerI38MONITOR_CMD_REQ_RuntimeConfig_PropertyE6DeleteEPS3_+0x2a)[0x67aecc]
./console(_ZN6google8protobuf8internal20RepeatedPtrFieldBase7DestroyINS0_16RepeatedPtrFieldI38MONITOR_CMD_REQ_RuntimeConfig_PropertyE11TypeHandlerEEEvv+0x46)[0x67a4e6]
./console(_ZN6google8protobuf16RepeatedPtrFieldI38MONITOR_CMD_REQ_RuntimeConfig_PropertyED2Ev+0x18)[0x677b72]
./console(_ZN42MONITOR_CMD_REQ_RuntimeConfig_GlobalConfigD1Ev+0x3a)[0x668b46]
./console(_ZN42MONITOR_CMD_REQ_RuntimeConfig_GlobalConfigD0Ev+0x18)[0x668b92]
./console(_ZN6google8protobuf8internal18GenericTypeHandlerI42MONITOR_CMD_REQ_RuntimeConfig_GlobalConfigE6DeleteEPS3_+0x2a)[0x67aff2]
./console(_ZN6google8protobuf8internal20RepeatedPtrFieldBase7DestroyINS0_16RepeatedPtrFieldI42MONITOR_CMD_REQ_RuntimeConfig_GlobalConfigE11TypeHandlerEEEvv+0x46)[0x67aa4e]
./console(_ZN6google8protobuf16RepeatedPtrFieldI42MONITOR_CMD_REQ_RuntimeConfig_GlobalConfigED1Ev+0x18)[0x678130]
./console(_ZN29MONITOR_CMD_REQ_RuntimeConfigD1Ev+0x3a)[0x66de52]
./console(_ZN29MONITOR_CMD_REQ_RuntimeConfigD0Ev+0x18)[0x66deae]
./console(_ZN32MONITOR_CMD_REQ_DeploymentConfig10SharedDtorEv+0xa4)[0x66f116]
./console(_ZN32MONITOR_CMD_REQ_DeploymentConfigD1Ev+0x2a)[0x66f018]
./console(_ZN32MONITOR_CMD_REQ_DeploymentConfigD0Ev+0x18)[0x66f064]
./console(_ZN15MONITOR_CMD_REQ10SharedDtorEv+0x4c)[0x671c1c]
./console(_ZN15MONITOR_CMD_REQD1Ev+0x2a)[0x671b76]
./console(_ZN15console_sessionD1Ev+0x3a)[0x5bb1e2]

the last frame it crashed:

void MONITOR_CMD_REQ_RuntimeConfig_Property::SharedDtor() {
  if (name_ != &::google::protobuf::internal::GetEmptyStringAlreadyInited()) {
    delete name_;        // <--- it crashed here, seems to be a double free?
  }
  if (value_ != &::google::protobuf::internal::GetEmptyStringAlreadyInited()) {
    delete value_;
  }
  if (this != default_instance_) {
  }
}

[xfxyjwf]

@kelvinh Do you have a simple program that can reproduce the problem? I can't help you much given just a stack trace because a stack trace like this usually indicates improper usage of protobuf API or memory corruption bug unrelated to protobuf.

[sillykelvin]

@xfxyjwf Sorry that my app is not a simple program, and there is no proper way to simplify it to make a demo here. And this problem is not easy to reproduce, about one crash in 200 times of running (not accurate, just a feeling). However, I will keep debugging and will update here if I finally find something.

[okapies]

I succeed to reproduce this issue in https://github.com/okapies/protobuf-shareddtor-crash. It becomes a problem when a library embed and hide both protobuf and generated models behind the library's ABI to improve platform compatibility of our binary, especially in libstdc++ ABI (protobuf has no C API and its C++ API expose std::* types).

It may be a problem if our library want to pass back the protobuf value to an user. The user need to link his/her program to ~.pb.cc and libprotobuf.so but SharedDtor() goes wrong in that situation.

[yejw5]

Passing protos across DLL interfaces cannot be supported due to the use of global data structures in protobuf so I closed this issue as "wontfix".

@xfxyjwf I'm confused that why the use of global data structures in protobuf causing protos across dynamic link library cannot be supported? Could you kindly explain it? Many thanks!

[thomasstep]

One way that I fixed this issue in my program was to make all of the parameters of the RPC I was invoking pointers (in C++ of course). For instance instead of using Request myRequest; I used Request* myRequest = new(Request);. Once I made everything work around the pointers instead of actual instances, then I did not have any more problems.

[bmcdorman]

I'm believe I'm having this same issue on macOS with an application that uses a plugin architecture and an InProcessChannel for message passing between plugins. This issue appears to arise when storing protobuf messages for longer than the duration of the RPC call, though I have some evidence to indicate that isn't the true issue.

@thomasstep Are you calling delete on those messages after allocating them on the heap? The problem probably still exists.

[thomasstep]

@bmcdorman Yes, I am manually creating and deleting those messages. It is painstaking and tedious but it seems to be the fastest solution for me.

[Rasie1]

@xfxyjwf Why is this closed and marked as wontfix? There are a lot of tools with hot reloading (i.e. Unreal Engine 4) that make use of multiple dlls.
This is serious bug and it should be fixed