Runtime: Profiler never get DLL_PROCESS_DETACH event

[k15tfu]

Hi!

My profiler expects DllMain with DLL_PROCESS_DETACH event before shutting down to save logs etc, and on Windows it works fine but on Linux DllMain(DLL_PROCESS_DETACH) is never get called.

I tried to find the reason and noticed that EEToProfInterfaceImpl::~EEToProfInterfaceImpl() doesn't call FreeLibrary() during the shutdown process. I think the reason why it works on Windows is that Windows itself calls DllMain(DLL_PROCESS_DETACH) for all loaded modules before the process exit, but coreclr does it only in FreeLibrary().

Is it possible to unify this flow and do the same thing on Linux/Mac?

[AaronRobinsonMSFT]

The DllMain contract is a Windows concept so calling DllMain at the same logical time is going to be very tough if not impossible since Linux/Mac know nothing about this export.

All is not lost though because if I recall gcc on Linux has the concept of hooks that can be defined and will be called at similar times. I have no experience with Mac in this regard so I don't know if this is supported.

The gcc attributes of interest are __attribute__((constructor)) and __attribute__((destructor)).

See https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes.

[jkotas]

Profilers on Unix should not export or otherwise depend on DllMain.

The DllMain contract is part partial Windows emulator (aka PAL) used by CoreCLR. It does not work same way as it works on Windows and it is likely going to disappear eventually.

[k15tfu]

@AaronRobinsonMSFT

Yep, I have already tried to use __attribute__((destructor)) but it didn't help because that destructor was called after destruction of some (all?) static object, so I was unable to successfully terminate my profiler. Specifying the destructor priority also doesn't work.

I thought you can iterate over the list of loaded modules and call DllMain(DLL_PROCESS_DETACH) for each module just to notify the process will exit soon, without calling FreeLibrary(). The point is that at this point all static objects will be alive, so everything will work the same way (*from the profiler's point of view =) ).

Actually I found how to do it the other way, but what do you think about it?

[AaronRobinsonMSFT]

As @jkotas said the semantics of DllMain from the coreclr perspective are apt to disappear and as such this isn't something we are likely to support since it really requires OS loader coordination (e.g. Windows and DllMain).

Glad to hear that you have figured out a workaround, please share so others can benefit.

[k15tfu]

@jkotas @AaronRobinsonMSFT

and it is likely going to disappear eventually

It's very undesirable because DllMain() is called right before the calling of DllGetClassObject() so I have a chance to prepare for creation of the profiler.

As I said, I made a fix that "emulates" DllMain(DLL_PROCESS_DETACH) behavior in the profiler by registering atexit() callback, and it is guaranteed to work on Linux/MacOs because that callback will be called before destruction of any static objects that were constructed at the time of DllMain(DLL_PROCESS_ATTACH). So it preserves construction/destruction order (see 1.c in the docs for std::exit).

I think I also can't use __attribute__((constructor)) because there is no guarantee it will be called after all static variables would be initialized. So DllMain(DLL_PROCESS_ATTACH) give us an important point when the registered atexit() callback will be called at the right time.

So I think it does matter to call DllMain(DLL_PROCESS_ATTACH) even on Linux/MacOs.

[jkotas]

called right before the calling of DllGetClassObject() so I have a chance to prepare for creation of the profiler.

So do the preparation for creation of the profiler as the first thing in DllGetClassObject()?

[AaronRobinsonMSFT]

@k15tfu The guidance we are suggesting is premised on being platform agnostic and ensuring compatibility. CoreCLR doesn't and can't push constraints down into any OS, that would defeat the entire purpose of being a "virtual machine".

A similar consideration should also be taken by authors of native extensions to CoreCLR. In this particular example, the DllMain construct is tightly coupled to the Windows OS and as mentioned it is close to impossible to emulate the semantic behavior on non-Windows platforms without working with each any every OS platform. I think both @jkotas and I would agree that we are happy you found a work around using atexit() but that is probably not the best solution because even it has a bit of nuance and too much latitude to OS implementors on its behavior to always be predictable - see https://pubs.opengroup.org/onlinepubs/9699919799/functions/atexit.html.

I think that @jkotas's suggestion is good because it allows you full control of the scenario without having to depend on any platform semantics and instead rely on CoreCLR contracts - which we fully control and can provide operational guarantees.

[k15tfu]

@jkotas @AaronRobinsonMSFT

I was a little inaccurate, sorry. The profiler initialization is already done in DllGetClassObject. I was talking about initialization of common stuffs, that are not related to that certain profiler.

I understand your desire to get rid of Windows specifics, and that you want to draw the line between yours and theirs responsibility.

Of course DllMain is truly Windows feature and from this point of view, you are not responsible for it. But please do not forget that the concept of COM objects was being created in the world where DllMain was de facto standard, and probably this allowed not even thought about the ways for global pre-initialization.

This is quite settled concept and developers relay on it. We cannot deny the fact that many COM libraries were architectured to use all its features, including attach/detach callbacks.

In this particular example, the DllMain construct is tightly coupled to the Windows OS and as mentioned it is close to impossible to emulate the semantic behavior on non-Windows platforms without working with each any every OS platform.

I'm not talkig about emulating quite specific behaviours or even entire VM. It is only about DllMain(DLL_PROCESS_ATTACH) because you can support it for free, and you already do. I see that your FreeLibrary calls DllMain(DLL_PROCESS_DETACH), and the reason it works differently on Windows is that Windows kernel calls it on exit even if no FreeLibrary was called. I agree that it's quite specific, so wontfix.

So do the preparation for creation of the profiler as the first thing in DllGetClassObject()?

Yes, it's possible, but it looks more like a trick. DllGetClassObject can be called few times so I have to check if these stuffs are already initialized. I think it's wrong that you decided to use existed COM DLL functions except the one. They are coupled.

About atexit: Unfortunately OS does not provide such "detach" callbacks right before dlclose'ing, so the other way is to make it possible for resource owner to notify the library before unloading. Doing this on IUnknown::Release when total_ref == 0 also looks awkward because after that DllGetClassObject can still be called. So in this case, fortunately, I can use atexit, but I understand that it's a temporary solution because it will not work on profiler detach. Probably later I'll make all static objects to be pointers, that will allow me to use __attribute__((destructor)), and you might say I can use __attribute__((contructor)) the same way, but it's not elegant and is error-prone because of manual control of static objects lifetimes and their dependencies.

[AaronRobinsonMSFT]

But please do not forget that the concept of COM objects was being created in the world where DllMain was de facto standard

@k15tfu That is true, but DllMain has nothing to do with COM and adding lots of initialization in DllMain has been an anti-pattern for years. There are a limited number of safe operations that can be done since the loader lock is taken which reduces the number of functions that can be called safely. Another alternative that is actually COM and the correct way is to perform "global" initialization for your COM object is the class's IClassFactory - which can be made a singleton and thus can be global for the class.

I can appreciate the nuances of DllGetClassObject and DllMain, but the latter has nothing to do with COM initialization or activation, rather it is a general Windows dynamic library contract and something that may be able to be simulated in CoreCLR for the specific scenario we are discussing here, but there are scenarios we won't be able to satisfy and as such is considered a dead end in CoreCLR.

[k15tfu]

@AaronRobinsonMSFT So, leaving it "as is" will help in this specific case, but later someone might submit a bug report about more specific implementation behavior of DllMain on Windows. Thus to be more consistent with this, it'll be fair not to support it at all. Did I get you right?

[jkotas]

Yes, cleaning this up and not supporting this at all would be great. We have https://github.com/dotnet/coreclr/issues/21009 opened on this.

[davmason]

It looks like there is nothing actionable for this issue, closing as part of cleanup for 3.0 triage. Please reopen if I missed something.