Runtime: Garbage Collection Thread is blocked waiting for another thread for 10 seconds or more.

Created on 15 Nov 2020 · 18Comments · Source: dotnet/runtime

Description

Performance issues observed in a web application when one of API end point starting responding slow due to slow file share dependency. But it triggered unexpected behavior which we have not seen before, i.e. 503, time out (Redis) exceptions & intermittent hangs. Primarily due to GC Thread is blocked by another thread.

Configuration

Web Application hosted on Windows Server 2016 using IIS
Targeting Framework 4.7.1
Server GC (Output of !eeheap 4.8.4180.0 free Server mode with 8 gc heaps)
Host has 8 cores, and 16 GB Memory, Process memory max utilization is around 4 GB. There are other .NET Framework process running on host, and did not observe any issues.

Analysis

To do further analysis, I took to two memory dumps (10 seconds apart) on one of the host for the affected process. Below command was used.
~
procdump -ma -s 10 -n 2 -64 -accepteula process-id
~

DebugDiag Analysis Report

DebugDiag produces the output, seen below in screen shot, as almost every thread in the process is waiting on Thread 41 to finish its work. Stack trace of Thread 41 is show below.
About 24% threads are waiting in Monitor.Wait are actually waiting to re-acquire a lock which they released, which I do not understand how. It is related to StackExchange.Redis library, I should file issue in that repo, but I would love to get more information as I feel like it can be related.

Stack Trace of Thread 41 - Which happens to be doing the work for APM Agent.

Complete CLR STACK of Thread 41 - via WinDbg
I can not make much sense of it, does this tell us that it blocked GC thread but why/how?
thirdparty-APM is dll of third party APM platform.
~
Thread 41
Current frame: ntdll!NtGetContextThread+0x14
Child-SP RetAddr Caller, Callee
000000f5a067e870 00007ffaf8a0f97a KERNELBASE!GetThreadContext+0xa, calling ntdll!NtGetContextThread
000000f5a067e8a0 00007ffaeda429d5 clr!Thread::SuspendThread+0x112, calling kernel32!GetThreadContextStub
000000f5a067e8d0 00007ffaf89e8d23 KERNELBASE!WaitForMultipleObjectsEx+0x1f3, calling ntdll!RtlActivateActivationContextUnsafeFast
000000f5a067e990 00007ffad3924cdf thirdparty-APM!updateManagedThreadPoolMetrics+0x3cf32f, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x3cf2d0
000000f5a067e9e0 00007ffaf9ef3889 SYMSRV: BYINDEX: 0x10
c:\symcache*http://msdl.microsoft.com/download/symbols
combase.pdb
A9A6647FB7664D5CAE48BB37C377F3801
SYMSRV: PATH: c:\symcache\combase.pdb\A9A6647FB7664D5CAE48BB37C377F3801\combase.pdb
SYMSRV: RESULT: 0x00000000
DBGHELP: combase - private symbols & lines
c:\symcache\combase.pdb\A9A6647FB7664D5CAE48BB37C377F3801\combase.pdb
combase!CObjectContext::QIHelper+0x109, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067ea10 00007ffaedbd5553 clr!SafeQueryInterface+0xe3
000000f5a067ea50 00007ffaedbc8977 clr!SafeReleasePreemp+0x87
000000f5a067ea90 00007ffaf9ef277c combase!CObjectContext::GetCurrentThreadType+0x1c, calling combase!IsApartmentInitialized
000000f5a067eac0 00007ffaedbca34b clr!GetCurrentThreadTypeNT5+0xcd, calling clr!SafeReleasePreemp
000000f5a067eae0 00007ffafc4b81a0 ntdll!RtlSetLastWin32Error+0x40, calling ntdll!_security_check_cookie
000000f5a067eb30 00007ffaedbd2928 clr!WaitForMultipleObjectsEx_SO_TOLERANT+0x99, calling kernel32!SetLastErrorStub
000000f5a067eb50 00007ffaeda1bf50 clr!ThreadStateNCStackHolder::~ThreadStateNCStackHolder+0x1c, calling clr!GetThread
000000f5a067eb60 00007ffaed9fb506 clr!Thread::GetFinalApartment+0x1a, calling clr!Thread::GetApartment
000000f5a067eb90 00007ffaedbd279d clr!Thread::DoAppropriateWaitWorker+0x26c, calling clr!ThreadStateNCStackHolder::~ThreadStateNCStackHolder
000000f5a067ebe0 00007ffaed9f6a6d clr!SafeHandle::Release+0xa5, calling clr!_security_check_cookie
000000f5a067ec90 00007ffaed9f563e clr!GCCoop::GCCoop+0xe, calling clr!GetThread
000000f5a067ecc0 00007ffaeda35c8f clr!ReleaseSafeHandle+0x27, calling clr!SafeHandle::Release
000000f5a067ed10 00007ffaeda35c2c clr!WaitHandleNative::CorWaitOneNative+0x1ec, calling clr!HelperMethodFrameRestoreState
000000f5a067ed90 00007ffaf89f255a KERNELBASE!ResetEvent+0xa, calling ntdll!NtClearEvent
000000f5a067eda0 00007ffaed9f5133 clr!CrstBase::Enter+0x7f, calling ntdll!RtlTryEnterCriticalSection
000000f5a067edc0 00007ffaf89f255a KERNELBASE!ResetEvent+0xa, calling ntdll!NtClearEvent
000000f5a067edf0 00007ffaeda39759 clr!CLREventBase::Reset+0x29, calling kernel32!ResetEvent
000000f5a067ee20 00007ffaede7227b clr!Thread::UserSuspendThread+0xc1, calling clr!Thread::SuspendThread
000000f5a067ee60 00007ffaedf9288e clr!ThreadIsRunning+0x31, calling clr!Thread::HasValidThreadHandle
000000f5a067ee90 00007ffaedf927a6 clr!ThreadNative::Suspend+0x116, calling clr!Thread::UserSuspendThread
000000f5a067eec0 00007ffaedb7f800 clr!ThreadNative::GetThreadState+0x1e0, calling clr!HelperMethodFrameRestoreState
000000f5a067eed0 00007ffaed9f4e65 clr!JIT_InitPInvokeFrame+0x15, calling clr!GetThread
000000f5a067efb0 00007ffa8ed6aa71 (MethodDesc 00007ffa8e842838 +0x11 System.Threading.WaitHandle.WaitOne(System.TimeSpan))
000000f5a067efd0 00007ffa8ed69507 (MethodDesc 00007ffa8e959bb8 +0x27 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e696460 +0 mscorlib.dll!Unknown)
000000f5a067efe8 00007ffaedf92717 clr!ThreadNative::Suspend+0x87, calling clr!LazyMachStateCaptureState
000000f5a067f000 00007ffa8ed6ac5b (MethodDesc 00007ffa8e964740 +0xb mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e5f84d8 +0 mscorlib.dll!Unknown)
000000f5a067f030 00007ffa8f306720 (MethodDesc 00007ffa8e959a58 +0x70 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e964790 +0 mscorlib.dll!Unknown)
000000f5a067f050 00007ffa8f2fcbd8 (MethodDesc 00007ffa8e959a48 +0x98 mscorlib.dll!Unknown), calling 00007ffa8e454740
000000f5a067f090 00007ffa8e9d109a (MethodDesc 00007ffa8e959a38 +0x46a mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e959a58 +0 mscorlib.dll!Unknown)
000000f5a067f0b0 00007ffad3baa845 thirdparty-APM!updateManagedThreadPoolMetrics+0x654e95, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x6511d0
000000f5a067f1a0 00007ffa8e9ced52 (MethodDesc 00007ffa8e699118 +0x2c2 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e959a38 +0 mscorlib.dll!Unknown)
000000f5a067f210 00007ffa8e9ce686 (MethodDesc 00007ffa8ea48300 +0x46 DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int32, Int64, Int32))
000000f5a067f230 00007ffaed9f83fe clr!SystemDomain::GetAppDomainAtId+0x43, calling clr!AppDomain::CanThreadEnter
000000f5a067f270 00007ffaed9f2483 clr!UM2MThunk_WrapperHelper+0x43
000000f5a067f290 00007ffaed9fa54c clr!Thread::EnterContextRestricted+0x19c, calling clr!ClrFlsIncrementValue
000000f5a067f2b0 00007ffaedb7c16a clr!UM2MThunk_Wrapper+0x60, calling clr!UM2MThunk_WrapperHelper
000000f5a067f2d0 00007ffaed9f83fe clr!SystemDomain::GetAppDomainAtId+0x43, calling clr!AppDomain::CanThreadEnter
000000f5a067f300 00007ffaed9fa9d0 clr!Thread::DoADCallBack+0x136
000000f5a067f320 00007ffad356e4a2 thirdparty-APM!updateManagedThreadPoolMetrics+0x18af2, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f330 00007ffa8e9dac5c (MethodDesc 00007ffa8e93ebe8 +0xc mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e93ec28 +0 mscorlib.dll!Unknown)
000000f5a067f360 00007ffa8e9d132b (MethodDesc 00007ffa8e959a38 +0x6fb mscorlib.dll!Unknown), calling thirdparty-APM!updateManagedThreadPoolMetrics+0x18ab0
000000f5a067f3c8 00007ffa8e9d132b (MethodDesc 00007ffa8e959a38 +0x6fb mscorlib.dll!Unknown), calling thirdparty-APM!updateManagedThreadPoolMetrics+0x18ab0
000000f5a067f470 00007ffa8e9cefa6 (MethodDesc 00007ffa8e699118 +0x516 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e699118 +0x655 mscorlib.dll!Unknown)
000000f5a067f4c0 00007ffaedb7c0e4 clr!UM2MDoADCallBack+0xb3, calling clr!Thread::DoADCallBack
000000f5a067f500 00007ffad3eee0e4 thirdparty-APM!updateManagedThreadPoolMetrics+0x998734, calling kernel32!HeapFreeStub
000000f5a067f540 00007ffaed9f242d clr!UMThunkStub+0x26d, calling clr!UM2MDoADCallBack
000000f5a067f5d0 00007ffad36014e8 thirdparty-APM!updateManagedThreadPoolMetrics+0xabb38, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f610 00007ffad360165e thirdparty-APM!updateManagedThreadPoolMetrics+0xabcae, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xabb10
000000f5a067f6d0 00007ffad36370f7 thirdparty-APM!updateManagedThreadPoolMetrics+0xe1747, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xe1020
000000f5a067f700 00007ffad36026b2 thirdparty-APM!updateManagedThreadPoolMetrics+0xacd02, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xabb50
000000f5a067f7e0 00007ffad35b5c20 thirdparty-APM!updateManagedThreadPoolMetrics+0x60270, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f8a0 00007ffad358f467 thirdparty-APM!updateManagedThreadPoolMetrics+0x39ab7, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f900 00007ffafc4b81a0 ntdll!RtlSetLastWin32Error+0x40, calling ntdll!_security_check_cookie
000000f5a067f920 00007ffad3ef1933 thirdparty-APM!updateManagedThreadPoolMetrics+0x99bf83, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x99b750
000000f5a067f950 00007ffad3c627e6 thirdparty-APM!updateManagedThreadPoolMetrics+0x70ce36, calling kernel32!TlsSetValueStub
000000f5a067f980 00007ffad3ebe880 thirdparty-APM!updateManagedThreadPoolMetrics+0x968ed0, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f9b0 00007ffafa4b84d4 kernel32!BaseThreadInitThunk+0x14, calling ntdll!LdrpDispatchUserCallTarget
000000f5a067f9e0 00007ffafc51e871 ntdll!RtlUserThreadStart+0x21, calling ntdll!LdrpDispatchUserCallTarget
~

WinDbg Analysis

I am newbie to WinDbg. But I was able to verify some basics diagnostics to rule out. But there are lot interesting details which I do not fully understand.

!VerifyHeap - No Corruption Found
!GCHandleLeaks - No Handle Leak Detected
!dlk No Dead Lock Detected - using sosex.dll extension
No Out of Memory Exception
Note: Below are details from one of the memory dump, other memory dump also report very similar stats.

Thread Pool Details

~~~
CPU utilization: 6%
Worker Thread: Total: 496 Running: 165 Idle: 331 MaxLimit: 1000 MinLimit: 496
Work Request in Queue: 141
Unknown Function: 00007ffae3b519d0 Context: 00000296f1755900
Unknown Function: 00007ffae3b519d0 Context: 00000296fd9b7718
Unknown Function: 00007ffae3b519d0 Context: 00000296f175a130

---removed entries

Number of Timers: 0

Completion Port Thread:Total: 2 Free: 2 MaxFree: 16 CurrentLimit: 2 MaxLimit: 1000 MinLimit: 496
~~~

GC Handles

~~~
Total 57233 objects

Handles:
Strong Handles: 8789
Pinned Handles: 49
Async Pinned Handles: 59
Ref Count Handles: 11
Weak Long Handles: 15004
Weak Short Handles: 33241
SizedRef Handles: 80
~~~
DotMemory Finalizable Objects - Which seems to match GC Handles - Is it because GC thread is blocked?

!Threads

I tried to see what Thread 41 Stack Trace can tell me, but I can not make much sense of it?

!dumpheap -live -thinlock

Note: All thin lock objects of same size which 24. Which is free object according to Maoni Stephens "You can not lock a free object."
I am using WinDbg Preview - Debugger engine version: 10.0.20153.1000 Debugger client version: 1.0.2007.06001
~
removed ...
00000295c8a7c530 0000029241e30610 24 ThinLock owner 2c (00000296d1268f60) Recursive 0
00000295c8a7c918 0000029241e30610 24 ThinLock owner 6e (00000296d59d2c70) Recursive 0
00000295c8a7cd90 0000029241e30610 24 ThinLock owner 6e (00000296d59d2c70) Recursive 0
Found 7190 objects.
~

0:041> !SyncBlk

Note: This also produces interesting, I believe these are coming from Bundling, but I am failing to conclude that this is the root cause? Help
~~~
Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock Owner
735 00000296ecbb30b8 1 1 00000296d15566c0 1370 227 0000029245808468 System.Web.DirectoryMonitor
1125 00000296e8630bc8 3 1 00000296d64f1780 e70 207 00000293c4548c38 log4net.Appender.RollingFileAppender
1852 00000296eca2ed78 3 1 00000296ed3fd5f0 2a04 424 000002944afd2168 System.Web.CachedPathData
2293 00000296ec4857d8 11 1 00000296ed4089a0 904 448 0000029547c603c8 System.Web.CachedPathData
2414 00000296d7ee8398 15 1 00000296ecad0580 2bd4 384 00000294c9f64768 System.Web.CachedPathData
2869 00000296f3e8be88 5 1 00000296ecaa1780 1704 224 00000292c8cd8838 System.Web.CachedPathData

2999 00000296e37ad368 21 1 00000296e3a37770 3a68 442 00000295c862d098 System.Web.CachedPathData

Total 3099
CCW 11
RCW 12
ComClassFactory 0
Free 921
~~~
I will appreciate any help in that regard to find the root cause. Kindly let me know if you need more information or what further analysis can be done on these memory dumps. Since latency, 503, Redis Timeout errors were observed only when File Share was responding slow, so I have not done further traces on production.

area-VM-coreclr question tenet-performance

Source

marafiq

👍1

Most helpful comment

could you please take a look at the first callstack on this issue? my guess is clr!JIT_InitPInvokeFrame is perhaps trying to suspend the thread because it observes suspension in progress and isn't successfully suspending thread?

@Maoni0 Yes, I believe that is the case. The managed function System.Threading.WaitHandle.WaitOne is making an FCall (no automatic GC transition) to WaitHandleNative::CorWaitOneNative and seems to be trying to transition but then is interrupted in its attempt and starts over again. I _think_ this is possibly occurring due to some sampling mechanism from the third party tool. The fact that this manifests as it does would indicate a classic GC starvation issue - eventually succeeds but after many attempts.

Alternatively, disable CPU profiling altogether (to verify) (Dynatrace > "Settings" > "Server-Side monitoring" > "Deep monitoring" > "Troubleshooting" > "Capture method hotspot information in PurePaths: Off")

That would be my first avenue for investigation.

AaronRobinsonMSFT on 17 Nov 2020

👍2

All 18 comments

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

Description:	Description Performance issues observed in a web application when one of API end point starting responding slow due to slow file share dependency. But it triggered unexpected behavior which we have not seen before, i.e. 503, time out (Redis) exceptions & intermittent hangs. Primarily due to GC Thread is blocked by another thread. Configuration Web Application hosted on Windows Server 2016 using IIS Targeting Framework 4.7.1 Server GC (Output of !eeheap 4.8.4180.0 free Server mode with 8 gc heaps) Host has 8 cores, and 16 GB Memory, Process memory max utilization is around 4 GB. There are other .NET Framework process running on host, and did not observe any issues. Analysis To do further analysis, I took to two memory dumps (10 seconds apart) on one of the host for the affected process. Below command was used. ~ procdump -ma -s 10 -n 2 -64 -accepteula process-id ~ DebugDiag Analysis Report DebugDiag produces the output, seen below in screen shot, as almost every thread in the process is waiting on Thread 41 to finish its work. Stack trace of Thread 41 is show below. About 24% threads are waiting in Monitor.Wait are actually waiting to re-acquire a lock which they released, which I do not understand how. It is related to StackExchange.Redis library, I should file issue in that repo, but I would love to get more information as I feel like it can be related. Stack Trace of Thread 41 - Which happens to be doing the work for APM Agent. Complete CLR STACK of Thread 41 - via WinDbg I can not make much sense of it, does this tell us that it blocked GC thread but why/how? thirdparty-APM is dll of third party APM platform. ~ Thread 41 Current frame: ntdll!NtGetContextThread+0x14 Child-SP RetAddr Caller, Callee 000000f5a067e870 00007ffaf8a0f97a KERNELBASE!GetThreadContext+0xa, calling ntdll!NtGetContextThread 000000f5a067e8a0 00007ffaeda429d5 clr!Thread::SuspendThread+0x112, calling kernel32!GetThreadContextStub 000000f5a067e8d0 00007ffaf89e8d23 KERNELBASE!WaitForMultipleObjectsEx+0x1f3, calling ntdll!RtlActivateActivationContextUnsafeFast 000000f5a067e990 00007ffad3924cdf thirdparty-APM!updateManagedThreadPoolMetrics+0x3cf32f, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x3cf2d0 000000f5a067e9e0 00007ffaf9ef3889 SYMSRV: BYINDEX: 0x10 c:\symcachehttp://msdl.microsoft.com/download/symbols combase.pdb A9A6647FB7664D5CAE48BB37C377F3801 SYMSRV: PATH: c:\symcache\combase.pdb\A9A6647FB7664D5CAE48BB37C377F3801\combase.pdb SYMSRV: RESULT: 0x00000000 DBGHELP: combase - private symbols & lines c:\symcache\combase.pdb\A9A6647FB7664D5CAE48BB37C377F3801\combase.pdb combase!CObjectContext::QIHelper+0x109, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067ea10 00007ffaedbd5553 clr!SafeQueryInterface+0xe3 000000f5a067ea50 00007ffaedbc8977 clr!SafeReleasePreemp+0x87 000000f5a067ea90 00007ffaf9ef277c combase!CObjectContext::GetCurrentThreadType+0x1c, calling combase!IsApartmentInitialized 000000f5a067eac0 00007ffaedbca34b clr!GetCurrentThreadTypeNT5+0xcd, calling clr!SafeReleasePreemp 000000f5a067eae0 00007ffafc4b81a0 ntdll!RtlSetLastWin32Error+0x40, calling ntdll!_security_check_cookie 000000f5a067eb30 00007ffaedbd2928 clr!WaitForMultipleObjectsEx_SO_TOLERANT+0x99, calling kernel32!SetLastErrorStub 000000f5a067eb50 00007ffaeda1bf50 clr!ThreadStateNCStackHolder::~ThreadStateNCStackHolder+0x1c, calling clr!GetThread 000000f5a067eb60 00007ffaed9fb506 clr!Thread::GetFinalApartment+0x1a, calling clr!Thread::GetApartment 000000f5a067eb90 00007ffaedbd279d clr!Thread::DoAppropriateWaitWorker+0x26c, calling clr!ThreadStateNCStackHolder::~ThreadStateNCStackHolder 000000f5a067ebe0 00007ffaed9f6a6d clr!SafeHandle::Release+0xa5, calling clr!_security_check_cookie 000000f5a067ec90 00007ffaed9f563e clr!GCCoop::GCCoop+0xe, calling clr!GetThread 000000f5a067ecc0 00007ffaeda35c8f clr!ReleaseSafeHandle+0x27, calling clr!SafeHandle::Release 000000f5a067ed10 00007ffaeda35c2c clr!WaitHandleNative::CorWaitOneNative+0x1ec, calling clr!HelperMethodFrameRestoreState 000000f5a067ed90 00007ffaf89f255a KERNELBASE!ResetEvent+0xa, calling ntdll!NtClearEvent 000000f5a067eda0 00007ffaed9f5133 clr!CrstBase::Enter+0x7f, calling ntdll!RtlTryEnterCriticalSection 000000f5a067edc0 00007ffaf89f255a KERNELBASE!ResetEvent+0xa, calling ntdll!NtClearEvent 000000f5a067edf0 00007ffaeda39759 clr!CLREventBase::Reset+0x29, calling kernel32!ResetEvent 000000f5a067ee20 00007ffaede7227b clr!Thread::UserSuspendThread+0xc1, calling clr!Thread::SuspendThread 000000f5a067ee60 00007ffaedf9288e clr!ThreadIsRunning+0x31, calling clr!Thread::HasValidThreadHandle 000000f5a067ee90 00007ffaedf927a6 clr!ThreadNative::Suspend+0x116, calling clr!Thread::UserSuspendThread 000000f5a067eec0 00007ffaedb7f800 clr!ThreadNative::GetThreadState+0x1e0, calling clr!HelperMethodFrameRestoreState 000000f5a067eed0 00007ffaed9f4e65 clr!JIT_InitPInvokeFrame+0x15, calling clr!GetThread 000000f5a067efb0 00007ffa8ed6aa71 (MethodDesc 00007ffa8e842838 +0x11 System.Threading.WaitHandle.WaitOne(System.TimeSpan)) 000000f5a067efd0 00007ffa8ed69507 (MethodDesc 00007ffa8e959bb8 +0x27 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e696460 +0 mscorlib.dll!Unknown) 000000f5a067efe8 00007ffaedf92717 clr!ThreadNative::Suspend+0x87, calling clr!LazyMachStateCaptureState 000000f5a067f000 00007ffa8ed6ac5b (MethodDesc 00007ffa8e964740 +0xb mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e5f84d8 +0 mscorlib.dll!Unknown) 000000f5a067f030 00007ffa8f306720 (MethodDesc 00007ffa8e959a58 +0x70 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e964790 +0 mscorlib.dll!Unknown) 000000f5a067f050 00007ffa8f2fcbd8 (MethodDesc 00007ffa8e959a48 +0x98 mscorlib.dll!Unknown), calling 00007ffa8e454740 000000f5a067f090 00007ffa8e9d109a (MethodDesc 00007ffa8e959a38 +0x46a mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e959a58 +0 mscorlib.dll!Unknown) 000000f5a067f0b0 00007ffad3baa845 thirdparty-APM!updateManagedThreadPoolMetrics+0x654e95, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x6511d0 000000f5a067f1a0 00007ffa8e9ced52 (MethodDesc 00007ffa8e699118 +0x2c2 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e959a38 +0 mscorlib.dll!Unknown) 000000f5a067f210 00007ffa8e9ce686 (MethodDesc 00007ffa8ea48300 +0x46 DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int32, Int64, Int32)) 000000f5a067f230 00007ffaed9f83fe clr!SystemDomain::GetAppDomainAtId+0x43, calling clr!AppDomain::CanThreadEnter 000000f5a067f270 00007ffaed9f2483 clr!UM2MThunk_WrapperHelper+0x43 000000f5a067f290 00007ffaed9fa54c clr!Thread::EnterContextRestricted+0x19c, calling clr!ClrFlsIncrementValue 000000f5a067f2b0 00007ffaedb7c16a clr!UM2MThunk_Wrapper+0x60, calling clr!UM2MThunk_WrapperHelper 000000f5a067f2d0 00007ffaed9f83fe clr!SystemDomain::GetAppDomainAtId+0x43, calling clr!AppDomain::CanThreadEnter 000000f5a067f300 00007ffaed9fa9d0 clr!Thread::DoADCallBack+0x136 000000f5a067f320 00007ffad356e4a2 thirdparty-APM!updateManagedThreadPoolMetrics+0x18af2, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f330 00007ffa8e9dac5c (MethodDesc 00007ffa8e93ebe8 +0xc mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e93ec28 +0 mscorlib.dll!Unknown) 000000f5a067f360 00007ffa8e9d132b (MethodDesc 00007ffa8e959a38 +0x6fb mscorlib.dll!Unknown), calling thirdparty-APM!updateManagedThreadPoolMetrics+0x18ab0 000000f5a067f3c8 00007ffa8e9d132b (MethodDesc 00007ffa8e959a38 +0x6fb mscorlib.dll!Unknown), calling thirdparty-APM!updateManagedThreadPoolMetrics+0x18ab0 000000f5a067f470 00007ffa8e9cefa6 (MethodDesc 00007ffa8e699118 +0x516 mscorlib.dll!Unknown), calling (MethodDesc 00007ffa8e699118 +0x655 mscorlib.dll!Unknown) 000000f5a067f4c0 00007ffaedb7c0e4 clr!UM2MDoADCallBack+0xb3, calling clr!Thread::DoADCallBack 000000f5a067f500 00007ffad3eee0e4 thirdparty-APM!updateManagedThreadPoolMetrics+0x998734, calling kernel32!HeapFreeStub 000000f5a067f540 00007ffaed9f242d clr!UMThunkStub+0x26d, calling clr!UM2MDoADCallBack 000000f5a067f5d0 00007ffad36014e8 thirdparty-APM!updateManagedThreadPoolMetrics+0xabb38, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f610 00007ffad360165e thirdparty-APM!updateManagedThreadPoolMetrics+0xabcae, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xabb10 000000f5a067f6d0 00007ffad36370f7 thirdparty-APM!updateManagedThreadPoolMetrics+0xe1747, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xe1020 000000f5a067f700 00007ffad36026b2 thirdparty-APM!updateManagedThreadPoolMetrics+0xacd02, calling thirdparty-APM!updateManagedThreadPoolMetrics+0xabb50 000000f5a067f7e0 00007ffad35b5c20 thirdparty-APM!updateManagedThreadPoolMetrics+0x60270, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f8a0 00007ffad358f467 thirdparty-APM!updateManagedThreadPoolMetrics+0x39ab7, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f900 00007ffafc4b81a0 ntdll!RtlSetLastWin32Error+0x40, calling ntdll!_security_check_cookie 000000f5a067f920 00007ffad3ef1933 thirdparty-APM!updateManagedThreadPoolMetrics+0x99bf83, calling thirdparty-APM!updateManagedThreadPoolMetrics+0x99b750 000000f5a067f950 00007ffad3c627e6 thirdparty-APM!updateManagedThreadPoolMetrics+0x70ce36, calling kernel32!TlsSetValueStub 000000f5a067f980 00007ffad3ebe880 thirdparty-APM!updateManagedThreadPoolMetrics+0x968ed0, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f9b0 00007ffafa4b84d4 kernel32!BaseThreadInitThunk+0x14, calling ntdll!LdrpDispatchUserCallTarget 000000f5a067f9e0 00007ffafc51e871 ntdll!RtlUserThreadStart+0x21, calling ntdll!LdrpDispatchUserCallTarget ~ WinDbg Analysis I am newbie to WinDbg. But I was able to verify some basics diagnostics to rule out. But there are lot interesting details which I do not fully understand. !VerifyHeap - No Corruption Found !GCHandleLeaks - No Handle Leak Detected !dlk No Dead Lock Detected - using sosex.dll extension No Out of Memory Exception Note:* Below are details from one of the memory dump, other memory dump also report very similar stats. Thread Pool Details ~~~ CPU utilization: 6% Worker Thread: Total: 496 Running: 165 Idle: 331 MaxLimit: 1000 MinLimit: 496 Work Request in Queue: 141 Unknown Function: 00007ffae3b519d0 Context: 00000296f1755900 Unknown Function: 00007ffae3b519d0 Context: 00000296fd9b7718 Unknown Function: 00007ffae3b519d0 Context: 00000296f175a130 ---removed entries Number of Timers: 0 Completion Port Thread:Total: 2 Free: 2 MaxFree: 16 CurrentLimit: 2 MaxLimit: 1000 MinLimit: 496 ~~~ GC Handles ~~~ Total 57233 objects Handles: Strong Handles: 8789 Pinned Handles: 49 Async Pinned Handles: 59 Ref Count Handles: 11 Weak Long Handles: 15004 Weak Short Handles: 33241 SizedRef Handles: 80 ~~~ DotMemory Finalizable Objects - Which seems to match GC Handles - Is it because GC thread is blocked? !Threads I tried to see what Thread 41 Stack Trace can tell me, but I can not make much sense of it? !dumpheap -live -thinlock Note: All thin lock objects of same size which 24. Which is free object according to Maoni Stephens "You can not lock a free object." I am using WinDbg Preview - Debugger engine version: 10.0.20153.1000 Debugger client version: 1.0.2007.06001 ~ removed ... 00000295c8a7c530 0000029241e30610 24 ThinLock owner 2c (00000296d1268f60) Recursive 0 00000295c8a7c918 0000029241e30610 24 ThinLock owner 6e (00000296d59d2c70) Recursive 0 00000295c8a7cd90 0000029241e30610 24 ThinLock owner 6e (00000296d59d2c70) Recursive 0 Found 7190 objects. ~ 0:041> !SyncBlk Note: This also produces interesting, I believe these are coming from Bundling, but I am failing to conclude that this is the root cause? Help ~~~ Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock Owner 735 00000296ecbb30b8 1 1 00000296d15566c0 1370 227 0000029245808468 System.Web.DirectoryMonitor 1125 00000296e8630bc8 3 1 00000296d64f1780 e70 207 00000293c4548c38 log4net.Appender.RollingFileAppender 1852 00000296eca2ed78 3 1 00000296ed3fd5f0 2a04 424 000002944afd2168 System.Web.CachedPathData 2293 00000296ec4857d8 11 1 00000296ed4089a0 904 448 0000029547c603c8 System.Web.CachedPathData 2414 00000296d7ee8398 15 1 00000296ecad0580 2bd4 384 00000294c9f64768 System.Web.CachedPathData 2869 00000296f3e8be88 5 1 00000296ecaa1780 1704 224 00000292c8cd8838 System.Web.CachedPathData 2999 00000296e37ad368 21 1 00000296e3a37770 3a68 442 00000295c862d098 System.Web.CachedPathData Total 3099 CCW 11 RCW 12 ComClassFactory 0 Free 921 ~~~ I will appreciate any help in that regard to find the root cause. Kindly let me know if you need more information or what further analysis can be done on these memory dumps. Since latency, 503, Redis Timeout errors were observed only when File Share was responding slow, so I have not done further traces on production.
Author:	marafiq
Assignees:	-
Labels:	`area-GC-coreclr`, `tenet-performance`, `untriaged`
Milestone:	-

msftbot[bot] on 16 Nov 2020

what thirdparty-APM tool are you using ? Does the issue repro with that disabled?

mangod9 on 16 Nov 2020

It's DynaTrace. I have not tried disabling it. As it involves process restart. In prod there s whole process of RFC sadly :)

By looking at stack trace, can we say explicitly it's DynaTrace at fault.

marafiq on 16 Nov 2020

Interestingly this issue only happened when FileShare was having latency issues. One API store& read data from it.
Since file share latency is gone, we r not experiencing this issue of suspension to noticeable scale.

marafiq on 16 Nov 2020

Some Dynatrace .NET Agent insights:

It starts it's own companion threads, this is completely normal.
The "mscorlib.dll!Unknown" are methods that are injected by Dynatrace into mscorlib via Profiling API, that's why they cannot be properly resolved. That's again normal (in case anyone is wondering).
At first glance the Dynatrace thread you referenced looks like it's involved in the continuous CPU Profiling ("method hotspots") functionality of the agent.

Here are some things you can try on the Dynatrace side:

Enable a different CPU profiling algorithm (Dynatrace > "Settings" > "Server-Side monitoring" > "Deep monitoring" > "New OneAgent features/Troubleshooting" > "Prefer .NET method hotspot information via CLR")
Alternatively, disable CPU profiling altogether (to verify) (Dynatrace > "Settings" > "Server-Side monitoring" > "Deep monitoring" > "Troubleshooting" > "Capture method hotspot information in PurePaths: Off")

If this does not help and it still looks like Dynatrace thread is the culprit please feel free to open a support ticket!

discostu105 on 16 Nov 2020

if there is a consistent repro yeah next step would be to ensure it reproes with it disabled or updated config as suggested above.

mangod9 on 16 Nov 2020

first of all Thanks for taking a look. We are in process of doing diagnostics on DynaTrace end.
I opened ticket here to know, under what circumstances GC thread can be blocked for 10 seconds, as seen in DebugDiag report.
How I can get details of Critical Section it is waiting on?
Is DT thread running on higher priority than GC Thread, can we confirm it from memory dump using WinDbg? Does trace tell us that it was able to create companion thread?
Is it even possible to get that details given its not full native dump, and DT is unmanaged code?
Can we do specific perfview trace to confirm if it is still happening, without involving process restarts?

Given I am newbie to WinDbg , I got following message. I do not fully understand what it means.
El4KIlwXUAEwPgf

marafiq on 16 Nov 2020

.NET CPU Method hotspots in the used variation needs to suspend threads to do a stack walk. It might be the case that the thread is not resumed any more for whatever reason. This is then handled by some watchdog thread.
For details of the critical section a full dump might be a good start.
You are already looking at the companion thread.
You might be able to sample by PerfView to see if the observed stack is hit many times.

georgschausberger on 16 Nov 2020

I was to grab details of Critical Section from dumps. It says "Uninitialized or deleted." Both memory dumps have same message when I do !cs debuginfoaddress
Would it be fair to conclude that GC was blocked, because CS was corrupted?

Also what this line means in stack trace?
~
000000f5a067eae0 00007ffafc4b81a0 ntdll!RtlSetLastWin32Error+0x40, calling ntdll!_security_check_cookie
~

Critical Section Info

~~~

DebugInfo = 0x00000296f7876af0
Critical section = 0x000002924211c928 (+0x2924211C928)
LOCKED
LockCount = 0x0
WaiterWoken = No
OwningThread = 0x00000000000028d0
RecursionCount = 0x1
LockSemaphore = 0xFFFFFFFF
SpinCount = 0x00000000020007cf

0:000> !cs 0x000002924211c928

Critical section = 0x000002924211c928 (+0x2924211C928)
DebugInfo = 0x00000296f7876af0
LOCKED
LockCount = 0x0
WaiterWoken = No
OwningThread = 0x00000000000028d0
RecursionCount = 0x1
LockSemaphore = 0xFFFFFFFF
SpinCount = 0x00000000020007cf

0:000> !cs 0x00000296f7876af0

Critical section = 0x00000296f7876af0 (+0x296F7876AF0)
Uninitialized or deleted.
LOCKED
LockCount = 0xEF7B8DB5
WaiterWoken = Yes
OwningThread = 0x00007ffafc5fb610
RecursionCount = 0x292
LockSemaphore = 0xF7CCDEB0
SpinCount = 0x0000000200000000
~~~

marafiq on 16 Nov 2020

please show the stack for the GC thread so we know what state GC is in.

Maoni0 on 16 Nov 2020

Finalizer Thread
~
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain Count Apt Exception
4 1 395c 00000292437e4a40 28220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 Ukn
39 2 19e8 0000029241bf9690 2b220 Preemptive 0000029547F22838:0000029547F22FF0 0000029241e4c730 0 MTA (Finalizer)
41 3 28d0 00000292432f9910 20220 Cooperative 000002944D02DDA0:000002944D02F2D8 00000296d4566ef0 1 Ukn
43 4 13b4 00000296d2d54190 102a220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 MTA (Threadpool Worker)
~
State of Finalizer Thread
~
0:039> !ThreadState 2b220
Legal to Join
Background
CLR Owns
CoInitialized
In Multi Threaded Apartment
Fully initialized
~
CLR Stack of Finalizer Thread
~
0:039> !clrstack [19e8]
OS Thread Id: 0x19e8 (39)
Child SP IP Call Site
000000f5a04ff6f0 00007ffafc556554 [DebuggerU2MCatchHandlerFrame: 000000f5a04ff6f0]
0:039> !clrstack
OS Thread Id: 0x19e8 (39)
Child SP IP Call Site
000000f5a04ff6f0 00007ffafc556554 [DebuggerU2MCatchHandlerFrame: 000000f5a04ff6f0]
~

Full Detail of threads - Stack trace of all GC Threads - towards the end. Sorry its kind of very long

~~~
0:032> !threads -special
ThreadCount: 802
UnstartedThread: 0
BackgroundThread: 690
PendingThread: 0
DeadThread: 112
Hosted Runtime: no
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain 4 1 395c 00000292437e4a40 28220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 39 2 19e8 0000029241bf9690 2b220 Preemptive 0000029547F22838:0000029547F22FF0 0000029241e4c730 0 41 3 28d0 00000292432f9910 20220 Cooperative 000002944D02DDA0:000002944D02F2D8 00000296d4566ef0 1 43 4 13b4 00000296d2d54190 102a220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 44 5 1730 0000029242e94f30 21220 Preemptive 000002954B6458D8:000002954B6477A0 0000029241e4c730 0 42 6 16d8 00000296d5d00850 20220 Preemptive 00000295CD395CA8:00000295CD3978E8 00000296d4566ef0 0 45 7 2384 00000292424a58c0 1020220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 46 14 1454 00000296d5a1c790 202b220 Preemptive 000002944E82AC18:000002944E82B228 00000296d4566ef0 0 47 15 1184 00000296d5a1bfc0 202b220 Preemptive 00000293CF5330C8:00000293CF534FE0 00000296d4566ef0 0 48 16 2894 00000296d5a1e6d0 202b220 Preemptive 000002924D309D38:000002924D30A3D8 00000296d4566ef0 0 49 17 3c6c 00000296d6407f50 202b220 Preemptive 000002944E8A5850:000002944E8A7288 00000296d4566ef0 0 50 18 4110 00000296d64048a0 202b220 Preemptive 00000292CE792460:00000292CE7935A0 00000296d4566ef0 0 51 19 39c4 00000296d6405070 202b220 Preemptive 00000294CFEE5938:00000294CFEE68C0 00000296d4566ef0 0 52 20 332c 00000296d6408720 202b220 Preemptive 00000295CEAB8CC0:00000295CEABA4B0 00000296d4566ef0 0 53 21 35a0 00000296d6408ef0 202b220 Preemptive 00000295CEABA6B8:00000295CEABC4B0 00000296d4566ef0 0 54 22 2ce0 00000296d6409e90 202b220 Preemptive 000002934CCE3080:000002934CCE39D8 00000296d4566ef0 0 55 23 4150 00000296d640ae30 202b220 Preemptive 00000294CFFD0A10:00000294CFFD24D0 00000296d4566ef0 0 56 25 2ef4 00000296d0a63490 3029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 57 34 224c 00000296d0d30dd0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 58 35 2b18 00000296d0d2a070 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 59 36 121c 00000296d0d2d720 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 60 37 159c 00000296d0d2def0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 61 38 18cc 00000296d0d2ee90 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 62 40 2e4c 00000296d0d2a840 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 63 41 f48 00000296d126dd80 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 64 42 3078 00000296d12677f0 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 65 43 3784 00000296d1267fc0 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 66 44 291c 00000296d1268f60 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 67 45 403c 00000296d126e550 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 68 46 183c 00000296d1268790 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 69 47 20bc 00000296d126be40 21220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 71 48 9ec 00000296d15ffe60 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 72 49 2b60 00000296d1600e00 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 74 53 2890 00000296d1601da0 202b220 Preemptive 00000292CC6714A0:00000292CC673410 00000296d4566ef0 1 9 54 1c10 00000296d1602d40 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 75 67 2418 00000296d1608330 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 76 73 2eb0 00000296e2feaee0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 77 0 00000296e2fec650 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 77 84 1b24 00000296e2fe9f40 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 78 88 39f4 0000029243aef070 202b220 Preemptive 00000295CD9EE548:00000295CD9EF2A8 00000296d2b000b0 0 79 89 31d8 0000029243ae8ae0 202b220 Preemptive 000002954CA45DA0:000002954CA46E08 00000296d2b000b0 0 80 90 3088 0000029243aee0d0 202b220 Preemptive 00000292CDB2FE78:00000292CDB31798 00000296d2b000b0 0 81 91 1e8c 0000029243aed900 202b220 Preemptive 000002954D188480:000002954D18A208 00000296d2b000b0 0 82 92 2484 0000029243af0fb0 202b220 Preemptive 000002944DF31210:000002944DF31DF8 00000296d2b000b0 0 83 93 101c 0000029243af1780 202b220 Preemptive 00000294CEAE2210:00000294CEAE2270 00000296d2b000b0 0 84 94 ad0 0000029243af2720 202b220 Preemptive 00000292CE23D8F8:00000292CE23ECD0 00000296d2b000b0 0 85 95 2170 0000029243af36c0 202b220 Preemptive 00000292CCFF7AF8:00000292CCFF8500 00000296d2b000b0 0 86 96 1b08 00000296dda13da0 202b220 Preemptive 00000292CDB3C8E0:00000292CDB3DBC0 00000296d2b000b0 0 87 97 1080 00000296dda14570 202b220 Preemptive 00000293CD6E82A0:00000293CD6E9108 00000296d2b000b0 0 88 98 12d4 00000296dda15510 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 89 99 2f4c 00000296dda0e7b0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 90 100 52c 00000296dda0f750 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 91 101 19b0 00000296dda0ef80 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 92 102 4138 00000296dda12630 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 93 103 2ad4 00000296dda17c20 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 94 104 3164 00000296dda183f0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 95 105 21f8 00000296dda18bc0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 1 5 106 78c 00000296dda19390 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 96 107 38a4 00000296dda1ca40 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 3 108 bd4 00000296dda1b2d0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 2 109 30c 00000296dda1c270 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 6 30 2d7c 00000296d59d43e0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 98 27 4024 00000296d59d3c10 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 99 110 2040 00000296d59d2c70 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 100 111 1054 00000296d59cedf0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 101 112 1c48 00000296d59d0d30 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 102 113 3984 00000296d59d6320 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 103 114 37dc 00000296d59d72c0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 104 115 1b1c 00000296d59d7a90 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 105 116 3f64 00000296d59d8260 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 106 117 14c0 00000296d59d9200 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 107 118 21fc 00000296d59dff60 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 108 119 1f68 00000296d59de020 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 109 120 bec 00000296d59e0f00 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 110 121 1ab8 00000296d59e0730 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 111 122 3bd0 00000296d59de7f0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 112 123 28b8 00000296d59df790 202b220 Preemptive 0000000000000000:0000000000000000 00000296e3904300 0 113 124 20 00000296d59d99d0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 115 126 3398 00000296d59e7c60 1029220 Preemptive 000002944E78FD10:000002944E790978 0000029241e4c730 0 8 132 189c 00000296d59eda20 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 11 133 1b40 00000296dda1a330 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 116 32 1200 00000296d0a653d0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 10 31 23d0 00000296d59ec2b0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 7 134 ff4 00000296d0d2fe30 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 121 135 31ac 00000296d1554f50 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 122 136 20a8 00000296d1555ef0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 117 137 1270 00000296d1558600 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 175 0 00000296d156e590 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 124 285 4088 00000296ed4106a0 202b220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 125 297 3c50 00000296d64db020 202b220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 126 131 2a30 00000296ecaaf240 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 127 130 289c 00000296ecab30c0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 128 129 3328 00000296ecab5000 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 129 127 3570 00000296ecab2120 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 130 87 26d4 00000296ecab6f40 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 131 85 2614 00000296ecab7ee0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 132 83 1274 00000296ecab8e80 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 133 81 27d0 00000296ecab9650 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 134 79 2dfc 00000296ecab5fa0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 135 76 12e8 00000296ecab9e20 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 136 75 2cc0 00000296ecabadc0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 137 72 18d0 00000296ecabd4d0 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 138 71 59c 00000296ecab6770 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 139 68 2068 00000296ed1d9470 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 140 63 1a24 00000296ed1dbb80 202b220 Preemptive 0000000000000000:0000000000000000 00000296e84e9680 0 123 388 f3c 00000296ed1e01d0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 145 341 fec 00000296ed1e57c0 20220 Preemptive 00000294CF541048:00000294CF5410B0 00000296d4566ef0 0 146 373 3cb0 00000296ed1e86a0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 143 374 3c14 00000296ed1e3880 20220 Preemptive 000002944CCC3D28:000002944CCC46D0 0000029241e4c730 0 120 379 3280 00000296ed1e09a0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 149 350 38b8 00000296ed1e4050 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 147 371 2cc4 00000296ed1e9640 2020220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 1 151 332 2d24 00000296ed1eadb0 20220 Preemptive 00000293CE38F6A0:00000293CE38F718 00000296d4566ef0 0 119 377 2c50 00000296ed1e1940 20220 Preemptive 00000292CE75AE38:00000292CE75AE40 00000296d4566ef0 0 142 378 2cbc 00000296ed1d9c40 20220 Preemptive 000002954D134228:000002954D134358 00000296d4566ef0 0 144 372 4ac 00000296ed1e4ff0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 148 338 3c58 00000296ed1ea5e0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 154 74 2ca0 00000296ed1f0b70 1029220 Preemptive 000002924C87F570:000002924C880140 0000029241e4c730 0 155 303 1490 00000296ed1f1340 1029220 Preemptive 00000294CFBA7E18:00000294CFBA9A10 0000029241e4c730 0 156 12 3d18 00000296ed1f6160 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 318 0 00000296ed1f03a0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 157 417 19b4 00000296ed1bee90 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 158 439 2168 00000296ecac4a00 1029220 Preemptive 00000292CE22C5B0:00000292CE22CCD0 0000029241e4c730 0 159 467 2c7c 00000296ecade810 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 160 480 40b8 00000296ecaaabf0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 162 492 1874 00000296d64d98b0 1029220 Preemptive 000002934BD0C9A0:000002934BD0D598 0000029241e4c730 0 161 493 2980 00000296d64e8ae0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 163 505 3d40 00000296d64ee8a0 1029220 Preemptive 00000294CE1DBA48:00000294CE1DD710 00000296d4566ef0 0 164 520 1538 00000296d64d13e0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 531 0 00000296ed4033b0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 165 534 1ee0 00000296ed4052f0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 166 549 2350 00000296ed414cf0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 167 575 2d90 00000296d2477700 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 168 597 27dc 00000296d16044b0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 169 608 3e48 00000296ed1fe660 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 170 613 3598 00000296ed20b180 3029220 Preemptive 00000294CE3DCD38:00000294CE3DEAC8 00000296d2b000b0 0 171 645 e30 00000296ed21da60 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 172 627 ab8 00000296ed2145f0 1029220 Preemptive 000002924D26E008:000002924D26FDF8 0000029241e4c730 0 173 674 1590 00000296ed22fb70 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 690 0 00000296ed238810 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 695 0 00000296d7f05010 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 175 696 3150 00000296d7f0b5a0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 174 711 24d4 00000296d7f0e480 1029220 Preemptive 00000294CF54E000:00000294CF54F0B0 0000029241e4c730 0 176 720 3cf4 00000296d7f10b90 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 177 738 1d38 00000296d7f1f5f0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 178 743 13ac 00000296d7f29230 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 746 0 00000296d7f28290 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 179 753 1708 00000296d7f24410 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 180 725 c8c 00000296d7f13a70 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 118 398 3250 00000296e9112bd0 20220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 153 397 9b4 00000296e9115ab0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 150 396 3dc0 00000296e9116a50 20220 Preemptive 000002944E92C860:000002944E92C878 00000296d4566ef0 0 182 364 23b0 00000296e90ed240 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 183 782 1144 00000296e90fb4d0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 781 0 00000296e9101a60 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 189 785 21ac 00000296e90f2830 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 190 784 18d8 00000296e90f3000 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 792 0 00000296e90f85f0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 184 795 2e2c 00000296e90f6e80 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 783 0 00000296e90fad00 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 185 788 8ac 00000296e90f9d60 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 186 787 2a5c 00000296e90f5710 1029220 Preemptive 00000292CE2374A8:00000292CE238CD0 0000029241e4c730 0 187 786 28e0 00000296e90f5ee0 1029220 Preemptive 000002954D144430:000002954D146208 0000029241e4c730 0 188 810 2f20 00000296e90e7c50 1029220 Preemptive 00000293CD8F6640:00000293CD8F72C8 0000029241e4c730 0 191 803 11e4 00000296e90eca70 3029220 Preemptive 00000292CD978088:00000292CD978A68 00000296d2b000b0 1 192 801 3d70 00000296e90e93c0 1029220 Preemptive 000002944D2E1880:000002944D2E2358 0000029241e4c730 0 193 778 294 00000296e90e4d70 1029220 Preemptive 00000293CE34A998:00000293CE34B638 0000029241e4c730 0 XXXX 776 0 00000296e90e64e0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 775 0 00000296e90e8bf0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 195 773 3540 00000296e90f10c0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 194 774 1b88 00000296e90f08f0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 196 772 12a4 00000296e90e9b90 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 197 771 3eb4 00000296e90f4770 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 198 638 315c 00000296e9102a00 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 199 764 412c 00000296e9118990 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 200 762 2714 00000296e9119930 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 201 759 9cc 00000296e911b870 1029220 Preemptive 000002934C89FFB8:000002934C8A0A78 0000029241e4c730 0 202 723 15d0 00000296d64dc790 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 203 742 1d10 00000296d64dcf60 1029220 Preemptive 00000295CD60C178:00000295CD60C198 00000296d4566ef0 1 XXXX 641 0 00000296d64ed900 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 204 693 2850 00000296d64eaa20 1029220 Preemptive 00000294CFC5C090:00000294CFC5C2C0 0000029241e4c730 0 205 644 15e4 00000296d64e92b0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 651 0 00000296d64f0fb0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 206 605 3d54 00000296d64eb9c0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 207 614 e70 00000296d64f1780 1029220 Preemptive 00000294D0055238:00000294D0055240 00000296d4566ef0 2 XXXX 546 0 00000296d64f5600 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 208 537 1944 00000296d64bcbc0 1029220 Preemptive 000002954C7C0C10:000002954C7C12E8 0000029241e4c730 0 209 529 3020 00000296d64bac80 1029220 Preemptive 000002944E92D068:000002944E92E878 0000029241e4c730 0 210 323 3bd4 00000296d64bd390 1029220 Preemptive 000002924C93CBE8:000002924C93DFF0 0000029241e4c730 0 211 24 3fa4 00000296d64bc3f0 1029220 Preemptive 000002944E6106A0:000002944E612530 0000029241e4c730 0 XXXX 362 0 00000296d64b6e00 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 212 361 1ddc 00000296d64b75d0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 213 357 32d0 00000296d64c5090 1029220 Preemptive 000002954D605BF0:000002954D606DD8 00000296d4566ef0 0 214 355 382c 00000296d64c7f70 1029220 Preemptive 0000029346F3CC20:0000029346F3E970 00000296d4566ef0 0 215 354 6ec 00000296d64c8f10 3029220 Preemptive 00000293CB021E28:00000293CB022648 00000296d2b000b0 0 XXXX 349 0 00000296d64c3150 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 216 346 3f74 00000296d64cdd30 1029220 Preemptive 00000295CE637040:00000295CE638D20 0000029241e4c730 0 217 343 3f4c 00000296d64d0c10 3029220 Preemptive 00000292479E64E0:00000292479E6718 00000296d2b000b0 0 218 339 232c 00000296d64d3af0 1029220 Preemptive 00000295CDE3B688:00000295CDE3B760 0000029241e4c730 0 219 337 26dc 00000296d64ccd90 1029220 Preemptive 000002934C4FB750:000002934C4FBD98 0000029241e4c730 0 220 335 2014 00000296d64d5a30 1029220 Preemptive 000002944DB70958:000002944DB722C0 0000029241e4c730 0 221 333 200c 00000296ecac4230 1029220 Preemptive 000002924CC71E70:000002924CC72570 0000029241e4c730 0 222 311 1858 00000296ecac9050 1029220 Preemptive 00000293CDAF50B0:00000293CDAF5C50 0000029241e4c730 0 223 309 1638 00000296ecad7ab0 1029220 Preemptive 00000292CCF9D1F0:00000292CCF9E500 0000029241e4c730 0 224 299 1704 00000296ecaa1780 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 1 225 298 3dfc 00000296ecaafa10 1029220 Preemptive 00000294CFEE0990:00000294CFEE28C0 0000029241e4c730 0 226 296 1600 00000296ecab3890 1029220 Preemptive 000002944DB097D0:000002944DB0A410 0000029241e4c730 0 227 294 1370 00000296d15566c0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 1 228 292 ca0 00000296d1552070 1029220 Preemptive 00000295CDE567F0:00000295CDE57610 0000029241e4c730 0 229 291 e24 00000296d1553fb0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 289 0 00000296d15537e0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 230 283 3610 00000296d15612a0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 231 279 1d4c 00000296d1564950 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 232 276 650 00000296d155e3c0 1029220 Preemptive 00000293CE350058:00000293CE351638 0000029241e4c730 0 233 274 3064 00000296d155eb90 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 234 266 1e40 00000296d15687d0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 235 257 1180 00000296ed1ff600 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 255 0 00000296ed200d70 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 236 249 3914 00000296ed2082a0 1029220 Preemptive 00000295CDFFF848:00000295CE000F80 0000029241e4c730 0 237 245 2a88 00000296ed201d10 1029220 Preemptive 00000295CD8C4A30:00000295CD8C5598 0000029241e4c730 0 238 239 25f0 00000296ed213650 1029220 Preemptive 000002934C2CCC50:000002934C2CEA88 0000029241e4c730 0 239 238 4060 00000296ed215590 1029220 Preemptive 00000294CB8FFBF8:00000294CB8FFCE0 00000296d2b000b0 0 240 235 1288 00000296ed211ee0 3029220 Preemptive 00000292C9D70130:00000292C9D710D0 00000296d2b000b0 0 241 233 20a4 00000296ed220940 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 225 0 00000296ed230340 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 242 221 2cfc 00000296f4b973b0 1029220 Preemptive 00000292CD99B618:00000292CD99CA98 0000029241e4c730 0 243 220 2cb0 00000296f4b9ba00 1029220 Preemptive 000002934C4CA2F8:000002934C4CA428 0000029241e4c730 0 244 219 141c 00000296f4b9d170 1029220 Preemptive 00000295C8FC3920:00000295C8FC4190 0000029241e4c730 0 245 218 1660 00000296f4b95c40 1029220 Preemptive 000002944CD38000:000002944CD39E00 0000029241e4c730 0 246 215 4380 00000296f4b9f0b0 1029220 Preemptive 00000294CFFDD750:00000294CFFDE4D0 0000029241e4c730 0 XXXX 211 0 00000296f4ba3ed0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 247 210 2b90 00000296f4ba5640 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 248 207 2d88 00000296f4ba0050 1029220 Preemptive 000002934B75C748:000002934B75DE98 0000029241e4c730 0 249 202 3eb8 00000296f4ba17c0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 250 201 1324 00000296f4ba1f90 1029220 Preemptive 000002954D6EC5A0:000002954D6ED298 0000029241e4c730 0 XXXX 193 0 00000296f4bab400 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 252 191 13bc 00000296f4bb40a0 1029220 Preemptive 000002944CE325D8:000002944CE32938 0000029241e4c730 0 XXXX 190 0 00000296f4bb5040 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 189 0 00000296f4bb4870 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 251 188 28bc 00000296f4b79720 1029220 Preemptive 000002954B641308:000002954B6417A0 0000029241e4c730 0 XXXX 186 0 00000296f4b7ae90 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 253 180 4284 00000296f4b77010 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 254 179 1eb4 00000296f4b7ed10 1029220 Preemptive 000002954D0BEBC8:000002954D0C00B0 0000029241e4c730 0 XXXX 176 0 00000296f4b78780 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 255 170 994 00000296f4b85a70 1029220 Preemptive 00000294CFF764A0:00000294CFF764D0 00000296d4566ef0 0 256 167 14c4 00000296f4b8c000 1029220 Preemptive 000002924D304F90:000002924D3063D8 0000029241e4c730 0 257 166 1018 00000296f4b8c7d0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 165 0 00000296f4b8cfa0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 160 0 00000296f4b8fe80 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 258 148 2ce8 00000296f4b90e20 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 150 0 00000296d7f07ef0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 259 139 2ff8 00000296d7f12300 3029220 Preemptive 00000294CC179AC8:00000294CC17B9C8 00000296d2b000b0 0 260 145 20e8 00000296d7f0ec50 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 261 141 34e8 00000296d7f19060 1029220 Preemptive 00000295CEAB6570:00000295CEAB84B0 0000029241e4c730 0 262 149 18e4 00000296d7f1bf40 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 263 140 1f5c 00000296d7f20590 1029220 Preemptive 00000294CDF5EAA8:00000294CDF5F6E0 0000029241e4c730 0 XXXX 143 0 00000296d7f29a00 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 264 78 41cc 00000296d7f30760 1029220 Preemptive 000002954CB9B6B8:000002954CB9D088 0000029241e4c730 0 265 39 1420 00000296d7f2d0b0 1029220 Preemptive 00000295CD8091F8:00000295CD809B08 0000029241e4c730 0 266 29 470 00000296d7f3b340 3029220 Preemptive 00000294CC175B78:00000294CC1779C8 00000296d2b000b0 0 267 813 3f6c 00000296d7f3ab70 1029220 Preemptive 00000294D0023858:00000294D0024AE0 0000029241e4c730 0 268 814 16d0 00000296d7f3cab0 1029220 Preemptive 00000294CEBAEB40:00000294CEBB0220 0000029241e4c730 0 269 816 2eac 00000296d7f36520 1029220 Preemptive 00000292CBFB1A60:00000292CBFB2A68 0000029241e4c730 0 270 819 34e0 00000296d7f418d0 1029220 Preemptive 00000292CE239370:00000292CE23ACD0 0000029241e4c730 0 271 821 271c 00000296d2464650 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 828 0 00000296d247e460 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 831 0 00000296e8b326c0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 273 822 488 00000296d2474820 1029220 Preemptive 00000295CD828DB8:00000295CD829718 0000029241e4c730 0 XXXX 823 0 00000296d247c520 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 274 826 3f90 00000296d247d4c0 1029220 Preemptive 000002954D0BB820:000002954D0BC0B0 0000029241e4c730 0 XXXX 829 0 00000296e8b2e070 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 275 830 8a4 00000296e8b2e840 1029220 Preemptive 000002924C7E64E8:000002924C7E6A70 0000029241e4c730 0 XXXX 384 0 00000296e8b34600 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 381 0 00000296f4c30780 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 276 650 360c 00000296ed223820 1029220 Preemptive 00000293CCA61AD8:00000293CCA62010 0000029241e4c730 0 277 370 1218 00000296de3a7000 1029220 Preemptive 000002924CC8A3F8:000002924CC8A420 00000296d4566ef0 2 278 607 3a98 00000296e8b3bb30 1029220 Preemptive 000002924D303008:000002924D3043D8 0000029241e4c730 0 280 602 38c8 00000296e8b3cad0 1029220 Preemptive 00000293CEBFD608:00000293CEBFF450 0000029241e4c730 0 279 601 3a90 00000296ecaab3c0 1029220 Preemptive 000002924C8135D0:000002924C8143E0 0000029241e4c730 0 281 591 9a4 00000296f15b67d0 1029220 Preemptive 00000292CD013988:00000292CD014AC8 0000029241e4c730 0 XXXX 665 0 00000296ecaa9c50 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 582 0 00000296d2471170 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 572 0 00000296d247b580 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 282 570 2ed0 00000296f15b4890 1029220 Preemptive 000002934C63F630:000002934C640558 0000029241e4c730 0 XXXX 568 0 00000296e8b40180 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 284 369 15e8 00000296de3a6830 3029220 Preemptive 00000292CC0E6BE8:00000292CC0E70D8 00000296d4566ef0 0 283 560 2f94 00000296ee594670 1029220 Preemptive 000002954B300618:000002954B300D08 0000029241e4c730 0 XXXX 556 0 00000296ed223ff0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 285 547 bc4 00000296ed21e230 1029220 Preemptive 00000294CD8515F0:00000294CD851AC0 0000029241e4c730 0 XXXX 386 0 00000296d24607d0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 297 524 2a68 00000296f15b5060 1029220 Preemptive 00000292CA811900:00000292CA811DF8 0000029241e4c730 0 328 522 162c 00000296ed234990 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 294 365 4030 00000296f1977140 1029220 Preemptive 00000295CD541730:00000295CD542D38 0000029241e4c730 0 299 519 2cf4 00000296e8b41120 1029220 Preemptive 000002934C6375A8:000002934C6386A8 0000029241e4c730 0 288 382 10b0 00000296d5981be0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 341 518 2c30 00000296f15b5830 1029220 Preemptive 00000292CE766CA8:00000292CE766CF0 0000029241e4c730 0 XXXX 517 0 00000296ecaae2a0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 515 0 00000296ed22d460 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 326 514 3fb8 00000296e8b42890 1029220 Preemptive 00000292CE232610:00000292CE232CD0 0000029241e4c730 0 296 375 1474 00000296f19780e0 1029220 Preemptive 000002924D3D21D8:000002924D3D4078 0000029241e4c730 0 295 507 2e14 00000296f4a93420 1029220 Preemptive 000002924C03B628:000002924C03B6C8 0000029241e4c730 0 292 385 1190 00000296d597f4d0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 333 504 16d4 00000296ecac9820 1029220 Preemptive 00000295CE53D600:00000295CE53E470 0000029241e4c730 0 293 497 25bc 00000296e8b43060 1029220 Preemptive 00000293CCA01060:00000293CCA02BE0 0000029241e4c730 0 329 496 2df0 00000296ed230b10 1029220 Preemptive 00000295CE06A8F0:00000295CE06C3B0 0000029241e4c730 0 330 484 2914 00000296f4a94b90 1029220 Preemptive 000002934B82C830:000002934B82D5C8 0000029241e4c730 0 325 483 3a08 00000296e8b45f40 1029220 Preemptive 000002944E606828:000002944E608680 0000029241e4c730 0 300 479 1ea8 00000296ee599490 1029220 Preemptive 000002954D54F020:000002954D550528 0000029241e4c730 0 286 468 30d0 0000029242a0b7e0 1029220 Preemptive 00000294CF556C30:00000294CF556F60 0000029241e4c730 0 287 464 1810 00000296ecad3c30 1029220 Preemptive 00000292CCE002D0:00000292CCE002E0 0000029241e4c730 0 290 457 d7c 0000029242a0c780 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 289 455 2050 00000296ecad1520 1029220 Preemptive 000002944E96B5C8:000002944E96BF48 0000029241e4c730 0 291 453 34fc 00000296d3fbb800 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 452 0 00000296ecadb930 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 449 0 0000029242a0cf50 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 298 447 bb8 00000296f19768a0 1029220 Preemptive 00000295CDF2C2F0:00000295CDF2D220 0000029241e4c730 0 XXXX 728 0 00000296ee40d510 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 334 444 191c 00000296ecad4bd0 1029220 Preemptive 00000295CDE54CF0:00000295CDE55610 0000029241e4c730 0 335 442 2b98 00000296ecada990 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 304 390 3da8 00000296e207cf40 1029220 Preemptive 00000293CBA0ACB8:00000293CBA0B890 0000029241e4c730 0 305 436 324 00000296f1975900 1029220 Preemptive 000002934CD75330:000002934CD755E8 0000029241e4c730 0 306 426 3228 00000296ecad6340 1029220 Preemptive 00000294CDF5FEB8:00000294CDF616E0 0000029241e4c730 0 XXXX 425 0 00000296ecade040 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 632 0 00000296d64cb620 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 309 424 437c 00000296f19760d0 1029220 Preemptive 000002924CF7EC60:000002924CF803E0 0000029241e4c730 0 307 415 4120 00000296e8b47e80 3029220 Preemptive 00000293CB0A1ED0:00000293CB0A2678 00000296d2b000b0 0 XXXX 421 0 00000296f7f55fb0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 311 406 30f0 00000296f6eff840 1029220 Preemptive 00000295CE542AA8:00000295CE544470 0000029241e4c730 0 308 411 434 00000296f7f5add0 1029220 Preemptive 000002954AA9CCA8:000002954AA9CCE0 00000296d2b000b0 0 312 414 1bec 00000296e8b476b0 1029220 Preemptive 000002934B8E1488:000002934B8E1B38 0000029241e4c730 0 316 404 80c 00000296f6f007e0 1029220 Preemptive 000002944DB6F0E8:000002944DB702C0 0000029241e4c730 0 310 403 3a2c 00000296f7f55010 1029220 Preemptive 000002934CD81518:000002934CD83498 0000029241e4c730 0 313 400 3b20 00000296e8b420c0 1029220 Preemptive 000002954D138770:000002954D13A358 0000029241e4c730 0 317 51 311c 00000296f6eff070 3029220 Preemptive 000002944AD0A4F0:000002944AD0AD58 00000296d2b000b0 0 314 52 1598 00000296e8b46ee0 1029220 Preemptive 000002954C5B8878:000002954C5B8C48 0000029241e4c730 0 315 70 3494 00000296f7f557e0 1029220 Preemptive 000002934C500700:000002934C501D98 0000029241e4c730 0 XXXX 278 0 00000296e8b48650 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 260 0 00000296f7f57720 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 318 251 1534 00000296ec19a2e0 1029220 Preemptive 000002944CD163D8:000002944CD180A0 0000029241e4c730 0 342 203 1ad8 00000296f7f57ef0 1029220 Preemptive 00000295CE53FBC0:00000295CE540470 0000029241e4c730 0 XXXX 58 0 00000296e8b3ea10 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 57 0 00000296ed0516c0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 332 367 738 00000296ee40c570 1029220 Preemptive 00000294CFF29A18:00000294CFF2A770 0000029241e4c730 0 319 9 2d6c 00000296ed050ef0 1029220 Preemptive 00000293CE41EDE0:00000293CE41FCF8 0000029241e4c730 0 320 834 2e94 00000296ed050720 1029220 Preemptive 000002924C879EE0:000002924C87A290 0000029241e4c730 0 XXXX 836 0 00000296dde07a80 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 321 838 187c 00000296dde06ae0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 322 394 19a4 00000296e207a060 1029220 Preemptive 000002954C935268:000002954C936F18 0000029241e4c730 0 323 735 e78 00000296d64b9ce0 1029220 Preemptive 000002924C02DC10:000002924C02F6C8 0000029241e4c730 0 324 387 1aa8 00000296e207c770 1029220 Preemptive 00000295CE660ED8:00000295CE662690 0000029241e4c730 0 XXXX 389 0 00000296e207a830 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 327 670 3140 00000296f6f3ed20 1029220 Preemptive 00000293CCABCA28:00000293CCABD9B0 0000029241e4c730 0 331 704 27fc 00000296ed219be0 1029220 Preemptive 00000292CABD3AC8:00000292CABD3AC8 00000296d2b000b0 0 XXXX 376 0 00000296ee40e4b0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 301 633 1588 00000296ee40fc20 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 302 689 16fc 0000029242a0a070 1029220 Preemptive 00000292CDA93548:00000292CDA954C8 0000029241e4c730 0 XXXX 843 0 00000296ed3b0a00 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 303 844 3a70 00000296ed3af290 1029220 Preemptive 00000292CBE83C08:00000292CBE84408 0000029241e4c730 0 336 846 2f98 00000296ed3b0230 1029220 Preemptive 00000293CCAFC8D0:00000293CCAFD080 0000029241e4c730 0 XXXX 847 0 00000296f196d0f0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 848 0 00000296f196d8c0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 337 849 2564 00000296f196e090 1029220 Preemptive 00000295CE548920:00000295CE54A470 0000029241e4c730 0 338 854 1e0c 00000296e8b52180 1029220 Preemptive 000002954CA42360:000002954CA42E08 0000029241e4c730 0 339 855 4050 00000296ed522ae0 1029220 Preemptive 000002954B68B560:000002954B68B800 0000029241e4c730 0 340 856 3684 00000296ed521370 1029220 Preemptive 000002934A77EBC0:000002934A77F3B0 0000029241e4c730 0 XXXX 857 0 00000296ed521b40 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 343 890 3608 00000296ed12cdc0 1029220 Preemptive 00000292CDAB3AF8:00000292CDAB54C8 0000029241e4c730 0 344 891 1574 00000296f20c3070 3029220 Preemptive 000002944DE38668:000002944DE39940 00000296d2b000b0 0 XXXX 892 0 00000296f20c3840 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 345 893 2380 00000296f20be250 1029220 Preemptive 00000292CE1D8BE8:00000292CE1D8CD0 0000029241e4c730 0 XXXX 896 0 00000296f20bf1f0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 346 897 3468 00000296f20bf9c0 1029220 Preemptive 000002944E827BF8:000002944E829228 0000029241e4c730 0 347 900 2f44 00000296f20c1900 1029220 Preemptive 000002954D1284E8:000002954D12A358 0000029241e4c730 0 XXXX 902 0 00000296f49a1070 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 348 906 fa8 00000296f20c20d0 1029220 Preemptive 00000295CD9F60C8:00000295CD9F72A8 0000029241e4c730 0 349 910 431c 00000296f7f60b90 1029220 Preemptive 000002924CC77470:000002924CC78570 0000029241e4c730 0 350 911 b44 00000296f0c370a0 1029220 Preemptive 00000292CB3C8B08:00000292CB3C97D0 0000029241e4c730 0 XXXX 912 0 00000296f0c37870 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 351 913 363c 00000296f0c38040 1029220 Preemptive 00000292CDAB05D8:00000292CDAB14C8 0000029241e4c730 0 352 914 43bc 00000296f0c38810 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 353 915 15f8 00000296f0c38fe0 1029220 Preemptive 00000295CDE5A788:00000295CDE5B610 0000029241e4c730 0 354 917 1250 00000296f0c368d0 1029220 Preemptive 000002944D9D1B70:000002944D9D35D8 0000029241e4c730 0 355 918 3548 00000296f0c85900 1029220 Preemptive 000002954BEBCE38:000002954BEBD0F0 00000296d4566ef0 0 356 920 8e8 00000296ed1cb9b0 3029220 Preemptive 00000294CBFC6D98:00000294CBFC79C8 00000296d2b000b0 0 357 921 1dd8 00000296f7f64080 1029220 Preemptive 00000293C9A45A20:00000293C9A46E70 0000029241e4c730 0 358 922 154c 00000296ec629090 1029220 Preemptive 000002944E5D2C70:000002944E5D47D0 0000029241e4c730 0 359 923 40c4 00000296ed1cc180 1029220 Preemptive 00000293CEC83D78:00000293CEC859D0 0000029241e4c730 0 360 924 374c 00000296ec629860 1029220 Preemptive 000002924D3CF5D8:000002924D3D0078 0000029241e4c730 0 361 925 10c4 00000296f0ca6670 1029220 Preemptive 00000295CE035850:00000295CE036B90 0000029241e4c730 0 XXXX 879 0 00000296ec6498a0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 362 882 35f0 00000296f20c0190 1029220 Preemptive 000002954D0BCFB0:000002954D0BE0B0 0000029241e4c730 0 364 883 379c 00000296f20bc310 1029220 Preemptive 00000294CF5576C8:00000294CF558F60 0000029241e4c730 0 363 928 1d9c 00000296ec6288c0 1029220 Preemptive 00000294CCF18AA0:00000294CCF19160 0000029241e4c730 0 365 929 337c 00000296f0ca00e0 1029220 Preemptive 000002954C959460:000002954C95ADF8 0000029241e4c730 0 366 933 169c 00000296f0ca08b0 1029220 Preemptive 000002944E5429C0:000002944E5447D0 0000029241e4c730 0 XXXX 934 0 00000296ed1cb1e0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 367 935 3ee4 00000296d597fca0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 368 936 256c 00000296ed1d36b0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 369 938 2844 00000296ed1d4e20 1029220 Preemptive 00000293CE62A920:00000293CE62ABA0 0000029241e4c730 0 XXXX 939 0 00000296eb849040 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 370 940 13f0 00000296f0ca2020 1029220 Preemptive 000002924D166438:000002924D166C50 0000029241e4c730 0 371 941 3de4 00000296f0ca27f0 1029220 Preemptive 000002934C85A1F0:000002934C85C1C8 0000029241e4c730 0 372 942 3b84 00000296f7f61360 1029220 Preemptive 00000295CE019720:00000295CE01AF80 0000029241e4c730 0 373 944 3368 00000296ed1d5dc0 1029220 Preemptive 000002954C8871C8:000002954C889020 0000029241e4c730 0 374 945 1868 00000296f0ca2fc0 1029220 Preemptive 00000292CABD1338:00000292CABD1AC8 00000296d2b000b0 0 375 947 1a28 00000296eb84b750 1029220 Preemptive 00000295CE54AAF8:00000295CE54C470 0000029241e4c730 0 376 948 197c 00000296f49b7f90 1029220 Preemptive 00000295CD852E50:00000295CD853088 0000029241e4c730 0 XXXX 949 0 00000296f7f62300 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 377 950 35ac 00000296f7f632a0 1029220 Preemptive 000002944EA7A3D0:000002944EA7BCD8 0000029241e4c730 0 378 951 4f8 00000296ecac0b80 1029220 Preemptive 000002944E938AA0:000002944E93A728 0000029241e4c730 0 XXXX 952 0 00000296f0c860d0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 379 953 281c 00000296ecac1350 1029220 Preemptive 00000294CF4F0520:00000294CF4F0A50 0000029241e4c730 0 384 954 2bd4 00000296ecad0580 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 2 382 955 e48 00000296ed1dc350 1029220 Preemptive 00000293CD7FC708:00000293CD7FDA68 0000029241e4c730 0 383 957 12b8 00000296ed1d6590 1029220 Preemptive 00000295CB673FB8:00000295CB674028 00000296d2b000b0 0 380 958 f40 00000296ed1d6d60 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 381 960 2764 00000296f0ca3f60 1029220 Preemptive 00000294CF550AA0:00000294CF5510B0 0000029241e4c730 0 385 966 3380 00000296f0c9f140 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 XXXX 983 0 00000296d79d1de0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 986 0 00000296d79cb080 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 962 0 00000296f0c88fb0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 386 985 12bc 00000296ed1e28e0 1029220 Preemptive 00000293CBB053D0:00000293CBB06E08 0000029241e4c730 0 XXXX 965 0 00000296ed1dea60 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 387 970 33dc 00000296eca9f070 1029220 Preemptive 000002944D79F1E8:000002944D79FAE8 0000029241e4c730 0 388 971 3fac 00000296d79cc7f0 1029220 Preemptive 00000293CEC5E218:00000293CEC5F9D0 0000029241e4c730 0 389 972 3d48 00000296d79d0e40 1029220 Preemptive 00000295CE646438:00000295CE646A80 0000029241e4c730 0 390 974 2408 00000296d79ccfc0 1029220 Preemptive 000002934C6B4278:000002934C6B6168 0000029241e4c730 0 XXXX 978 0 00000296ed1d7d00 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 984 0 00000296eb846160 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 394 981 3004 00000296ed1e7700 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 392 982 2e90 00000296f0ca56d0 1029220 Preemptive 00000293CCB08580:00000293CCB08F30 0000029241e4c730 0 XXXX 980 0 00000296d79cdf60 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 964 0 00000296f0c88010 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 391 976 c44 00000296e3a367d0 1029220 Preemptive 00000293CD85F678:00000293CD861198 0000029241e4c730 0 XXXX 880 0 00000296f49b6820 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 393 888 804 00000296f49b6ff0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 152 987 1760 00000296d79d5c60 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 272 988 2b30 00000296ed3f87d0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 395 989 3024 00000296ed3f7060 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 397 991 40b0 00000296d24709a0 20220 Preemptive 00000294CD70B930:00000294CD70B9E0 00000296d4566ef0 0 398 992 2ed8 00000296e3a432f0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 399 993 1148 00000296ed3fce20 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 400 994 3550 00000296ed3ff530 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 401 995 3b50 00000296d246ea60 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 402 996 ae4 00000296ed3fc650 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 403 997 4064 00000296f9bc50a0 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 396 990 1870 00000296d79d8b40 20220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 404 535 4388 00000296f9bc6810 20220 Preemptive 000002954C85F3D0:000002954C85F410 00000296d4566ef0 0 405 877 17c4 00000296f49f8d50 3029220 Preemptive 00000294CF544460:00000294CF5450B0 00000296d2b000b0 0 406 809 3cc8 00000296f49f8580 1029220 Preemptive 000002944EA4E6F0:000002944EA4FCD8 0000029241e4c730 0 407 599 39a0 00000296ed402be0 1029220 Preemptive 00000293CDADD2B0:00000293CDADDEF0 0000029241e4c730 0 408 155 2fbc 00000296d79d73d0 1029220 Preemptive 00000292CE22EF00:00000292CE230CD0 0000029241e4c730 0 409 878 2e80 00000296d79d6430 1029220 Preemptive 00000293CD7C1058:00000293CD7C20F8 0000029241e4c730 0 410 875 175c 00000296d79d8370 1029220 Preemptive 000002944DF325F0:000002944DF33DF8 0000029241e4c730 0 411 635 166c 00000296d79d7ba0 1029220 Preemptive 000002924D2FECB0:000002924D3003D8 0000029241e4c730 0 412 748 1cfc 00000296d79d5490 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 XXXX 619 0 00000296ed3fbe80 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 413 876 38f8 00000296ed400ca0 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 414 874 db8 00000296ed3f9770 1029220 Preemptive 00000293CE18B010:00000293CE18CF98 0000029241e4c730 0 415 868 4250 00000296ed401470 1029220 Preemptive 00000292CD9F8270:00000292CD9F8CA0 0000029241e4c730 0 416 865 1668 00000296f9bc5870 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 417 656 35b8 00000296f9bc7f80 1029220 Preemptive 000002954C97E260:000002954C97E8B8 0000029241e4c730 0 418 837 3a78 00000296f9bc6fe0 1029220 Preemptive 00000293CE7B7DC8:00000293CE7B9450 0000029241e4c730 0 419 713 3bf0 00000296d79cf6d0 1029220 Preemptive 000002934C992A70:000002934C992AA8 00000296d4566ef0 0 420 681 39d4 00000296d79d1610 1029220 Preemptive 00000292CD684230:00000292CD684E28 0000029241e4c730 0 XXXX 873 0 00000296ed130c40 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 421 869 2258 00000296ed131410 1029220 Preemptive 000002924D1833A0:000002924D184F20 0000029241e4c730 0 422 864 3880 00000296ed133b20 1029220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 423 223 4170 00000296ed401c40 1029220 Preemptive 00000293CD8B8600:00000293CD8B9210 0000029241e4c730 0 424 324 2a04 00000296ed3fd5f0 3029220 Preemptive 000002944AFD2590:000002944AFD31D8 00000296d2b000b0 1 425 334 1e88 00000296ed402410 1029220 Preemptive 00000293CE3522D8:00000293CE353638 0000029241e4c730 0 426 777 31f4 00000296ed3fb6b0 1029220 Preemptive 0000029348917098:0000029348917430 00000296d2b000b0 0 427 667 1c1c 00000296d2466d60 1029220 Preemptive 00000294CCE4B390:00000294CCE4D130 0000029241e4c730 0 428 659 c80 00000296d79d9310 1029220 Preemptive 00000293CCAD5B70:00000293CCAD75C0 0000029241e4c730 0 429 603 2290 00000296ed12ed00 1029220 Preemptive 000002954D6CB688:000002954D6CB688 00000296d4566ef0 0 430 526 235c 00000296ed12ae80 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 432 487 3670 00000296e3a3bdc0 1029220 Preemptive 000002944E932020:000002944E932878 0000029241e4c730 0 431 456 3a5c 00000296f9bc77b0 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 433 55 fdc 00000296d2462710 1029220 Preemptive 00000294CFBA1930:00000294CFBA1A10 00000296d4566ef0 0 434 125 3544 00000296d2461770 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 XXXX 199 0 00000296d79d0670 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 435 550 3a30 00000296f9bc8750 1029220 Preemptive 00000292CABD5AC8:00000292CABD5AC8 00000296d2b000b0 0 436 551 1ff4 00000296d245f830 3029220 Preemptive 00000295CB6898B0:00000295CB68A058 00000296d2b000b0 0 437 553 1a98 00000296f0c87070 1029220 Preemptive 000002944DB6BF08:000002944DB6C2C0 0000029241e4c730 0 XXXX 576 0 00000296d2467530 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 438 578 c04 00000296f0c85130 1029220 Preemptive 00000293CC0F1E90:00000293CC0F23C8 0000029241e4c730 0 439 595 22b0 00000296d2460000 1029220 Preemptive 00000293CC39EAF0:00000293CC39F120 0000029241e4c730 0 XXXX 610 0 00000296f49a2010 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 440 791 3044 00000296e3a36fa0 3029220 Preemptive 00000294CC20BFF8:00000294CC20D218 00000296d2b000b0 0 441 646 bcc 00000296ed12be20 1029220 Preemptive 000002954D252F10:000002954D253368 0000029241e4c730 0 442 392 3a68 00000296e3a37770 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 2 443 832 43a0 00000296d79d44f0 1029220 Preemptive 000002934BC8D468:000002934BC8D598 0000029241e4c730 0 XXXX 870 0 00000296e3a3ae20 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 444 820 f08 00000296ecac51d0 1029220 Preemptive 00000295CE63DF90:00000295CE63EBD0 0000029241e4c730 0 445 815 2d0c 00000296ecabfbe0 1029220 Preemptive 000002954C918700:000002954C919308 0000029241e4c730 0 446 812 3fec 00000296ed3f8000 1029220 Preemptive 00000292CCC52E18:00000292CCC53158 0000029241e4c730 0 447 64 1d58 00000296f49a3780 1029220 Preemptive 00000292CD9727F8:00000292CD972A68 0000029241e4c730 0 448 66 904 00000296ed4089a0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 1 449 153 297c 00000296ed406a60 1029220 Preemptive 00000294CFB80F00:00000294CFB819B0 0000029241e4c730 0 450 151 21c0 00000296f0c82a20 1029220 Preemptive 00000292CDAB1E48:00000292CDAB34C8 0000029241e4c730 0 XXXX 154 0 00000296f49a3f50 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 451 142 1524 00000296f4c2e840 1029220 Preemptive 000002934894D400:000002934894D430 00000296d2b000b0 0 452 144 334c 00000296f49f6e10 1029220 Preemptive 000002944E4AFF88:000002944E4B1CF8 0000029241e4c730 0 453 159 d5c 00000296e3a3fc40 1029220 Preemptive 000002944943D6D0:000002944943DBE0 00000296d4566ef0 0 454 162 3524 00000296f0c831f0 1029220 Preemptive 00000293CD81A7E0:00000293CD81B678 0000029241e4c730 0 455 163 dd4 00000296e3a41b80 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 456 138 1878 00000296ecac59a0 3029220 Preemptive 00000294CC155098:00000294CC1559C8 00000296d2b000b0 0 457 172 375c 00000296ed12dd60 1029220 Preemptive 00000292CD0306F8:00000292CD030828 0000029241e4c730 0 458 169 1e1c 00000296ecabe470 1029220 Preemptive 000002934B8511D8:000002934B8516E0 0000029241e4c730 0 459 174 190c 00000296f49fc400 1029220 Preemptive 00000293CE42FA10:00000293CE42FA58 0000029241e4c730 0 460 185 35fc 00000296ed129ee0 1029220 Preemptive 000002954D142828:000002954D144208 0000029241e4c730 0 461 192 2ce4 00000296f49f56a0 1029220 Preemptive 000002954CE4D7F8:000002954CE4E080 0000029241e4c730 0 462 195 15a8 00000296f49f9520 1029220 Preemptive 000002924C7F0A38:000002924C7F2920 0000029241e4c730 0 463 200 2994 00000296ed134ac0 1029220 Preemptive 000002924D081DA0:000002924D0824F0 0000029241e4c730 0 464 204 1058 00000296ed1f9070 1029220 Preemptive 00000295CE6630A8:00000295CE664690 0000029241e4c730 0 465 208 1948 00000296ed40a8e0 1029220 Preemptive 000002954CA435D0:000002954CA44E08 0000029241e4c730 0 466 212 36c4 00000296ed40b0b0 1029220 Preemptive 000002944E923968:000002944E924878 0000029241e4c730 0 XXXX 217 0 00000296ed404b20 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 467 222 1168 00000296ed405ac0 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 468 226 24a4 00000296ecabec40 1029220 Preemptive 000002944D95A0B0:000002944D95BAE8 0000029241e4c730 0 XXXX 229 0 00000296ecac1b20 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 469 230 17f8 00000296f0c8b6c0 1029220 Preemptive 000002944E818148:000002944E819228 0000029241e4c730 0 470 231 720 00000296ed1342f0 1029220 Preemptive 00000294CD7E79F8:00000294CD7E7C10 0000029241e4c730 0 XXXX 232 0 00000296f4c2ffb0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 472 234 32a8 00000296ecac03b0 1029220 Preemptive 000002954D2A5598:000002954D2A7368 0000029241e4c730 0 471 237 3594 00000296f0c8fd10 1029220 Preemptive 000002924D3D0278:000002924D3D2078 0000029241e4c730 0 473 240 818 00000296f4c2e070 1029220 Preemptive 000002944A03EC70:000002944A03ECE8 00000296d2b000b0 0 474 241 10e4 00000296d1559d70 1029220 Preemptive 000002954D6FB280:000002954D6FB298 00000296d4566ef0 0 475 242 2870 00000296d64b8570 1029220 Preemptive 00000292CE2335A8:00000292CE234CD0 0000029241e4c730 0 476 246 2e9c 00000296d64be330 1029220 Preemptive 000002934C4FC090:000002934C4FDD98 0000029241e4c730 0 XXXX 247 0 00000296ecac3290 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 477 252 3480 00000296d155d420 1029220 Preemptive 00000293CE5B2390:00000293CE5B2BA0 0000029241e4c730 0 478 256 11fc 00000296ed2024e0 1029220 Preemptive 00000295CE511968:00000295CE512470 0000029241e4c730 0 XXXX 258 0 00000296d155f360 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 479 259 2fac 00000296ecabf410 1029220 Preemptive 00000292CBFD7D18:00000292CBFD8678 0000029241e4c730 0 XXXX 261 0 00000296d155ad10 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 480 262 6b0 00000296d1560300 1029220 Preemptive 000002944E780C68:000002944E782978 0000029241e4c730 0 481 263 1694 00000296d15595a0 1029220 Preemptive 00000293CC259FB0:00000293CC25A128 00000296d4566ef0 0 482 264 3214 00000296ecacd6a0 1029220 Preemptive 00000292CD124F00:00000292CD126390 0000029241e4c730 0 483 271 3ad0 00000296d1562240 1029220 Preemptive 00000295CD44CD40:00000295CD44D5D8 0000029241e4c730 0 484 277 7e4 00000296f7f64850 1029220 Preemptive 00000292CDE444A8:00000292CDE44998 0000029241e4c730 0 485 280 d10 00000296f49f5e70 1029220 Preemptive 00000294CFA3CA40:00000294CFA3D040 0000029241e4c730 0 486 281 2610 00000296d155bcb0 1029220 Preemptive 00000292CD97F4E8:00000292CD980A68 0000029241e4c730 0 487 286 34d4 00000296d155c480 1029220 Preemptive 000002934CCF0410:000002934CCF1888 0000029241e4c730 0 488 290 3dbc 00000296d155b4e0 1029220 Preemptive 000002944D390E80:000002944D391F68 0000029241e4c730 0 489 293 1920 00000296d1567060 1029220 Preemptive 000002954D6E08A0:000002954D6E13E8 0000029241e4c730 0 491 300 1e90 00000296d15639b0 1029220 Preemptive 00000295CE5A7D40:00000295CE5A8470 0000029241e4c730 0 490 304 354c 00000296f7f67730 1029220 Preemptive 000002954B0125D8:000002954B0125E8 00000296d2b000b0 0 492 306 64c 00000296f15b40c0 1029220 Preemptive 000002944CCED0B0:000002944CCEE2E0 0000029241e4c730 0 493 307 39bc 00000296ed20a9b0 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 494 310 3ab8 00000296d64bbc20 1029220 Preemptive 000002954BE9D0D8:000002954BE9D390 0000029241e4c730 0 495 312 250c 00000296f173efb0 1029220 Preemptive 00000294CF4C1B50:00000294CF4C2B98 0000029241e4c730 0 496 314 2c58 00000296ed40b880 1029220 Preemptive 00000292CDAAE4D0:00000292CDAAF4C8 0000029241e4c730 0 XXXX 320 0 00000296ed204420 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 497 321 1cf0 00000296ed4081d0 1029220 Preemptive 000002944DC740B8:000002944DC75F00 0000029241e4c730 0 499 325 1c8c 00000296d15658f0 1029220 Preemptive 0000029347D11B88:0000029347D125E8 0000029241e4c730 0 498 326 24e4 00000296ed404350 1029220 Preemptive 000002954D6EF5E0:000002954D6F1298 0000029241e4c730 0 500 327 3614 00000296ed203480 1029220 Preemptive 00000295CE66C8B8:00000295CE66E540 0000029241e4c730 0 501 329 29a0 00000296ed40df90 1029220 Preemptive 000002954D1BB008:000002954D1BCAB8 0000029241e4c730 0 502 330 290c 00000296ed40ef30 1029220 Preemptive 00000293CC9FCAA0:00000293CC9FCBE0 0000029241e4c730 0 503 331 2158 00000296d64b7da0 1029220 Preemptive 0000000000000000:0000000000000000 00000296d4566ef0 0 504 336 35e0 00000296ed4125e0 1029220 Preemptive 000002924C941510:000002924C941FF0 0000029241e4c730 0 505 342 d28 00000296f1929f70 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 506 347 e4c 00000296ed411e10 3029220 Preemptive 00000295499171C0:0000029549917BC8 00000296d2b000b0 0 507 353 27e0 00000296d1566890 1029220 Preemptive 000002954D6CD700:000002954D6CF688 0000029241e4c730 0 508 356 1df8 00000296ecad1cf0 1029220 Preemptive 00000295CD900A20:00000295CD900C68 0000029241e4c730 0 509 358 e44 00000296ecad24c0 1029220 Preemptive 00000293CE2D6E38:00000293CE2D7818 0000029241e4c730 0 510 359 41bc 00000296d2467d00 1029220 Preemptive 000002934C7B5C88:000002934C7B6168 0000029241e4c730 0 511 360 13e8 00000296d2461f40 1029220 Preemptive 00000295CE644138:00000295CE644A80 0000029241e4c730 0 512 363 1db0 00000296d2460fa0 1029220 Preemptive 00000292CDAA4AA0:00000292CDAA54C8 0000029241e4c730 0 513 530 31b0 00000296d246bb80 1029220 Preemptive 000002924D06D548:000002924D06E4F0 0000029241e4c730 0 514 596 1ee8 00000296d246fa00 1029220 Preemptive 0000029449E1C5F8:0000029449E1CE48 0000029241e4c730 0 516 716 1720 00000296ed413580 1029220 Preemptive 00000295CDE727E0:00000295CDE73370 0000029241e4c730 0 515 643 10c0 00000296d246a410 1029220 Preemptive 00000292CCE6EA08:00000292CCE70500 0000029241e4c730 0 XXXX 684 0 00000296d64bfaa0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 518 747 3bc8 00000296d156f530 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 517 757 3624 00000296ed40f700 1029220 Preemptive 00000294CFA3DA78:00000294CFA3F040 0000029241e4c730 0 519 760 1888 00000296ed40c820 1029220 Preemptive 000002954D56BFE0:000002954D56C528 0000029241e4c730 0 520 766 38b0 00000296d156fd00 1029220 Preemptive 00000292CDB625B0:00000292CDB63A70 0000029241e4c730 0 521 767 1110 00000296d1570ca0 1029220 Preemptive 000002944E9404A8:000002944E9405D8 0000029241e4c730 0 522 768 1ca8 00000296d1571470 1029220 Preemptive 000002944E95CD88:000002944E95E1E8 0000029241e4c730 0 XXXX 769 0 00000296f4a8d660 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 523 800 3ae8 00000296ed413d50 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 524 807 1f50 00000296ed411640 1029220 Preemptive 00000292CE22D0D8:00000292CE22ECD0 0000029241e4c730 0 XXXX 790 0 0000029240ef5230 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 805 0 00000296f49b77c0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 540 793 31f0 00000296f4a93bf0 1029220 Preemptive 00000294CFAC1720:00000294CFAC33A0 0000029241e4c730 0 XXXX 799 0 00000296f4a91cb0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 794 0 00000296ed535210 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 534 802 3054 00000296f49b6050 1029220 Preemptive 000002934C32F210:000002934C331190 0000029241e4c730 0 528 811 32a4 00000296f4a92c50 1029220 Preemptive 00000293CDA634B0:00000293CDA63AB0 0000029241e4c730 0 527 779 17b0 00000296f0ca4f00 1029220 Preemptive 0000029247D8F908:0000029247D90508 00000296d4566ef0 0 535 797 1824 00000296f4844730 1029220 Preemptive 00000292CE74EE80:00000292CE750E40 0000029241e4c730 0 536 780 3b7c 00000296f4842020 1029220 Preemptive 000002954D271970:000002954D273368 0000029241e4c730 0 537 806 3838 00000296f19630c0 1029220 Preemptive 00000292CB47F480:00000292CB4813E0 0000029241e4c730 0 529 383 355c 00000296d64b8d40 1029220 Preemptive 000002934C992BC8:000002934C994AA8 0000029241e4c730 0 525 368 1118 00000296f1964060 1029220 Preemptive 000002934C636158:000002934C6366A8 0000029241e4c730 0 526 393 20d0 00000296d64c0270 1029220 Preemptive 00000294CEB42F00:00000294CEB444C0 0000029241e4c730 0 530 796 4390 00000296f4a8de30 1029220 Preemptive 00000293CEC01510:00000293CEC03450 0000029241e4c730 0 531 621 1248 00000296d64c0a40 1029220 Preemptive 00000293CE34E880:00000293CE34F638 0000029241e4c730 0 532 639 2718 00000296f7990770 1029220 Preemptive 000002944DB67550:000002944DB682C0 0000029241e4c730 0 533 722 303c 00000296ecacaf90 1029220 Preemptive 000002944CED9060:000002944CED92D8 0000029241e4c730 0 XXXX 630 0 00000296ed20a1e0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 538 617 36d8 00000296ebe01d80 1029220 Preemptive 00000295CD732E18:00000295CD734198 0000029241e4c730 0 XXXX 726 0 0000029200f72060 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 539 727 3c28 00000296ed20b950 1029220 Preemptive 000002944E5FB930:000002944E5FC7D0 0000029241e4c730 0 XXXX 756 0 0000029200f75ee0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 541 754 4078 00000296ed20d0c0 1029220 Preemptive 000002954D1865A8:000002954D188208 0000029241e4c730 0 542 752 3fbc 00000296ed205b90 1029220 Preemptive 000002934C62FB50:000002934C630678 0000029241e4c730 0 543 751 aec 00000296ed207ad0 3029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 XXXX 724 0 00000296ed20d890 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 750 0 00000296f1f33090 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 544 749 37bc 00000296f1f36f10 1029220 Preemptive 00000295CE65FC98:00000295CE660690 0000029241e4c730 0 545 745 3e70 00000296f1f36740 3029220 Preemptive 00000292CE791328:00000292CE7915A0 00000296d4566ef0 0 XXXX 744 0 00000296f1f34800 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 546 741 3924 00000296f1f33860 1029220 Preemptive 00000293CDAFA3B0:00000293CDAFBB00 0000029241e4c730 0 547 740 21a8 00000296ed206360 1029220 Preemptive 00000292CE23F358:00000292CE240CD0 0000029241e4c730 0 548 739 2c28 0000029200f78dc0 1029220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 637 0 00000296ed20e060 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 550 736 3cd8 00000296d64b9510 1029220 Preemptive 000002944E7855D0:000002944E786978 0000029241e4c730 0 549 734 3764 00000296ed207300 1029220 Preemptive 000002924C8869E8:000002924C887FF0 0000029241e4c730 0 551 733 3378 0000029200f79d60 1029220 Preemptive 00000295485B6A68:00000295485B7320 00000296d4566ef0 0 552 732 2be8 00000296ed414520 1029220 Preemptive 000002954D253F98:000002954D255368 0000029241e4c730 0 XXXX 731 0 00000296d64c77a0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 730 0 00000296ecad3460 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 553 623 3f14 00000296ecad9220 1029220 Preemptive 000002934C6B6C18:000002934C6B8168 0000029241e4c730 0 554 715 3f40 00000296ed20e830 1029220 Preemptive 000002924D3C5AB0:000002924D3C61C8 0000029241e4c730 0 555 721 3a60 00000296ed412db0 1029220 Preemptive 0000029449E61E70:0000029449E62908 0000029241e4c730 0 XXXX 719 0 00000296f1f357a0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 556 717 3638 00000296ecad2c90 1029220 Preemptive 0000000000000000:0000000000000000 00000296d2b000b0 0 561 714 39ec 00000296d64c40f0 1029220 Preemptive 000002954D127FD8:000002954D128358 0000029241e4c730 0 557 712 3ca8 00000296ed215d60 1029220 Preemptive 000002934B8038E0:000002934B803B08 0000029241e4c730 0 558 636 1894 00000296ed4154c0 1029220 Preemptive 000002954D6D6198:000002954D6D7538 0000029241e4c730 0 XXXX 710 0 0000029200f7a530 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 559 708 1d2c 00000296ed216d00 1029220 Preemptive 00000295CDE654F0:00000295CDE674C0 0000029241e4c730 0 560 707 3a10 00000296f1f3a5c0 1029220 Preemptive 000002934C298318:000002934C299268 0000029241e4c730 0 XXXX 706 0 00000296ecad8a50 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 562 705 2c44 00000296ecada1c0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 XXXX 697 0 00000296e90fdbe0 1039820 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 563 615 448 00000296d64deea0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 566 860 1f6c 00000296d64c48c0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 565 872 4144 00000296f9382d10 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 567 147 2e28 00000296d64df670 8029220 Preemptive 00000293CEC61CD0:00000293CEC639D0 0000029241e4c730 0 568 866 1510 00000296e9107ff0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 569 181 26a8 00000296e9101290 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 570 184 2f8c 00000296e91031d0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 571 197 3e74 00000296e91039a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 572 804 ce4 00000296ed20c8f0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 574 213 1d14 00000296e90fd410 8029220 Preemptive 00000293CBC00148:00000293CBC00E40 0000029241e4c730 0 573 683 3c38 00000296d64c21b0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 575 284 288c 00000296f9385420 1029220 Preemptive 000002954C97BCC0:000002954C97C8B8 0000029241e4c730 0 576 592 41c8 00000296d246abe0 1029220 Preemptive 000002924D1E6FC8:000002924D1E7548 0000029241e4c730 0 577 682 f50 00000296f9380dd0 1029220 Preemptive 000002944EA7E7B8:000002944EA7FCD8 0000029241e4c730 0 579 680 1b64 00000296d24636b0 1029220 Preemptive 00000293CDAE7B90:00000293CDAE7DA0 0000029241e4c730 0 578 679 4090 00000296ed221110 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 580 867 22b4 00000296e9107820 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 581 673 2108 00000296ed21c2f0 1029220 Preemptive 000002944D8A3B28:000002944D8A5AE8 0000029241e4c730 0 582 672 2304 00000296eb845990 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 583 671 26e8 00000296ed2220b0 1029220 Preemptive 00000294CFB43BB0:00000294CFB45100 0000029241e4c730 0 584 669 2938 00000296f9388ad0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 585 676 3a9c 00000296f937f660 1029220 Preemptive 000002934B9AEDF0:000002934B9B03E8 0000029241e4c730 0 586 668 b20 00000296eb846930 1029220 Preemptive 000002944DB979D8:000002944DB97ED0 0000029241e4c730 0 587 666 2058 00000296f93892a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 588 664 2ddc 00000296f9387360 1029220 Preemptive 000002934C4D84A8:000002934C4DA188 0000029241e4c730 0 589 663 3d10 00000296f937ee90 1029220 Preemptive 000002954CB8FF40:000002954CB911D8 0000029241e4c730 0 590 662 1db4 00000296f93815a0 1029220 Preemptive 00000293CE2CE710:00000293CE2CF968 0000029241e4c730 0 591 661 c0 00000296f9380600 1029220 Preemptive 00000295CDE780C0:00000295CDE79220 0000029241e4c730 0 592 660 26f8 00000296f93863c0 1029220 Preemptive 000002924C08C7D0:000002924C08D308 0000029241e4c730 0 593 634 2954 00000296f937e6c0 1029220 Preemptive 00000293CDAD21E8:00000293CDAD3EC0 0000029241e4c730 0 594 629 17f0 00000296f9384c50 1029220 Preemptive 00000292CD68DF98:00000292CD68ECD8 0000029241e4c730 0 595 657 123c 00000296f9385bf0 1029220 Preemptive 00000294CFF3D300:00000294CFF3E4D0 0000029241e4c730 0 596 655 1d94 00000296f9387b30 1029220 Preemptive 000002924D300F38:000002924D3023D8 0000029241e4c730 0 598 862 19f4 00000296fb2225a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 597 654 3be4 00000296f9388300 1029220 Preemptive 000002954D1B6FC0:000002954D1B8AB8 0000029241e4c730 0 599 653 e80 00000296f9381d70 1029220 Preemptive 000002954B08C0D8:000002954B08C1F8 00000296d2b000b0 0 600 626 2774 00000296fb21b070 1029220 Preemptive 000002924CF84598:000002924CF863E0 0000029241e4c730 0 601 631 4114 00000296f9389a70 1029220 Preemptive 00000295CDFFD420:00000295CDFFEF80 0000029241e4c730 0 602 649 3d08 00000296fb222d70 1029220 Preemptive 00000292CDAA6CF8:00000292CDAA74C8 0000029241e4c730 0 603 648 abc 00000296f9382540 1029220 Preemptive 00000294CD795540:00000294CD7959E0 0000029241e4c730 0 604 640 3e40 00000296fb21b840 1029220 Preemptive 000002934C659690:000002934C65A168 0000029241e4c730 0 605 647 1d18 00000296f938a240 1029220 Preemptive 000002954CB92558:000002954CB931D8 0000029241e4c730 0 606 612 19c8 00000296fb220e30 1029220 Preemptive 00000293CE425C98:00000293CE427BA8 0000029241e4c730 0 607 606 136c 00000296f938d8f0 1029220 Preemptive 00000293CF54AFD0:00000293CF54AFE0 00000296f7775c60 1 608 598 3e90 00000296fb223540 1029220 Preemptive 000002924D42A058:000002924D42A078 00000296d4566ef0 0 610 593 2770 00000296f938c950 1029220 Preemptive 000002924BB0AFB8:000002924BB0BEA0 0000029241e4c730 0 609 585 3c0c 00000296fb21eef0 1029220 Preemptive 000002954C9D9D18:000002954C9DAE08 0000029241e4c730 0 611 590 2c34 00000296fb21df50 1029220 Preemptive 00000294D0051240:00000294D0051240 00000296d4566ef0 0 612 588 3774 00000296fb21c010 1029220 Preemptive 000002944E635348:000002944E635FF0 0000029241e4c730 0 613 584 2278 00000296f938aa10 1029220 Preemptive 00000294CFAD3578:00000294CFAD5100 0000029241e4c730 0 626 589 13d8 00000296fb223d10 1029220 Preemptive 00000292CDAA8448:00000292CDAA94C8 0000029241e4c730 0 629 587 24ac 00000296f938b1e0 1029220 Preemptive 00000294D0029D50:00000294D002A990 0000029241e4c730 0 616 586 35d8 00000296fb21c7e0 1029220 Preemptive 000002924D3B7528:000002924D3B8468 0000029241e4c730 0 637 583 3430 00000296e91087c0 1029220 Preemptive 000002954C914360:000002954C915308 0000029241e4c730 0 630 581 4190 00000296f938b9b0 1029220 Preemptive 000002924B9D5750:000002924B9D6740 0000029241e4c730 0 631 577 2920 00000296f938c180 1029220 Preemptive 000002954B059C28:000002954B05A348 0000029241e4c730 0 682 580 43e0 00000296eb849fe0 1029220 Preemptive 00000293CCA0A7B0:00000293CCA0ABE0 0000029241e4c730 0 632 574 3bf4 00000296f938e0c0 1029220 Preemptive 000002924D0D6BC8:000002924D0D84F0 0000029241e4c730 0 638 573 1e5c 00000296e910b6a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 622 571 273c 00000296fb21f6c0 1029220 Preemptive 000002944E847478:000002944E849288 0000029241e4c730 0 634 569 381c 00000296f938d120 1029220 Preemptive 00000292CDAA9F80:00000292CDAAB4C8 0000029241e4c730 0 635 567 391c 00000296fb2244e0 1029220 Preemptive 00000292CE75EB40:00000292CE75EE40 0000029241e4c730 0 653 566 34f8 00000296f7f59e30 1029220 Preemptive 000002954C8F7870:000002954C8F9050 0000029241e4c730 0 660 564 1f38 00000296fb21fe90 1029220 Preemptive 000002934BD2EF30:000002934BD2F1A8 0000029241e4c730 0 654 563 1f90 00000296f7f54070 1029220 Preemptive 00000293CD8D3900:00000293CD8D3958 0000029241e4c730 0 661 562 42dc 00000296fb224cb0 1029220 Preemptive 000002944E8A37E0:000002944E8A5288 0000029241e4c730 0 656 561 31a8 00000296f7f5d4e0 1029220 Preemptive 000002924AFD21C8:000002924AFD22D8 00000296d2b000b0 0 663 559 3a94 00000296fb21d780 1029220 Preemptive 00000295CDF58898:00000295CDF59220 0000029241e4c730 0 657 558 3ac8 00000296f7f5e480 1029220 Preemptive 00000292CD1179A0:00000292CD118630 0000029241e4c730 0 664 557 2b3c 00000296fb225480 1029220 Preemptive 000002924C7A9CC0:000002924C7AA838 0000029241e4c730 0 658 555 1434 00000296f7f5dcb0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 665 552 3294 00000296fb22a2a0 1029220 Preemptive 00000294CB8A98A0:00000294CB8A9C80 00000296d2b000b0 0 667 544 1680 00000296fb226420 1029220 Preemptive 000002944E633818:000002944E633FF0 0000029241e4c730 0 659 543 2adc 00000296f7f5ec50 1029220 Preemptive 00000292CCD41680:00000292CCD41AE0 0000029241e4c730 0 668 542 408c 00000296fb226bf0 1029220 Preemptive 000002944DF51A48:000002944DF53A08 0000029241e4c730 0 669 541 35ec 00000296fb225c50 1029220 Preemptive 00000294CF54AF98:00000294CF54B0B0 0000029241e4c730 0 670 540 3b88 00000296fb2273c0 1029220 Preemptive 000002954BE9FC38:000002954BEA1390 0000029241e4c730 0 636 539 2298 00000296ed208a70 1029220 Preemptive 000002934A7772B8:000002934A7773B0 00000296d4566ef0 0 671 538 41d8 00000296fb229ad0 1029220 Preemptive 00000295CD4C7CB0:00000295CD4C9638 0000029241e4c730 0 624 536 18a4 00000296fb227b90 1029220 Preemptive 00000292CE23BB70:00000292CE23CCD0 0000029241e4c730 0 639 532 2d50 00000296ed209a10 1029220 Preemptive 00000295CDFF7AC0:00000295CDFF8F80 0000029241e4c730 0 625 528 19cc 00000296fb228360 1029220 Preemptive 000002944CCCB098:000002944CCCC580 0000029241e4c730 0 640 527 30a0 00000296ed21ab80 1029220 Preemptive 00000293CE384EA0:00000293CE385718 0000029241e4c730 0 641 525 3794 00000296ed21b350 1029220 Preemptive 000002924CF82968:000002924CF843E0 0000029241e4c730 0 642 521 3124 00000296ed219410 1029220 Preemptive 00000292CDAAC828:00000292CDAAD4C8 0000029241e4c730 0 643 516 2640 00000296ed2218e0 1029220 Preemptive 00000294CFEE2B00:00000294CFEE48C0 0000029241e4c730 0 673 513 2bf0 00000296eb8480a0 3029220 Preemptive 0000029449E04D48:0000029449E04E48 00000296d2b000b0 0 644 512 4160 00000296ed21cac0 1029220 Preemptive 000002934C6BA338:000002934C6BC168 0000029241e4c730 0 674 511 31c4 00000296eb84c6f0 1029220 Preemptive 00000292CE235810:00000292CE236CD0 0000029241e4c730 0 645 510 1adc 00000296ed2295e0 1029220 Preemptive 000002944DF56510:000002944DF57A08 0000029241e4c730 0 675 509 39f0 00000296eb84bf20 1029220 Preemptive 00000293CC0B45D0:00000293CC0B4A58 0000029241e4c730 0 646 508 12e0 00000296ed22bcf0 1029220 Preemptive 00000294CFFDA630:00000294CFFDC4D0 0000029241e4c730 0 676 506 1820 00000296eb848870 1029220 Preemptive 000002934C6BCAB0:000002934C6BE168 0000029241e4c730 0 647 503 308 00000296ed22b520 1029220 Preemptive 000002944E847268:000002944E847288 00000296d4566ef0 0 677 502 1d88 00000296d247bd50 1029220 Preemptive 000002954CA54778:000002954CA54E38 0000029241e4c730 0 648 501 24e8 00000296ed229db0 1029220 Preemptive 000002934C740460:000002934C742168 0000029241e4c730 0 678 500 1ce4 00000296d2477ed0 1029220 Preemptive 000002944E95B3F0:000002944E95C1E8 0000029241e4c730 0 649 499 1b48 00000296ed22cc90 1029220 Preemptive 000002944D359618:000002944D35A208 0000029241e4c730 0 679 498 1df0 00000296d2476760 1029220 Preemptive 00000295CEB17C98:00000295CEB17C98 00000296d4566ef0 0 650 495 1f70 00000296ed22c4c0 1029220 Preemptive 00000292CACA78B8:00000292CACA7C18 00000296d2b000b0 0 680 494 3c34 00000296d2473880 1029220 Preemptive 000002924D30AD48:000002924D30C3D8 0000029241e4c730 0 651 491 322c 00000296ed2276a0 1029220 Preemptive 000002934C4C4D68:000002934C4C63F8 0000029241e4c730 0 681 490 1838 00000296d247adb0 3029220 Preemptive 000002944A283D78:000002944A285B40 00000296d2b000b0 0 662 675 2d08 00000296d64da080 1029220 Preemptive 000002934CCFC9A8:000002934CCFD5E8 0000029241e4c730 0 652 489 3e0c 00000296ed22dc30 1029220 Preemptive 0000029349625E30:0000029349626060 0000029241e4c730 0 683 486 2934 00000296d24786a0 1029220 Preemptive 000002954C94DCB8:000002954C94EDF8 0000029241e4c730 0 666 482 3364 00000296ed226700 1029220 Preemptive 00000294CFACE950:00000294CFACF100 0000029241e4c730 0 684 481 2c14 00000296d24730b0 1029220 Preemptive 000002934CCF7638:000002934CCF7738 0000029241e4c730 0 685 477 17bc 00000296d2479640 1029220 Preemptive 000002954D185A20:000002954D186208 0000029241e4c730 0 614 476 3ef8 00000296d2476f30 1029220 Preemptive 00000293CAECD0F0:00000293CAECE150 0000029241e4c730 0 615 475 2cb4 00000296d2475f90 1029220 Preemptive 00000295CD532C30:00000295CD532FD8 0000029241e4c730 0 617 474 1764 00000296d2472110 1029220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 618 472 4288 00000296d2471940 1029220 Preemptive 00000292CAD19928:00000292CAD1B080 0000029241e4c730 0 619 471 1a3c 00000296d2478e70 1029220 Preemptive 00000293CE424458:00000293CE425BA8 0000029241e4c730 0 620 469 c58 00000296d2474050 1029220 Preemptive 000002934CAD8958:000002934CAD99A8 0000029241e4c730 0 621 466 2998 00000296fb229300 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 623 465 3e9c 00000296e90dd840 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 627 463 34bc 00000296ed226ed0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 628 462 3890 00000296ed22ebd0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 633 677 1b20 00000296ed21bb20 1029220 Preemptive 000002934CD89328:000002934CD89348 00000296d4566ef0 0 655 461 644 00000296f7f5f420 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 672 458 341c 00000296d64f5dd0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 686 454 1fc0 00000296e90e1e90 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 687 451 204c 00000296ee595610 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 688 450 3f9c 00000296ed22f3a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 689 459 2bfc 00000296fccffe70 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 690 443 38f4 00000296f1f47500 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 691 448 40e4 00000296ecaaa420 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 692 446 1b0c 00000296fcd015e0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 693 445 1120 00000296ee595de0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 694 440 1df4 00000296ee5984f0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 695 438 3b18 00000296f1f3f030 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 696 437 3590 00000296fccfa0b0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 697 435 3848 00000296ed2370a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 698 434 2858 00000296d64e1d80 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 699 432 3a3c 00000296d15660c0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 700 431 399c 00000296ed2312e0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 701 430 1128 00000296ed231ab0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 702 429 3434 00000296e90e0720 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 703 427 25e4 00000296d155fb30 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 704 423 3744 00000296ecaa4660 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 705 422 1c4c 00000296f0c8be90 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 708 420 2b7c 00000296f1f43680 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 706 419 2288 00000296f1f49440 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 707 418 12cc 00000296ec64e6c0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 709 412 3ae4 00000296d64ea250 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 710 410 4084 00000296eccf90a0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 711 409 1bdc 00000296eccf9870 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 712 408 1fec 00000296d79cb850 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 713 407 3d24 00000296e90de7e0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 714 405 2378 00000296ed232a50 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 715 402 3b1c 00000296e90e8420 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 716 401 500 00000296e90ea360 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 717 287 1e50 00000296ec62deb0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 718 313 b8 00000296eecc4690 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 719 316 1c70 00000296e90ee9b0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 720 26 6b8 00000296d64e3cc0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 721 399 4080 00000296ee40dce0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 722 28 35dc 00000296e90f1890 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 723 33 1ab0 00000296f1f4a3e0 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 724 62 60c 00000296eecc5630 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 725 59 3c80 00000296eecc5e00 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 726 128 3bd8 0000029200fa6600 8039220 Preemptive 0000000000000000:0000000000000000 0000029241e4c730 0 Lock
Count Apt Exception
Ukn
MTA (Finalizer)
Ukn
MTA (Threadpool Worker)
Ukn
Ukn
Ukn (Threadpool Worker)
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA (Threadpool Worker)
MTA
MTA
MTA
MTA
MTA
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
MTA
MTA
MTA
Ukn
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
Ukn
MTA
Ukn
Ukn
Ukn
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA (Threadpool Worker)
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn (Threadpool Worker)
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
MTA
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn
Ukn
Ukn
Ukn
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker) System.TimeoutException 00000294d0054d58
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
Ukn
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Worker)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Worker)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
MTA (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
MTA (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
MTA (Threadpool Worker)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)
Ukn (Threadpool Completion Port)

      OSID Special thread type
   22 2120 DbgHelper 
   23 3300 ProfilingAPIDetach 
   31  a10 GC 
   32 17d4 GC 
   33  9c0 GC 
   34 2b04 GC 
   35 145c GC 
   36 3e1c GC 
   37 3920 GC 
   38 14dc GC 
   39 19e8 Finalizer 
   40  f18 ProfilingAPIAttach 
   43 13b4 Timer 
   44 1730 ADUnloadHelper 
   45 2384 Wait 
   56 2ef4 ThreadpoolWorker 
   62 2e4c GC 
   63  f48 GC 
   64 3078 GC 
   65 3784 GC 
   66 291c GC 
   67 403c GC 
   68 183c GC 
   69 20bc GC 
   75 2418 ThreadpoolWorker 
   76 2eb0 ThreadpoolWorker 
   77 1b24 ThreadpoolWorker 
  115 3398 ThreadpoolWorker 
  141 36dc Gate 
  154 2ca0 ThreadpoolWorker 
  155 1490 ThreadpoolWorker 
  156 3d18 ThreadpoolWorker 
  157 19b4 ThreadpoolWorker 
  158 2168 ThreadpoolWorker 
  159 2c7c ThreadpoolWorker 
  160 40b8 ThreadpoolWorker 
  161 2980 ThreadpoolWorker 
  162 1874 ThreadpoolWorker 
  163 3d40 ThreadpoolWorker 
  164 1538 ThreadpoolWorker 
  165 1ee0 ThreadpoolWorker 
  166 2350 ThreadpoolWorker 
  167 2d90 ThreadpoolWorker 
  168 27dc ThreadpoolWorker 
  169 3e48 ThreadpoolWorker 
  170 3598 ThreadpoolWorker 
  171  e30 ThreadpoolWorker 
  172  ab8 ThreadpoolWorker 
  173 1590 ThreadpoolWorker 
  174 24d4 ThreadpoolWorker 
  175 3150 ThreadpoolWorker 
  176 3cf4 ThreadpoolWorker 
  177 1d38 ThreadpoolWorker 
  178 13ac ThreadpoolWorker 
  179 1708 ThreadpoolWorker 
  180  c8c ThreadpoolWorker 
  183 1144 ThreadpoolWorker 
  184 2e2c ThreadpoolWorker 
  185  8ac ThreadpoolWorker 
  186 2a5c ThreadpoolWorker 
  187 28e0 ThreadpoolWorker 
  188 2f20 ThreadpoolWorker 
  189 21ac ThreadpoolWorker 
  190 18d8 ThreadpoolWorker 
  191 11e4 ThreadpoolWorker 
  192 3d70 ThreadpoolWorker 
  193  294 ThreadpoolWorker 
  194 1b88 ThreadpoolWorker 
  195 3540 ThreadpoolWorker 
  196 12a4 ThreadpoolWorker 
  197 3eb4 ThreadpoolWorker 
  198 315c ThreadpoolWorker 
  199 412c ThreadpoolWorker 
  200 2714 ThreadpoolWorker 
  201  9cc ThreadpoolWorker 
  202 15d0 ThreadpoolWorker 
  203 1d10 ThreadpoolWorker 
  204 2850 ThreadpoolWorker 
  205 15e4 ThreadpoolWorker 
  206 3d54 ThreadpoolWorker 
  207  e70 ThreadpoolWorker 
  208 1944 ThreadpoolWorker 
  209 3020 ThreadpoolWorker 
  210 3bd4 ThreadpoolWorker 
  211 3fa4 ThreadpoolWorker 
  212 1ddc ThreadpoolWorker 
  213 32d0 ThreadpoolWorker 
  214 382c ThreadpoolWorker 
  215  6ec ThreadpoolWorker 
  216 3f74 ThreadpoolWorker 
  217 3f4c ThreadpoolWorker 
  218 232c ThreadpoolWorker 
  219 26dc ThreadpoolWorker 
  220 2014 ThreadpoolWorker 
  221 200c ThreadpoolWorker 
  222 1858 ThreadpoolWorker 
  223 1638 ThreadpoolWorker 
  224 1704 ThreadpoolWorker 
  225 3dfc ThreadpoolWorker 
  226 1600 ThreadpoolWorker 
  227 1370 ThreadpoolWorker 
  228  ca0 ThreadpoolWorker 
  229  e24 ThreadpoolWorker 
  230 3610 ThreadpoolWorker 
  231 1d4c ThreadpoolWorker 
  232  650 ThreadpoolWorker 
  233 3064 ThreadpoolWorker 
  234 1e40 ThreadpoolWorker 
  235 1180 ThreadpoolWorker 
  236 3914 ThreadpoolWorker 
  237 2a88 ThreadpoolWorker 
  238 25f0 ThreadpoolWorker 
  239 4060 ThreadpoolWorker 
  240 1288 ThreadpoolWorker 
  241 20a4 ThreadpoolWorker 
  242 2cfc ThreadpoolWorker 
  243 2cb0 ThreadpoolWorker 
  244 141c ThreadpoolWorker 
  245 1660 ThreadpoolWorker 
  246 4380 ThreadpoolWorker 
  247 2b90 ThreadpoolWorker 
  248 2d88 ThreadpoolWorker 
  249 3eb8 ThreadpoolWorker 
  250 1324 ThreadpoolWorker 
  251 28bc ThreadpoolWorker 
  252 13bc ThreadpoolWorker 
  253 4284 IOCompletion 
  254 1eb4 ThreadpoolWorker 
  255  994 ThreadpoolWorker 
  256 14c4 ThreadpoolWorker 
  257 1018 ThreadpoolWorker 
  258 2ce8 ThreadpoolWorker 
  259 2ff8 ThreadpoolWorker 
  260 20e8 ThreadpoolWorker 
  261 34e8 ThreadpoolWorker 
  262 18e4 ThreadpoolWorker 
  263 1f5c ThreadpoolWorker 
  264 41cc ThreadpoolWorker 
  265 1420 ThreadpoolWorker 
  266  470 ThreadpoolWorker 
  267 3f6c ThreadpoolWorker 
  268 16d0 ThreadpoolWorker 
  269 2eac ThreadpoolWorker 
  270 34e0 ThreadpoolWorker 
  271 271c ThreadpoolWorker 
  273  488 ThreadpoolWorker 
  274 3f90 ThreadpoolWorker 
  275  8a4 ThreadpoolWorker 
  276 360c ThreadpoolWorker 
  277 1218 ThreadpoolWorker 
  278 3a98 ThreadpoolWorker 
  279 3a90 ThreadpoolWorker 
  280 38c8 ThreadpoolWorker 
  281  9a4 ThreadpoolWorker 
  282 2ed0 ThreadpoolWorker 
  283 2f94 ThreadpoolWorker 
  284 15e8 ThreadpoolWorker 
  285  bc4 ThreadpoolWorker 
  286 30d0 ThreadpoolWorker 
  287 1810 ThreadpoolWorker 
  288 10b0 ThreadpoolWorker 
  289 2050 ThreadpoolWorker 
  290  d7c ThreadpoolWorker 
  291 34fc ThreadpoolWorker 
  292 1190 ThreadpoolWorker 
  293 25bc ThreadpoolWorker 
  294 4030 ThreadpoolWorker 
  295 2e14 ThreadpoolWorker 
  296 1474 ThreadpoolWorker 
  297 2a68 ThreadpoolWorker 
  298  bb8 ThreadpoolWorker 
  299 2cf4 ThreadpoolWorker 
  300 1ea8 ThreadpoolWorker 
  301 1588 ThreadpoolWorker 
  302 16fc ThreadpoolWorker 
  303 3a70 ThreadpoolWorker 
  304 3da8 ThreadpoolWorker 
  305  324 ThreadpoolWorker 
  306 3228 ThreadpoolWorker 
  307 4120 ThreadpoolWorker 
  308  434 ThreadpoolWorker 
  309 437c ThreadpoolWorker 
  310 3a2c ThreadpoolWorker 
  311 30f0 ThreadpoolWorker 
  312 1bec ThreadpoolWorker 
  313 3b20 ThreadpoolWorker 
  314 1598 ThreadpoolWorker 
  315 3494 ThreadpoolWorker 
  316  80c ThreadpoolWorker 
  317 311c ThreadpoolWorker 
  318 1534 ThreadpoolWorker 
  319 2d6c ThreadpoolWorker 
  320 2e94 ThreadpoolWorker 
  321 187c ThreadpoolWorker 
  322 19a4 ThreadpoolWorker 
  323  e78 ThreadpoolWorker 
  324 1aa8 ThreadpoolWorker 
  325 3a08 ThreadpoolWorker 
  326 3fb8 ThreadpoolWorker 
  327 3140 ThreadpoolWorker 
  328 162c ThreadpoolWorker 
  329 2df0 ThreadpoolWorker 
  330 2914 ThreadpoolWorker 
  331 27fc ThreadpoolWorker 
  332  738 ThreadpoolWorker 
  333 16d4 ThreadpoolWorker 
  334 191c ThreadpoolWorker 
  335 2b98 ThreadpoolWorker 
  336 2f98 ThreadpoolWorker 
  337 2564 ThreadpoolWorker 
  338 1e0c ThreadpoolWorker 
  339 4050 ThreadpoolWorker 
  340 3684 ThreadpoolWorker 
  341 2c30 ThreadpoolWorker 
  342 1ad8 ThreadpoolWorker 
  343 3608 ThreadpoolWorker 
  344 1574 ThreadpoolWorker 
  345 2380 ThreadpoolWorker 
  346 3468 ThreadpoolWorker 
  347 2f44 ThreadpoolWorker 
  348  fa8 ThreadpoolWorker 
  349 431c ThreadpoolWorker 
  350  b44 ThreadpoolWorker 
  351 363c ThreadpoolWorker 
  352 43bc ThreadpoolWorker 
  353 15f8 ThreadpoolWorker 
  354 1250 ThreadpoolWorker 
  355 3548 ThreadpoolWorker 
  356  8e8 ThreadpoolWorker 
  357 1dd8 ThreadpoolWorker 
  358 154c ThreadpoolWorker 
  359 40c4 ThreadpoolWorker 
  360 374c ThreadpoolWorker 
  361 10c4 ThreadpoolWorker 
  362 35f0 ThreadpoolWorker 
  363 1d9c ThreadpoolWorker 
  364 379c ThreadpoolWorker 
  365 337c ThreadpoolWorker 
  366 169c ThreadpoolWorker 
  367 3ee4 ThreadpoolWorker 
  368 256c ThreadpoolWorker 
  369 2844 ThreadpoolWorker 
  370 13f0 ThreadpoolWorker 
  371 3de4 ThreadpoolWorker 
  372 3b84 ThreadpoolWorker 
  373 3368 ThreadpoolWorker 
  374 1868 ThreadpoolWorker 
  375 1a28 ThreadpoolWorker 
  376 197c ThreadpoolWorker 
  377 35ac ThreadpoolWorker 
  378  4f8 ThreadpoolWorker 
  379 281c ThreadpoolWorker 
  380  f40 ThreadpoolWorker 
  381 2764 ThreadpoolWorker 
  382  e48 ThreadpoolWorker 
  383 12b8 ThreadpoolWorker 
  384 2bd4 ThreadpoolWorker 
  385 3380 ThreadpoolWorker 
  386 12bc ThreadpoolWorker 
  387 33dc ThreadpoolWorker 
  388 3fac ThreadpoolWorker 
  389 3d48 ThreadpoolWorker 
  390 2408 ThreadpoolWorker 
  391  c44 ThreadpoolWorker 
  392 2e90 ThreadpoolWorker 
  393  804 ThreadpoolWorker 
  394 3004 ThreadpoolWorker 
  405 17c4 ThreadpoolWorker 
  406 3cc8 ThreadpoolWorker 
  407 39a0 ThreadpoolWorker 
  408 2fbc ThreadpoolWorker 
  409 2e80 ThreadpoolWorker 
  410 175c ThreadpoolWorker 
  411 166c ThreadpoolWorker 
  412 1cfc ThreadpoolWorker 
  413 38f8 ThreadpoolWorker 
  414  db8 ThreadpoolWorker 
  415 4250 ThreadpoolWorker 
  416 1668 ThreadpoolWorker 
  417 35b8 ThreadpoolWorker 
  418 3a78 ThreadpoolWorker 
  419 3bf0 ThreadpoolWorker 
  420 39d4 ThreadpoolWorker 
  421 2258 ThreadpoolWorker 
  422 3880 ThreadpoolWorker 
  423 4170 ThreadpoolWorker 
  424 2a04 ThreadpoolWorker 
  425 1e88 ThreadpoolWorker 
  426 31f4 ThreadpoolWorker 
  427 1c1c ThreadpoolWorker 
  428  c80 ThreadpoolWorker 
  429 2290 ThreadpoolWorker 
  430 235c ThreadpoolWorker 
  431 3a5c ThreadpoolWorker 
  432 3670 ThreadpoolWorker 
  433  fdc ThreadpoolWorker 
  434 3544 ThreadpoolWorker 
  435 3a30 ThreadpoolWorker 
  436 1ff4 ThreadpoolWorker 
  437 1a98 ThreadpoolWorker 
  438  c04 ThreadpoolWorker 
  439 22b0 ThreadpoolWorker 
  440 3044 ThreadpoolWorker 
  441  bcc ThreadpoolWorker 
  442 3a68 ThreadpoolWorker 
  443 43a0 ThreadpoolWorker 
  444  f08 ThreadpoolWorker 
  445 2d0c ThreadpoolWorker 
  446 3fec ThreadpoolWorker 
  447 1d58 ThreadpoolWorker 
  448  904 ThreadpoolWorker 
  449 297c ThreadpoolWorker 
  450 21c0 ThreadpoolWorker 
  451 1524 ThreadpoolWorker 
  452 334c ThreadpoolWorker 
  453  d5c ThreadpoolWorker 
  454 3524 ThreadpoolWorker 
  455  dd4 ThreadpoolWorker 
  456 1878 ThreadpoolWorker 
  457 375c ThreadpoolWorker 
  458 1e1c ThreadpoolWorker 
  459 190c ThreadpoolWorker 
  460 35fc ThreadpoolWorker 
  461 2ce4 ThreadpoolWorker 
  462 15a8 ThreadpoolWorker 
  463 2994 ThreadpoolWorker 
  464 1058 ThreadpoolWorker 
  465 1948 ThreadpoolWorker 
  466 36c4 ThreadpoolWorker 
  467 1168 ThreadpoolWorker 
  468 24a4 ThreadpoolWorker 
  469 17f8 ThreadpoolWorker 
  470  720 ThreadpoolWorker 
  471 3594 ThreadpoolWorker 
  472 32a8 ThreadpoolWorker 
  473  818 ThreadpoolWorker 
  474 10e4 ThreadpoolWorker 
  475 2870 ThreadpoolWorker 
  476 2e9c ThreadpoolWorker 
  477 3480 ThreadpoolWorker 
  478 11fc ThreadpoolWorker 
  479 2fac ThreadpoolWorker 
  480  6b0 ThreadpoolWorker 
  481 1694 ThreadpoolWorker 
  482 3214 ThreadpoolWorker 
  483 3ad0 ThreadpoolWorker 
  484  7e4 ThreadpoolWorker 
  485  d10 ThreadpoolWorker 
  486 2610 ThreadpoolWorker 
  487 34d4 ThreadpoolWorker 
  488 3dbc ThreadpoolWorker 
  489 1920 ThreadpoolWorker 
  490 354c ThreadpoolWorker 
  491 1e90 ThreadpoolWorker 
  492  64c ThreadpoolWorker 
  493 39bc ThreadpoolWorker 
  494 3ab8 ThreadpoolWorker 
  495 250c ThreadpoolWorker 
  496 2c58 ThreadpoolWorker 
  497 1cf0 ThreadpoolWorker 
  498 24e4 ThreadpoolWorker 
  499 1c8c ThreadpoolWorker 
  500 3614 ThreadpoolWorker 
  501 29a0 ThreadpoolWorker 
  502 290c ThreadpoolWorker 
  503 2158 ThreadpoolWorker 
  504 35e0 ThreadpoolWorker 
  505  d28 ThreadpoolWorker 
  506  e4c ThreadpoolWorker 
  507 27e0 ThreadpoolWorker 
  508 1df8 ThreadpoolWorker 
  509  e44 ThreadpoolWorker 
  510 41bc ThreadpoolWorker 
  511 13e8 ThreadpoolWorker 
  512 1db0 ThreadpoolWorker 
  513 31b0 ThreadpoolWorker 
  514 1ee8 ThreadpoolWorker 
  515 10c0 ThreadpoolWorker 
  516 1720 ThreadpoolWorker 
  517 3624 ThreadpoolWorker 
  518 3bc8 ThreadpoolWorker 
  519 1888 ThreadpoolWorker 
  520 38b0 ThreadpoolWorker 
  521 1110 ThreadpoolWorker 
  522 1ca8 ThreadpoolWorker 
  523 3ae8 ThreadpoolWorker 
  524 1f50 ThreadpoolWorker 
  525 1118 ThreadpoolWorker 
  526 20d0 ThreadpoolWorker 
  527 17b0 ThreadpoolWorker 
  528 32a4 ThreadpoolWorker 
  529 355c ThreadpoolWorker 
  530 4390 ThreadpoolWorker 
  531 1248 ThreadpoolWorker 
  532 2718 ThreadpoolWorker 
  533 303c ThreadpoolWorker 
  534 3054 ThreadpoolWorker 
  535 1824 ThreadpoolWorker 
  536 3b7c ThreadpoolWorker 
  537 3838 ThreadpoolWorker 
  538 36d8 ThreadpoolWorker 
  539 3c28 ThreadpoolWorker 
  540 31f0 ThreadpoolWorker 
  541 4078 ThreadpoolWorker 
  542 3fbc ThreadpoolWorker 
  543  aec ThreadpoolWorker 
  544 37bc ThreadpoolWorker 
  545 3e70 ThreadpoolWorker 
  546 3924 ThreadpoolWorker 
  547 21a8 ThreadpoolWorker 
  548 2c28 ThreadpoolWorker 
  549 3764 ThreadpoolWorker 
  550 3cd8 ThreadpoolWorker 
  551 3378 ThreadpoolWorker 
  552 2be8 ThreadpoolWorker 
  553 3f14 ThreadpoolWorker 
  554 3f40 ThreadpoolWorker 
  555 3a60 ThreadpoolWorker 
  556 3638 ThreadpoolWorker 
  557 3ca8 ThreadpoolWorker 
  558 1894 ThreadpoolWorker 
  559 1d2c ThreadpoolWorker 
  560 3a10 ThreadpoolWorker 
  561 39ec ThreadpoolWorker 
  562 2c44 IOCompletion 
  563  448 IOCompletion 
  565 4144 IOCompletion 
  566 1f6c IOCompletion 
  567 2e28 IOCompletion 
  568 1510 IOCompletion 
  569 26a8 IOCompletion 
  570 2f8c IOCompletion 
  571 3e74 IOCompletion 
  572  ce4 IOCompletion 
  573 3c38 IOCompletion 
  574 1d14 IOCompletion 
  575 288c ThreadpoolWorker 
  576 41c8 ThreadpoolWorker 
  577  f50 ThreadpoolWorker 
  578 4090 IOCompletion 
  579 1b64 ThreadpoolWorker 
  580 22b4 IOCompletion 
  581 2108 ThreadpoolWorker 
  582 2304 IOCompletion 
  583 26e8 ThreadpoolWorker 
  584 2938 IOCompletion 
  585 3a9c ThreadpoolWorker 
  586  b20 ThreadpoolWorker 
  587 2058 IOCompletion 
  588 2ddc ThreadpoolWorker 
  589 3d10 ThreadpoolWorker 
  590 1db4 ThreadpoolWorker 
  591   c0 ThreadpoolWorker 
  592 26f8 ThreadpoolWorker 
  593 2954 ThreadpoolWorker 
  594 17f0 ThreadpoolWorker 
  595 123c ThreadpoolWorker 
  596 1d94 ThreadpoolWorker 
  597 3be4 ThreadpoolWorker 
  598 19f4 IOCompletion 
  599  e80 ThreadpoolWorker 
  600 2774 ThreadpoolWorker 
  601 4114 ThreadpoolWorker 
  602 3d08 ThreadpoolWorker 
  603  abc ThreadpoolWorker 
  604 3e40 ThreadpoolWorker 
  605 1d18 ThreadpoolWorker 
  606 19c8 ThreadpoolWorker 
  607 136c ThreadpoolWorker 
  608 3e90 ThreadpoolWorker 
  609 3c0c ThreadpoolWorker 
  610 2770 ThreadpoolWorker 
  611 2c34 ThreadpoolWorker 
  612 3774 ThreadpoolWorker 
  613 2278 ThreadpoolWorker 
  614 3ef8 ThreadpoolWorker 
  615 2cb4 ThreadpoolWorker 
  616 35d8 ThreadpoolWorker 
  617 1764 ThreadpoolWorker 
  618 4288 ThreadpoolWorker 
  619 1a3c ThreadpoolWorker 
  620  c58 ThreadpoolWorker 
  621 2998 IOCompletion 
  622 273c ThreadpoolWorker 
  623 3e9c IOCompletion 
  624 18a4 ThreadpoolWorker 
  625 19cc ThreadpoolWorker 
  626 13d8 ThreadpoolWorker 
  627 34bc IOCompletion 
  628 3890 IOCompletion 
  629 24ac ThreadpoolWorker 
  630 4190 ThreadpoolWorker 
  631 2920 ThreadpoolWorker 
  632 3bf4 ThreadpoolWorker 
  633 1b20 ThreadpoolWorker 
  634 381c ThreadpoolWorker 
  635 391c ThreadpoolWorker 
  636 2298 ThreadpoolWorker 
  637 3430 ThreadpoolWorker 
  638 1e5c IOCompletion 
  639 2d50 ThreadpoolWorker 
  640 30a0 ThreadpoolWorker 
  641 3794 ThreadpoolWorker 
  642 3124 ThreadpoolWorker 
  643 2640 ThreadpoolWorker 
  644 4160 ThreadpoolWorker 
  645 1adc ThreadpoolWorker 
  646 12e0 ThreadpoolWorker 
  647  308 ThreadpoolWorker 
  648 24e8 ThreadpoolWorker 
  649 1b48 ThreadpoolWorker 
  650 1f70 ThreadpoolWorker 
  651 322c ThreadpoolWorker 
  652 3e0c ThreadpoolWorker 
  653 34f8 ThreadpoolWorker 
  654 1f90 ThreadpoolWorker 
  655  644 IOCompletion 
  656 31a8 ThreadpoolWorker 
  657 3ac8 ThreadpoolWorker 
  658 1434 IOCompletion 
  659 2adc ThreadpoolWorker 
  660 1f38 ThreadpoolWorker 
  661 42dc ThreadpoolWorker 
  662 2d08 ThreadpoolWorker 
  663 3a94 ThreadpoolWorker 
  664 2b3c ThreadpoolWorker 
  665 3294 ThreadpoolWorker 
  666 3364 ThreadpoolWorker 
  667 1680 ThreadpoolWorker 
  668 408c ThreadpoolWorker 
  669 35ec ThreadpoolWorker 
  670 3b88 ThreadpoolWorker 
  671 41d8 ThreadpoolWorker 
  672 341c IOCompletion 
  673 2bf0 ThreadpoolWorker 
  674 31c4 ThreadpoolWorker 
  675 39f0 ThreadpoolWorker 
  676 1820 ThreadpoolWorker 
  677 1d88 ThreadpoolWorker 
  678 1ce4 ThreadpoolWorker 
  679 1df0 ThreadpoolWorker 
  680 3c34 ThreadpoolWorker 
  681 1838 ThreadpoolWorker 
  682 43e0 ThreadpoolWorker 
  683 2934 ThreadpoolWorker 
  684 2c14 ThreadpoolWorker 
  685 17bc ThreadpoolWorker 
  686 1fc0 IOCompletion 
  687 204c IOCompletion 
  688 3f9c IOCompletion 
  689 2bfc IOCompletion 
  690 38f4 IOCompletion 
  691 40e4 IOCompletion 
  692 1b0c IOCompletion 
  693 1120 IOCompletion 
  694 1df4 IOCompletion 
  695 3b18 IOCompletion 
  696 3590 IOCompletion 
  697 3848 IOCompletion 
  698 2858 IOCompletion 
  699 3a3c IOCompletion 
  700 399c IOCompletion 
  701 1128 IOCompletion 
  702 3434 IOCompletion 
  703 25e4 IOCompletion 
  704 3744 IOCompletion 
  705 1c4c IOCompletion 
  706 2288 IOCompletion 
  707 12cc IOCompletion 
  708 2b7c IOCompletion 
  709 3ae4 IOCompletion 
  710 4084 IOCompletion 
  711 1bdc IOCompletion 
  712 1fec IOCompletion 
  713 3d24 IOCompletion 
  714 2378 IOCompletion 
  715 3b1c IOCompletion 
  716  500 IOCompletion 
  717 1e50 IOCompletion 
  718   b8 IOCompletion 
  719 1c70 IOCompletion 
  720  6b8 IOCompletion 
  721 4080 IOCompletion 
  722 35dc IOCompletion 
  723 1ab0 IOCompletion 
  724  60c IOCompletion 
  725 3c80 IOCompletion 
  726 3bd8 IOCompletion

0:032> [a10]s
ntdll!NtWaitForAlertByThreadId+0x14:
00007ffafc559174 c3 ret 0:031> ~k # Child-SP RetAddr Call Site 00 000000f5a00ff478 00007ffafc51581f ntdll!NtWaitForAlertByThreadId+0x14 01 000000f5a00ff480 00007ffafc515726 ntdll!RtlpWaitOnAddressWithTimeout+0x43 02 000000f5a00ff4b0 00007ffafc5155b7 ntdll!RtlpWaitOnAddress+0xb2 03 000000f5a00ff520 00007ffafc4d4c64 ntdll!RtlpWaitOnCriticalSection+0xdb 04 000000f5a00ff5f0 00007ffafc4d4b90 ntdll!RtlpEnterCriticalSectionContended+0xcc 05 000000f5a00ff620 00007ffaed9fb762 ntdll!RtlEnterCriticalSection+0x40 06 000000f5a00ff650 00007ffaedbcc4a2 clr!CrstBase::SpinEnter+0xac 07 000000f5a00ff680 00007ffaed9f541a clr!CrstBase::Enter+0x13f 08 000000f5a00ff6d0 00007ffaeda3fdc7 clr!ThreadSuspend::LockThreadStore+0x72 09 000000f5a00ff700 00007ffaeda43263 clr!ThreadSuspend::SuspendEE+0x94 0a 000000f5a00ff800 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x1c8 0b 000000f5a00ff840 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a 0c 000000f5a00ff880 00007ffafa4b84d4 clr!<lambda_984b1062d0d4b010966442b24889003f>::<lambda_invoker_cdecl>+0x51 0d 000000f5a00ff8b0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 0e 000000f5a00ff8e0 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:031> ~~[9c0]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:033> ~k
# Child-SP RetAddr Call Site
00 000000f5a01ff758 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a01ff760 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a01ff800 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a01ff840 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a01ff8a0 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71
05 000000f5a01ff930 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63
06 000000f5a01ff970 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a
07 000000f5a01ffbb0 00007ffafa4b84d4 clr!::+0x51
08 000000f5a01ffbe0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a01ffc10 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:033> ~~[2b04]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:034> ~k # Child-SP RetAddr Call Site 00 000000f5a027f4c8 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a027f4d0 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a027f570 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a027f5b0 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a027f610 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71 05 000000f5a027f6a0 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63 06 000000f5a027f6e0 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a 07 000000f5a027fa20 00007ffafa4b84d4 clr!<lambda_984b1062d0d4b010966442b24889003f>::<lambda_invoker_cdecl>+0x51 08 000000f5a027fa50 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a027fa80 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:034> ~~[145c]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:035> ~k
# Child-SP RetAddr Call Site
00 000000f5a02ff568 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a02ff570 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a02ff610 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a02ff650 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a02ff6b0 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71
05 000000f5a02ff740 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63
06 000000f5a02ff780 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a
07 000000f5a02ffbc0 00007ffafa4b84d4 clr!::+0x51
08 000000f5a02ffbf0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a02ffc20 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:035> ~~[3e1c]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:036> ~k # Child-SP RetAddr Call Site 00 000000f5a037f678 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a037f680 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a037f720 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a037f760 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a037f7c0 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71 05 000000f5a037f850 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63 06 000000f5a037f890 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a 07 000000f5a037fdd0 00007ffafa4b84d4 clr!<lambda_984b1062d0d4b010966442b24889003f>::<lambda_invoker_cdecl>+0x51 08 000000f5a037fe00 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a037fe30 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:036> ~~[3920]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:037> ~k
# Child-SP RetAddr Call Site
00 000000f5a03ff1a8 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a03ff1b0 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a03ff250 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a03ff290 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a03ff2f0 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71
05 000000f5a03ff380 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63
06 000000f5a03ff3c0 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a
07 000000f5a03ffa00 00007ffafa4b84d4 clr!::+0x51
08 000000f5a03ffa30 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a03ffa60 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:037> ~~[14dc]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:038> ~k # Child-SP RetAddr Call Site 00 000000f5a047edf8 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a047ee00 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a047eea0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a047eee0 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a047ef40 00007ffaeda431c3 clr!CLREventBase::WaitEx+0x71 05 000000f5a047efd0 00007ffaedb3c288 clr!SVR::gc_heap::gc_thread_function+0x63 06 000000f5a047f010 00007ffaedd24821 clr!SVR::gc_heap::gc_thread_stub+0x9a 07 000000f5a047f750 00007ffafa4b84d4 clr!<lambda_984b1062d0d4b010966442b24889003f>::<lambda_invoker_cdecl>+0x51 08 000000f5a047f780 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a047f7b0 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:038> ~~[2e4c]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:062> ~k
# Child-SP RetAddr Call Site
00 000000f5a197f718 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a197f720 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a197f7c0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a197f800 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a197f860 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71
05 000000f5a197f8f0 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58
06 000000f5a197f940 00007ffaed9fb5b5 clr!::operator()+0x7d
07 000000f5a197f970 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b
08 000000f5a197fa30 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a197fa60 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:062> ~~[f48]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:063> ~k # Child-SP RetAddr Call Site 00 000000f5a19ff608 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a19ff610 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a19ff6b0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a19ff6f0 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a19ff750 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71 05 000000f5a19ff7e0 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58 06 000000f5a19ff830 00007ffaed9fb5b5 clr!<lambda_30a9ff4b0090b4a937571d5c15ac7072>::operator()+0x7d 07 000000f5a19ff860 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b 08 000000f5a19ff9a0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a19ff9d0 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:063> ~~[3078]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:064> ~k
# Child-SP RetAddr Call Site
00 000000f5a1a7f528 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a1a7f530 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a1a7f5d0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a1a7f610 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a1a7f670 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71
05 000000f5a1a7f700 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58
06 000000f5a1a7f750 00007ffaed9fb5b5 clr!::operator()+0x7d
07 000000f5a1a7f780 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b
08 000000f5a1a7f940 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a1a7f970 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:064> ~~[3784]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:065> ~k # Child-SP RetAddr Call Site 00 000000f5a1aff3f8 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a1aff400 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a1aff4a0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a1aff4e0 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a1aff540 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71 05 000000f5a1aff5d0 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58 06 000000f5a1aff620 00007ffaed9fb5b5 clr!<lambda_30a9ff4b0090b4a937571d5c15ac7072>::operator()+0x7d 07 000000f5a1aff650 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b 08 000000f5a1aff890 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a1aff8c0 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:065> ~~[291c]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:066> ~k
# Child-SP RetAddr Call Site
00 000000f5a1b7f888 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a1b7f890 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a1b7f930 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a1b7f970 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a1b7f9d0 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71
05 000000f5a1b7fa60 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58
06 000000f5a1b7fab0 00007ffaed9fb5b5 clr!::operator()+0x7d
07 000000f5a1b7fae0 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b
08 000000f5a1b7fda0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a1b7fdd0 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:066> ~~[403c]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:067> ~k # Child-SP RetAddr Call Site 00 000000f5a1bff848 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a1bff850 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a1bff8f0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a1bff930 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a1bff990 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71 05 000000f5a1bffa20 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58 06 000000f5a1bffa70 00007ffaed9fb5b5 clr!<lambda_30a9ff4b0090b4a937571d5c15ac7072>::operator()+0x7d 07 000000f5a1bffaa0 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b 08 000000f5a1bffde0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a1bffe10 0000000000000000 ntdll!RtlUserThreadStart+0x21 0:067> ~~[183c]s ntdll!NtWaitForSingleObject+0x14: 00007ffafc555a84 c3 ret
0:068> ~k
# Child-SP RetAddr Call Site
00 000000f5a1c7f848 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14
01 000000f5a1c7f850 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f
02 000000f5a1c7f8f0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c
03 000000f5a1c7f930 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f
04 000000f5a1c7f990 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71
05 000000f5a1c7fa20 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58
06 000000f5a1c7fa70 00007ffaed9fb5b5 clr!::operator()+0x7d
07 000000f5a1c7faa0 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b
08 000000f5a1c7fe60 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14
09 000000f5a1c7fe90 0000000000000000 ntdll!RtlUserThreadStart+0x21
0:068> ~~[20bc]s
ntdll!NtWaitForSingleObject+0x14:
00007ffafc555a84 c3 ret 0:069> ~k # Child-SP RetAddr Call Site 00 000000f5a1cff348 00007ffaf89d4daf ntdll!NtWaitForSingleObject+0x14 01 000000f5a1cff350 00007ffaedbc74b8 KERNELBASE!WaitForSingleObjectEx+0x8f 02 000000f5a1cff3f0 00007ffaedbc746e clr!CLREventWaitHelper2+0x3c 03 000000f5a1cff430 00007ffaedbc73d4 clr!CLREventWaitHelper+0x1f 04 000000f5a1cff490 00007ffaeda613c0 clr!CLREventBase::WaitEx+0x71 05 000000f5a1cff520 00007ffaedd242d5 clr!SVR::gc_heap::bgc_thread_function+0x58 06 000000f5a1cff570 00007ffaed9fb5b5 clr!<lambda_30a9ff4b0090b4a937571d5c15ac7072>::operator()+0x7d 07 000000f5a1cff5a0 00007ffafa4b84d4 clr!Thread::intermediateThreadProc+0x8b 08 000000f5a1cff9e0 00007ffafc51e871 kernel32!BaseThreadInitThunk+0x14 09 000000f5a1cffa10 00000000`00000000 ntdll!RtlUserThreadStart+0x21
~

marafiq on 16 Nov 2020

yeah, as I mentioned on twitter, GC hasn't started yet, it's just waiting for SuspendEE to return -

00 000000f5`a00ff478 00007ffa`fc51581f     ntdll!NtWaitForAlertByThreadId+0x14
01 000000f5`a00ff480 00007ffa`fc515726     ntdll!RtlpWaitOnAddressWithTimeout+0x43
02 000000f5`a00ff4b0 00007ffa`fc5155b7     ntdll!RtlpWaitOnAddress+0xb2
03 000000f5`a00ff520 00007ffa`fc4d4c64     ntdll!RtlpWaitOnCriticalSection+0xdb
04 000000f5`a00ff5f0 00007ffa`fc4d4b90     ntdll!RtlpEnterCriticalSectionContended+0xcc
05 000000f5`a00ff620 00007ffa`ed9fb762     ntdll!RtlEnterCriticalSection+0x40
06 000000f5`a00ff650 00007ffa`edbcc4a2     clr!CrstBase::SpinEnter+0xac
07 000000f5`a00ff680 00007ffa`ed9f541a     clr!CrstBase::Enter+0x13f
08 000000f5`a00ff6d0 00007ffa`eda3fdc7     clr!ThreadSuspend::LockThreadStore+0x72
09 000000f5`a00ff700 00007ffa`eda43263     clr!ThreadSuspend::SuspendEE+0x94
0a 000000f5`a00ff800 00007ffa`edb3c288     clr!SVR::gc_heap::gc_thread_function+0x1c8
0b 000000f5`a00ff840 00007ffa`edd24821     clr!SVR::gc_heap::gc_thread_stub+0x9a
0c 000000f5`a00ff880 00007ffa`fa4b84d4     clr!<lambda_984b1062d0d4b010966442b24889003f>::<lambda_invoker_cdecl>+0x51
0d 000000f5`a00ff8b0 00007ffa`fc51e871     kernel32!BaseThreadInitThunk+0x14
0e 000000f5`a00ff8e0 00000000`00000000     ntdll!RtlUserThreadStart+0x21

because that one thread is not getting suspended. interop folks might want to take a look - CC @AaronRobinsonMSFT. Aaron, could you please take a look at the first callstack on this issue? my guess is clr!JIT_InitPInvokeFrame is perhaps trying to suspend the thread because it observes suspension in progress and isn't successfully suspending thread?

Maoni0 on 16 Nov 2020

Thanks again for taking time look at it. In case this is helpful. I did !ThreadState on all GC Threads. Much slim output. Oh it indicates Stack Overflow, Do not know what to make of that.

~
31 a10 GC
32 17d4 GC
33 9c0 GC
34 2b04 GC
35 145c GC
36 3e1c GC
37 3920 GC
38 14dc GC
39 19e8 Finalizer
40 f18 ProfilingAPIAttach
43 13b4 Timer
44 1730 ADUnloadHelper
45 2384 Wait
56 2ef4 ThreadpoolWorker
62 2e4c GC
63 f48 GC
64 3078 GC
65 3784 GC
66 291c GC
67 403c GC
68 183c GC
69 20bc GC
0:045> !ThreadState a10
GC On Transitions
Background
Dead
0:045> !ThreadState 20bc
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Legal to Join
Hijacked by the GC
CoInitialized
0:045> !ThreadState 183c
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Legal to Join
Dead
CLR Owns
0:045> !ThreadState 403c
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Legal to Join
In Single Threaded Apartment
0:045> !ThreadState 291c
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Blocking GC for Stack Overflow
Dead
CoInitialized
0:045> !ThreadState 3784
User Suspend Pending
Hijacked by the GC
Blocking GC for Stack Overflow
Background
Unstarted
CLR Owns
CoInitialized
0:045> !ThreadState 3078
Debug Suspend Pending
GC On Transitions
Legal to Join
Yield Requested
CLR Owns
CoInitialized
0:045> !ThreadState f48
Debug Suspend Pending
Yield Requested
Blocking GC for Stack Overflow
Background
Unstarted
Dead
0:045> !ThreadState 2e4c
User Suspend Pending
Debug Suspend Pending
Yield Requested
Background
Unstarted
Dead
CoInitialized
0:045> !ThreadState 14dc
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Yield Requested
Hijacked by the GC
Unstarted
CLR Owns
0:045> !ThreadState 3920
Legal to Join
Blocking GC for Stack Overflow
Dead
CLR Owns
CoInitialized
0:045> !ThreadState 3e1c
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Background
Unstarted
Dead
CLR Owns
CoInitialized
0:045> !ThreadState 145c
User Suspend Pending
Debug Suspend Pending
GC On Transitions
Yield Requested
Unstarted
CLR Owns
0:045> !ThreadState 2b04
User Suspend Pending
Blocking GC for Stack Overflow
Background
Dead
CoInitialized
0:045> !ThreadState 9c0
Yield Requested
Hijacked by the GC
Blocking GC for Stack Overflow
Dead
0:045> !ThreadState 17d4
User Suspend Pending
GC On Transitions
Yield Requested
Hijacked by the GC
Blocking GC for Stack Overflow
Background
Unstarted
CLR Owns
~

marafiq on 16 Nov 2020

could you please take a look at the first callstack on this issue? my guess is clr!JIT_InitPInvokeFrame is perhaps trying to suspend the thread because it observes suspension in progress and isn't successfully suspending thread?

Alternatively, disable CPU profiling altogether (to verify) (Dynatrace > "Settings" > "Server-Side monitoring" > "Deep monitoring" > "Troubleshooting" > "Capture method hotspot information in PurePaths: Off")

That would be my first avenue for investigation.

AaronRobinsonMSFT on 17 Nov 2020

👍2

@AaronRobinsonMSFT thanks. Is there non-intrusive way to collect trace using perfview in prod for classic GC starvation issue for this specific issue

marafiq on 18 Nov 2020

Is there non-intrusive way to collect trace using perfview in prod for classic GC starvation issue for this specific issue

@marafiq Not that I am aware of.

Let me break this down a bit more. There are two P/Invoke mechanisms that are considered an implementation detail from the CLR's perspective. The first is a QCall and the second is an FCall - details on both of these can be found here. Both are designed for the runtime to call a runtime implemented native function from a managed environment. Consider any case where it would be easier to implement some piece of logic in C/C++ as opposed to a .NET language (e.g. C#). In .NET Core the number of places where we natively implemented logic has been and will continue to be reduced. However, in .NET Framework there are many such functions and although it is now possible few were ever reimplemented in C# for various reasons (e.g. required architectural changes, risk aversion of bug tail, lack of business need, etc).

The key distinction between the QCall and FCall is the GC transition. For a QCall the thread performs a GC mode transition (COOP -> PREMPT) - in this regard a QCall is practically identical to a typical P/Invoke. However, for an FCall no GC mode transition occurs. This means that the GC believes the thread in question is under its control and won't do anything until it places the thread in the correct mode for the GC to proceed - this was @Maoni0's statement about waiting for SuspendEE. This lack of ability to suspend is what is meant by "starvation" - since the thread didn't tell the GC it is leaving its control the GC is desperately trying to do its job but is starved for control. It would appear that the sampler here is exacerbating the sensitive transition from managed => native => managed and decreasing the time when the thread is controllable by the GC.

I supposed it would be possible for the sampler to create a good/bad list of methods that indicate when sampling is to be avoided or at least temporarily suspended. This could be done by inspecting the metadata of all methods in mscorlib.dll. Methods that are marked with MethodImplOptions.InternalCall are FCalls. If the sampler observes a function with this metadata on the stack, the sampler should stop sampling for a period of time to avoid perturbing this sensitive operation. Note that even a single stack walk at this time could impact this so this isn't a perfect solution. Ideally the best solution here is a kernel mode sampler that doesn't run in user mode (e.g. ETW).

AaronRobinsonMSFT on 18 Nov 2020

❤1

Thanks makes lot of sense now.

marafiq on 19 Nov 2020

@marafiq I am going to close this issue since it is "by-design" from the CLR perspective. Please let us know if we can help in anyway to workaround this issue.

AaronRobinsonMSFT on 19 Nov 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings