Hello,

There is a powerful though less known feature called "Analyze wait chain" added to Windows Vista. This feature was accessible through Resource Monitor until Windows 8.

And it is accessible via Task Manager since Windows 8.

As a Premier Field Engineer, I’m in frequent contact with developers and administrators from different industries and countries. Whenever, I mention Wait Chain Analyzer I’m surprized to see that many people are not aware of it. I get surprized because it is a powerful feature to see whether a process is Waiting on something that is noticable w/o attaching the debugger. The feature has its limitations but still very useful.

Recently, I was approached by a customer to solve the hang/freeze problems of their LOB application. The catch was that there are around 20K end users scattered around 1000 offices. Each end user has usually a non-zero length queue of customers waiting to be served. Hence, it is difficult for the IT support professionals to check the Wait Chain of the relevant process(es) get dump files, etc. To worsen the situation there are many reasons for the hangs. I’ve realized that I didn' know any tool that would generate Wait Chain list for process(es). We have only the UI that can be launched either from Resource Monitor or Task Manager. And, that UI shows one process’s Wait Chain at once. To speed up data collection I’ve developed a simple console based tool called “WaitChain”. It can either collect:

• Every process’s Wait Chain when no particular process is specified
• A particular process’s Wait Chain list if the Process ID is specified
• The Wait Chain list of each process instance of an application if the process name is specified

The WaitChain.exe tool is based on the WCT API example in MSDN. It can also generate the dump file of the particular process(es) and each process that shows up in the Wait Chain list. This feature requires procdump to produce the dump files. If WaitChain is asked to generate the Wait Chain for a particular process it reports all the running processes and their IDs. This comes handy when dump files are generated and a callstack waits on a process that was not captured by WCT API and we want to know the name of the process that thread is waiting on.

How to use WaitChain.exe?

Getting help

“WaitChain.exe /?” documents the functionality of WaitChain.exe.

Getting the Wait Chain for all processes running on the system

Simply execute WaitChain.exe without any parameters.

While executing WaitChain.exe puts a dot for every process that it is going to analyze. When finished the result can be read in the report file. WAITCHAIN_ComputerName.LOG is the report file’s name format and the file is created in the same folder as WaitChain.exe is running.

The report has 3 sections:

• Start Information
  The command line information and Process ID (in both hex and dec format) of WaitChain.exe is recorded. Also, the start time of the WaitChain.exe is logged. e.g.

PROGRESS INFORMATION
========================================
Command line : "WaitChain.exe"
PID          : (0x2F90:0n12176)
Started on   : 02/15/2018  14:55:14

• Wait Chain List of each process
  WaitChain tries to connect to each process and get the Wait List of every thread. WaitChain.exe asks for Elevated Usage. This is to open processes running under different security context than WaitChain.exe. Some processes do not allow to be opened.
  They will be listed but no information will be available for them. e.g.

Process (0x4:0n4) - N/A
----------------------------------

Process (0x48:0n72) - N/A
----------------------------------

Process (0x24c:0n588) - smss.exe
----------------------------------

Process (0x320:0n800) - csrss.exe
----------------------------------

Process (0x388:0n904) - wininit.exe
----------------------------------

Process (0x3d0:0n976) - services.exe
----------------------------------

Process (0x72d4:0n29396) - SyncWaitTest.exe
----------------------------------
30424: [72d4:76d8:Blocked]->[End]
17008: [72d4:4270:Blocked]->[End]
25476: [72d4:6384:Blocked]->[CriticalSection:Owned]->[72d4:7688:Blocked]->[CriticalSection:Owned]->[72d4:6384:Blocked]->[End] !!!Deadlock!!!
30344: [72d4:7688:Blocked]->[CriticalSection:Owned]->[72d4:6384:Blocked]->[CriticalSection:Owned]->[72d4:7688:Blocked]->[End] !!!Deadlock!!!

• End Information
  Contains the completion time of the WaitChain.exe process. E.g.

PROGRESS INFORMATION
========================================
Finished on  : 02/15/2018  14:55:50

It should take less than a minute to walk each and every process.

Getting the Wait Chain for a specific process

There are two ways to do this. Either:
i. Specify the Process ID
ii. Or, provide the process name (including the .exe extension)

Specifying a Process ID

The ProcessID should be passed as an argument to WaitChain.exe. e.g. WaitChain should generate a report for the process with ID 29396.

Note: The ProcessID argument should be in decimal format.

The report is slightly different than when WaitChain.exe is executed without arguments:

1. The start and end progress information is just the same.
2. Only the Wait List of the process whose ProcessID is given is reported.
3. The ProcessID (in both hex and dec format) and name of all running processes is listed. This list is useful for two cases:

• When the WaitList contains another process than the inspected one. The ProcessID can be looked up from that list and the process name can be retrieved. E.g.

Process (0x7a24:0n31268) - SyncWaitTest.exe
----------------------------------
9728: [7a24:2600:Blocked]->[End]
21180: [7a24:52bc:Running]->[End]
19380: [7a24:4bb4:Running]->[End]
17168: [7a24:4310:Running]->[End]
22840: [7a24:5938:Blocked]->[ProcWait:Owned]->[4e4c:0:PidOnly]->[End]

Relevant Processes: 0x4e4c->[END]

Thread 22840 of the SyncWaitTest.exe process is waiting on the process with ProcessID 0x4e4c. Now it is time to lookup the process with ProcessID 4e4c.

RUNNING PROCESSES
=================

PID(hex:dec)    NAME
…
0x4e4c:0n20044 notepad.exe ===> That is the process that the SyncWaitTest.exe process is waiting on.

• When a dump file is generated and and a callstack is showing that the thread is waiting on another process and the WaitChain.exe does not report a target ProcessID:ThreadID. WCT APIs do not report each and every wait. Therefore, generating a dump file of the inspected process might be useful. And, having a ProcessID:ProcessName list can help to find the target process on which the inspected process is waiting for.

Getting the Wait Chain for a specific Process Name

In this case the process name should be provided as an argument. WaitChain.exe will analyze each instance of that application’s process. E.g. it is common to see multiple instances of w3wp.exe, svchost.exe.

This time the report will list each instance of SyncWaitTest.exe’s Wait Chain. E.g.

Process (0x7a24:0n31268) - SyncWaitTest.exe
----------------------------------
9728: [7a24:2600:Blocked]->[End]
22840: [7a24:5938:Blocked]->[ProcWait:Owned]->[4e4c:0:PidOnly]->[End]

Relevant Processes: 0x4e4c->[END]

Process (0x6e44:0n28228) - SyncWaitTest.exe
----------------------------------
27980: [6e44:6d4c:Blocked]->[End]
28912: [6e44:70f0:Blocked]->[End]
31704: [6e44:7bd8:Blocked]->[CriticalSection:Owned]->[6e44:3388:Blocked]->[CriticalSection:Owned]->[6e44:7bd8:Blocked]->[End] !!!Deadlock!!!
13192: [6e44:3388:Blocked]->[CriticalSection:Owned]->[6e44:7bd8:Blocked]->[CriticalSection:Owned]->[6e44:3388:Blocked]->[End] !!!Deadlock!!!

The report file (WAITCHAIN log) will also contain the start end information of WaitChain.exe and the list of running processes.

Generating memory dump files

WCT API is nice to detect the reason of some wait types. However, it has its limitations and usually it is required to understand why a process has ended up in that situation. In other words, a call stack information might be necessary. To walk this extra step a memory dump file is needed. WaitChain spawns an instance of ProcDump.exe to generate the memory dump file. “/d” argument will instruct WaitChain.exe to generate dump files. “/d” can be used together with ProcessID or ProcessName arguments.

Generating dumps with ProcessID

In this case WaitChain.exe will generate the dump file of the process with ProcessID. And, it will generate the dump file of the processes which show up in any Wait Chain of the process’s threads. E.g. in the following output the process with ProcID 4e4c is in the Wait Chain of the process 31268 (SyncWaitTest.exe).

Process (0x7a24:0n31268) - SyncWaitTest.exe
----------------------------------
9728: [7a24:2600:Blocked]->[End]
22840: [7a24:5938:Blocked]->[ProcWait:Owned]->[4e4c:0:PidOnly]->[End]

Hence, WaitChain.exe will generate a dump file for both processes with ProcessID 31268 and 4e4c.

Generating dumps with Process Name

In this case WaitChain.exe is supplied with a Process Name. It will note each process that shows up in the Wait Chain of any instance of the ProcessName. At the end it will generate the dump files of each instance of ProcessName and every process that was found to be involved in the Wait Chain lists.

How to interpret the Wait Chain Output in the WAITCHAIN log file?

WaitChain.exe gets each thread’s Wait Chain. It will list the output no matter there is a WaitChain or not.

9728: [7a24:2600:Blocked]->[End]

This line is read as “The thread with ThreadID 9728’s Wait Chain starts with the process with ProcessID 7a24 (31268 in decimal format) and ThreadID 2600 (9728 in decimal format). And, that node is the end node at the same time. Hence, there is no nodes in the Wait Chain. The thread is waiting.”

22840: [7a24:5938:Blocked]->[ProcWait:Owned]->[4e4c:0:PidOnly]->[End]

This line is read as “The thread with ThreadID 22840’s Wait Chain starts with the process with ProcessID 7a24 (31268 in decimal format) and ThreadID 5938 (22840 in decimal format). This node is waiting on a Process. That process has a ProcessID 4e4c. And, the process with ProcessID 4e4c has no further nodes in the Wait Chain”. Once the process 4e4c terminates the thread will continue execution.

As explained above the WCT API has its limitations in detecting the wait types. The details are documented in Wait Chain Traversal document. I’ve created cases to understand the possible output that I might expect from WCT API. Below, I’ll give an example for each case and add the screen shot of the UI of the “Analyze wait chain” feature, too.

ALPC

ALPC stands for Asynchronous Local Inter-Process Communication. It is a message based communication implemented in the Windows Kernel. It can be used between user mode processes, between user mode and kernel drivers and between kernel drivers. An example is the LRPC protocol in RPC. It is based on ALPC. To see the outcome I’ve created a RPC server and client application. The client application is making a call to a method on the RPC interface at the server and that call takes long time to finish its execution. When WaitChain.exe is executed the following output is in the WAITHCAIN log file for the process that is waiting the RCP call’s result:

6840: [2600:1ab8:Blocked]->[Alpc:RPC ControlLRPC-b06e4faaf1376606cd]->[6d00:0:PidOnly]->[End]

The line can be read as “Thread 1ab8 (6840 in decimal) of process 2600 (9728 in decimal) is waiting on ALPC request that is an LRPC end point hosted by process 6d00 (27904 in decimal)”. We can expect that once the RPC server process (PID:27904) has completed execution for the LRPC request the waiting process’s (PID:9728) thread (TID:6840) can continue execution.

The "Analyze wait chain" for the client process reports as follows:

Just FYI, the RPC EndPoint port listening on the LRPC protocol can be find via winobj.exe as well. In fact, WinObj can list each ALPC Port.

If WaitChain.exe is executed with the “/d” argument then the reason for wait could be understood better with checking the RPC server callstacks. E.g. below is the callstack of the RPC Server process (PID:27904) that is not responding:

0:002> kbn
# ChildEBP RetAddr  Args to Child
00 04e7f4d0 7735290b 00000000 04e7f514 e1eb5440 ntdll!NtDelayExecution+0xc
01 04e7f538 7735285f 0001d4c0 00000000 04e7f554 KERNELBASE!SleepEx+0x9b
02 04e7f548 0005225a 0001d4c0 04e7f56c 759f2620 KERNELBASE!Sleep+0xf
03 04e7f554 759f2620 0001d4c0 00000246 00000001 CssRpcSrv!s_ExecuteLong+0x1a [c:codecsscssrpcsrvcssrpcsrvimpl.cpp @ 36]
04 04e7f56c 759b5e98 00052240 04e7f7a0 00000001 rpcrt4!Invoke+0x34
05 04e7f9bc 759b4969 00000000 00000000 0300a178 rpcrt4!NdrStubCall2+0x448
06 04e7f9d8 759dcd40 0300a178 9df72f94 0300a178 rpcrt4!NdrServerCall2+0x19
07 04e7fa1c 759dc78f 00058294 0300a178 04e7faf8 rpcrt4!DispatchToStubInCNoAvrf+0x20
08 04e7fa88 75a10733 0300a178 00000000 00000000 rpcrt4!RPC_INTERFACE::DispatchToStubWorker+0x18f
09 04e7fab4 75a03819 0300a178 00000000 00000000 rpcrt4!RPC_INTERFACE::DispatchToStub+0xeb
0a 04e7fb1c 759dbd13 04e7fb54 0300a0c0 0300fec0 rpcrt4!LRPC_SCALL::DispatchRequest+0x280a9
0b 04e7fba4 759d7335 030b7f50 030b6e48 00000000 rpcrt4!LRPC_SCALL::HandleRequest+0x383
0c 04e7fbf0 759d7c62 030b7f50 04e7fc68 030b6e48 rpcrt4!LRPC_ADDRESS::HandleRequest+0x2e5
0d 04e7fce4 759dfa98 00000000 03018a40 759df9f0 rpcrt4!LRPC_ADDRESS::ProcessIO+0x252
0e 04e7fd28 777e31c9 04e7fe60 0308ef04 03018a40 rpcrt4!LrpcIoComplete+0xa8
0f 04e7fd58 777e409f 04e7fe60 03018a40 00000000 ntdll!TppAlpcpExecuteCallback+0xa9
10 04e7ff4c 74288654 03043e78 74288630 73bd2ce2 ntdll!TppWorkerThread+0x81f
11 04e7ff60 77814a77 03043e78 2e81e9fe 00000000 kernel32!BaseThreadInitThunk+0x24
12 04e7ffa8 77814a47 ffffffff 77839ef5 00000000 ntdll!__RtlUserThreadStart+0x2f
13 04e7ffb8 00000000 777e3880 03043e78 00000000 ntdll!_RtlUserThreadStart+0x1b

0:002> ? 1d4c0
Evaluate expression: 120000 = 0001d4c0

The thread that is serving the request has called the Sleep function with the duration parameter set to 120000 msec.

COM

The next set of wait types that WCT API can detect is COM. It can detect waits on both activation and method invocations.

Waiting on COM activation

I’ve created a simple COM object (WctActTest) and a COM+ Application (WctTestApp) that is hosting the WctActTest object. I’ve enabled object pooling and limited the maximum number of objects in the pool to be 5. The test is a short VBScript that creates 6 objects. After 5 objects are created and not released the COM+ infrastructure won’t allow the 6th activation request. Hence, the script host will wait.

The “Analyze wait chain” UI shows the following Wait Chain for the wscript.exe:

And, the corresponding entry in the WAITCHAIN log that is generated by WaitChain.exe is as follows:

Process (0x3778:0n14200) - wscript.exe
----------------------------------
28184: [3778:6e18:Blocked]->[Com:Owned:{b09cebcd-ba04-4c01-8927-b4122380c61d}]->[528:0:PidOnlyRpcss]->[ComActivation:Owned]->[5ee8:0:PidOnly]->[End]
9024: [3778:2340:Running]->[End]
31160: [3778:79b8:Running]->[End]
26232: [3778:6678:Running]->[End]
1712: [3778:6b0:Blocked]->[End]
11600: [3778:2d50:Running]->[End]
25832: [3778:64e8:Running]->[End]
30376: [3778:76a8:Blocked]->[End]
21164: [3778:52ac:Blocked]->[End]
11268: [3778:2c04:Blocked]->[End]

Relevant Processes: 0x528->0x5ee8->0x3778->[END]

It can be read as “The thread 6e18 (28184 in decimal) of process 3778 (14200 in decimal) is waiting on the COM object with ClassID {b09cebcd-ba04-4c01-8927-b4122380c61d}. The COM object is waiting to be activated. The activation request is delivered to the  process 528 (1320 in decimal, this is the RPCSS service). And, RPCSS is waiting on the process 5ee8 (24296 in decimal, dllhost.exe) that will do the activation”.

“When a process calls CoGetClassObject or CoCreateInstance (CreateObject or new in VB) to activate a component, the COM runtime contacts its local SCM COM activator (RPCSS service) in order to launch the corresponding COM server  that will host the desired component. If the component is remote, the local RPCSS service will forward the request to the RPCSS service of the remote machine.” Therefore, RPCSS is involved.

There is another entry called “Relevant Processes”. This is a list of the processes involved in the Wait Chains of the threads of the wscript.exe process.

Waiting on a COM method call

In this case, there is a DCOM server process called WctComTest.exe and it is hosting a COM object with ProgID:wctCOM and ClassID: {5D74C767-00F4-400B-BE9F-11EA6F532AC2}. The COM object has a method that takes long time to finish execution. The client application that activates the wctCOM object and makes a method call on it is again a simple VBScript. Once more, the wscript.exe process is in focus as it is the waiting party.

The "Analyze wait chain" UI shows the wait chain as follows :

And, the corresponding entry in WaitChain.exe’s WAITCHAIN log file is:

Process (0x63d4:0n25556) - wscript.exe
----------------------------------
26872: [63d4:68f8:Blocked]->[Com:Owned]->[2840:2fdc:Blocked]->[End]
2752: [63d4:ac0:Blocked]->[End]
14620: [63d4:391c:Running]->[End]
29076: [63d4:7194:Running]->[End]
25908: [63d4:6534:Blocked]->[End]
10436: [63d4:28c4:Blocked]->[End]

Relevant Processes: 0x2840->0x63d4->[END]

This time the entry can be read as “The thread 68f8 (26872) of process 63d4 (25556 in decimal) is waiting on a COM method call that is executing on process 2840’s (10304 in decimal) thread 2fdc (12252 in decimal)”.

Critical Sections

Critical sections can be used within a process for synchronization. It cannot be used to synchronize multiple processes. Hence, any output related to Critical Section in the WAITCHAIN log should be evaulated per process base. I have two test cases for Critical Section:

• One is creating a deadlock
• The other is the case where a thread has exited without leaving the entered critical section and another thread is waiting to enter that critical section

Deadlock

In this classic case there are two threads and two critical sections. Each thread enters a critical section and then tries to enter the other critical section. Hence, an infinite wait.

The “Analyze wait chain” UI has a precise explanation for the situation. The WAITCHAIN log has similar entries:

Process (0x4540:0n17728) - SyncWaitTest.exe
----------------------------------
22744: [4540:58d8:Blocked]->[End]
23712: [4540:5ca0:Blocked]->[End]
7896: [4540:1ed8:Blocked]->[CriticalSection:Owned]->[4540:39e0:Blocked]->[CriticalSection:Owned]->[4540:1ed8:Blocked]->[End] !!!Deadlock!!!
14816: [4540:39e0:Blocked]->[CriticalSection:Owned]->[4540:1ed8:Blocked]->[CriticalSection:Owned]->[4540:39e0:Blocked]->[End] !!!Deadlock!!!

The first line is read as “Thread 1ed8 (7896 in decimal) waits on a critical section that is entered by the thread 39e0 (14816 in decimal) in the same process 4540 (17728 in decimal). And, thread 39e0 tries to enter to a critical section that is already entered by thread 1ed8. Hence, a deadlock.”. The other line in the report is similar. This time the Wait Chain for the thread 39e0 is listed therefore the order of the wait chain is reversed.

Waiting on an abandoned critical section

Sometimes -usually due to – a thread that has entered to a critical section exits without leaving the critical section. If that is the case then any thread that is waiting on that critical section will wait forever. Because, there is no way for the already exited thread to leave that critical section anymore. This is a wait on an abandoned critical section. The WCT API reports this situation as follows:

The WAITCHAIN log has a very similar entry:
Process (0x76b4:0n30388) - SyncWaitTest.exe
----------------------------------
25960: [76b4:6568:Blocked]->[End]
23680: [76b4:5c80:Blocked]->[End]
29536: [76b4:7360:Blocked]->[CriticalSection:Abandoned]->[End]

Since the WaitChain tool reports the type of the wait objects and their status the explanation is a bit more descriptive. Here the thread 7360 is waiting on an abandoned critical section. This requires further troubleshooting. Application Verifier can be used to break into debugger when a thread exists without leaving the critical sections that it has entered before.

Mutual Exclusions (mutexes)

Mutexes are synchronization objects that can be used to synchronise more than one process. I’ve created a situation where one process acquires a mutex but does not release it. A second instance of that application tries to acquire the same mutex. However, the second instance will wait for the first process instance to release that mutex object.

The WAITCHAIN report will have the wait object type therefore a bit more information:

Process (0x6524:0n25892) - SyncWaitTest.exe
----------------------------------
13244: [6524:33bc:Blocked]->[End]
24676: [6524:6064:Blocked]->[End]

Process (0x7b4c:0n31564) - SyncWaitTest.exe
----------------------------------
13468: [7b4c:349c:Blocked]->[End]
19848: [7b4c:4d88:Blocked]->[Mutex:Owned:Sessions43BaseNamedObjectsSyncWaitMutex]->[6524:6064:Blocked]->[End]

Relevant Processes: 0x6524->[END]

The second instance of the SyncWaitTest.exe is waiting on a mutex named “SyncWaitMutex” that is acquired by the other instance of the SyncWaitTest.exe process.

SendMessage

SendMessage is a blocking function. It can be cumbersome to figure out which window’s Windows Procedure is targeted. SPY++ and/or dump files are necessary. WCT API is very convenient in this case to find out the process that hosts the target window for the SendMessage call.
To test the WCT API I’ve two applications:

• One client application “SyncWaitTest.exe” that sends a message with MsgID : WM_USER+100 to the window whose class type is SampleWindowClass and window name is “LearntoProgramWindows”.
• The SampleGUI.exe application which hosts this window will show a message box when that particular window message is received. Then it waits for the user to hit the OK button in the messagebox.

Until the OK button is clicked the SyncWaitTest.exe process’s thread which called the SendMessage API will wait.

This how the “Analyze wait chain” UI reports it:

And, this is the content of the WAITCHAIN log:

Process (0x3984:0n14724) - SyncWaitTest.exe
----------------------------------
21768: [3984:5508:Blocked]->[SendMessage:Owned]->[36e4:66c:Blocked]->[End]

Once more the wait type inclusion gives further hints for the nature of the wait.

Wait operations on processes and threads

WCT APIs can detect whether a thread is waiting on another thread to complete or a process to terminate. Let’s create a process (Notepad.exe) and wait for the termination of it. The “Analyze wait chain” UI shows it as follows:

And, it is listed in the WAITCHAIN log as follows:

Process (0x597c:0n22908) - SyncWaitTest.exe
----------------------------------
18568: [597c:4888:Blocked]->[End]
15796: [597c:3db4:Blocked]->[ProcWait:Owned]->[5ba8:0:PidOnly]->[End]

Relevant Processes: 0x5ba8->[END]

Waiting on a thread has similar outcome. For the sake of completeness, I’m including the case of waiting on a thread. “Analyze wait chain” UI describes the situation as follows:

And, the WAITCHAIN log has the following information:

Process (0x2e9c:0n11932) - SyncWaitTest.exe
----------------------------------
21336: [2e9c:5358:Blocked]->[End]
29804: [2e9c:746c:Blocked]->[ThreadWait:Owned]->[2e9c:7894:Blocked]->[End]
30868: [2e9c:7894:Blocked]->[End]

Caveat

I could not find a way through the WCT API to detect the Wait Chain of a thread that is waiting on multiple objects. E.g. assume a thread creates two notepad.exe instances and then waits on both of them to terminate. At that time the waiting thread’s callstack would look like this:

0:004> r;knL
rax=000000000000005b rbx=0000000000000002 rcx=0000000000000002
rdx=0000004a040ffa80 rsi=0000000000000000 rdi=0000000000000002
rip=00007ff8263f09c4 rsp=0000004a040ff6b8 rbp=0000000000000000
r8=000001d784e9b230  r9=0000000000000000 r10=0000000000000000
r11=0000000000000246 r12=0000000000000008 r13=0000004a040ffa80
r14=0000000000000000 r15=0000000000000001
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!NtWaitForMultipleObjects+0x14:
00007ff8`263f09c4 c3              ret
# Child-SP          RetAddr           Call Site
00 0000004a`040ff6b8 00007ff8`22faa966 ntdll!NtWaitForMultipleObjects+0x14
01 0000004a`040ff6c0 00007ff8`22faa84e KERNELBASE!WaitForMultipleObjectsEx+0x106
02 0000004a`040ff9c0 00007ff6`1fbb1e73 KERNELBASE!WaitForMultipleObjects+0xe
03 0000004a`040ffa00 00007ff8`26031fe4 SyncWaitTest!ChildProcesses+0x133
04 0000004a`040ffb10 00007ff8`263befb1 kernel32!BaseThreadInitThunk+0x14
05 0000004a`040ffb40 00000000`00000000 ntdll!RtlUserThreadStart+0x21

0:004> dq @rdx L2
0000004a`040ffa80  00000000`00000108 00000000`00000094

0:004> !handle 108 f;!handle 94 f
Handle 0000000000000108
Type          Process
Attributes    0
GrantedAccess 0x1fffff:
Delete,ReadControl,WriteDac,WriteOwner,Synch
Terminate,CreateThread,,VMOp,VMRead,VMWrite,DupHandle,CreateProcess,SetQuota,SetInfo,QueryInfo,SetPort
HandleCount   8
PointerCount  294752
Name          
Object specific information
Process Id  20376
Parent Process  20392
Base Priority 8
Handle 0000000000000094
Type          Process
Attributes    0
GrantedAccess 0x1fffff:
Delete,ReadControl,WriteDac,WriteOwner,Synch
Terminate,CreateThread,,VMOp,VMRead,VMWrite,DupHandle,CreateProcess,SetQuota,SetInfo,QueryInfo,SetPort
HandleCount   8
PointerCount  294634
Name          
Object specific information
Process Id  24984
Parent Process  20392
Base Priority 8

However, nor “Analyze wait chain” nor WaitChain.exe does not reveal any clue.

Process (0x4fa8:0n20392) - SyncWaitTest.exe
----------------------------------
18504: [4fa8:4848:Blocked]->[End]
31400: [4fa8:7aa8:Running]->[End]
28240: [4fa8:6e50:Running]->[End]
8200: [4fa8:2008:Running]->[End]
18716: [4fa8:491c:Blocked]->[End]

Relevant Processes: 0x4fa8->[END]

Therefore, no reported Wait Chain does not always mean that a process is not waiting on any object.

Conclusion

In case you want a tool that produces a report of all or specific processes' WaitChain lists, generate dump files of the processes showing up in those Wait Chains then feel free to download it from here WaitChain

If you are interested in the source code then you are welcome to access it from here.

Enjoy!