What are the thresholds (estimates, not a hard and fast rule)?
32-bit Windows XP, Server 2003, Vista, Server 2008, 7:
For most processes, if higher than 2,500 handles open, investigate.
64-bit Windows XP, Server 2003, Vista, Server 2008, 7, Server 2008 R2:
For most processes, if higher than 3,000 handles open, investigate.
Now the question becomes, what type of handles are open by the application?
Click on View > Select Columns…
Click on the ‘Process Performance’ tab > ‘Handle Count’
What you will notice is the new column called ‘Handles’
Move the ‘Handles’ columns next to ‘Working Set’
Sort it by the ‘Handles’ columns
Now, we have the process that is consuming the most amount of handles just like in Task Manager.
But we still don’t see what type of handles they are.
Click on View > Lower Pane View > Handles
Note: This will enable the “Show Lower Pane” by default.
Click View > Show Unnamed Handles and Mappings
What you will see is the second pane at the bottom that has ‘Type’ and ‘Name’
In our sample application, they are ‘Event’ handles to ‘Sessions11BaseNamedObjectsTestLimit’
So, each handle that is open, will consume paged or nonpaged pool (kernel memory) which on 32-bit systems, there is a finite amount available.
Thus, handle leaks, can and will cause Windows clients and servers to hang.
In the example above, we could see that it consumes 112 Non-Paged pool memory(kernel memory).
Note: To see what type of handles are the most used for the application, you will want to click on File > Save As > .txt and load it in Excel to sort by handle type.
Ok, so now that we have the handle type and name, what’s our next steps?
If you are lucky, if the handle leak is related to a Microsoft process, you could do a search in our (Microsoft) Knowledge Base http://support.microsoft.com and find the root cause.
Keyword: Handle count
If it is a 3rd party process, please check their Knowledge Base articles.
Hint: A lot of times, the 3rd party vendors and Independent Software Vendors (isv’s) will not use the word ‘handle leak’, instead will use ‘memory leak’ in their KB’s. Thus, if you don’t get any results using ‘handle leak’, change the keyword in your searches to ‘memory leak’.
After you search the Microsoft and 3rd party KB’s and end up empty handed. To tackle a ‘new’ handle leak, we will need to:
Enable handle tracking on the application using Microsoft Application Verifier or use Gflags which is a part of of the Debugging Tools for Windows.
Once you have enabled any of the handle tracking tools, we will need to use one of the debuggers such as WinDbg Download and Install Debugging Tools for Windows
Option 1a) Enable handle tracing on the process by using Global Flags (GFlags):
Go to c:Program FilesDebugging Tools for Windows (x64)
Double click on gflags.exe
Click on the ‘Kernel Flags’ tab
Check the box next to ‘Enable bad handles detection’
Click on the ‘Image File’ tab
Next to ‘Image: (TAB to refresh)’, enter the path and the executable name.
c:program filesMicrosoft Sysinternalstestlimit64.exe
And then click on the ‘Tab’ key
Next to ‘Stack Backtrace: (Megs):’ enter ‘10’
Click on Apply
Click on Ok
Option 1b) Enable handle tracing on the process by using Global Flags (GFlags):
Install Application Verifier (x64), you can download it from https://www.microsoft.com/en-us/download/details.aspx?id=20028
Click on All Programs > Application Verifier (x64) > Application Verifier (x64)
You will see the “Application Verifier 4.0 (x64) tool above.
Click on File > Add Application
Point it to c:program filesmicrosoft sysinternals folder
Next to ‘File name:’ testlimit64.exe
Click on Open
You will see this UI.
Expand ‘Basics’ and you will notice that there are a good amount of options (Exceptions, Handles, Heaps, Input/Output, Leak, Locks, Memory, Threadpool, TLS) checked by default.
Uncheck all, except for ‘Handles’ since it is what we are troubleshooting.
Click on Save
You will get this informational, click on OK
Click on Exit
Right click on WinDbg > Run as administrator
Click on File > Symbol File Path…
Under “Symbol Search Path”, type srv*c:websymbols*http://msdl.microsoft.com/download/symbols
Click on “File, Attach to a Process…”
Scroll down to the application that you want to monitor for high handle consumption.
In our example here, we are clicking on “testlimit64.exe”
Click on Ok
Once the debugger (in this case, WinDbg) opens up.
Type “.logopen c:temphandleleak_testlimit64_MMDDYEAR_HHMMSS.txt” without the quotation marks and then press Enter.
Note: Where MMDDYEAR is the Month, Date and Year.
Where HHMMSS is the Hour, Minute and Seconds.
and ApplicationName is the actual process name.
Type “!htrace enable” without the quotation marks and then press Enter.
Note: Notice that you will get information such as “Handle” and that it’s opening (OPEN).
And you will also see the stack being called.
If you don’t have symbol resolution to the application that you are troubleshooting, then you will get an offset that is big instead of the function names.
In this example:
Type “!htrace –snapshot” w/o the quotation marks, and then Press Enter.
You will get the following return informational “Handle tracing information snapshot successfully taken”
Type “g” without the quotation marks and then Press Enter.
Note: g for Go.
Reproduce the issue.
Once the reproduction is done.
Click on Debug > Break (or just CTRL+Break)
Type “!htrace –diff” without the quotation marks and then press Enter.
Note: By default, Windows Vista, Windows Server 2008, Windows 7 and Windows Server 2008 R2 keep a history of 4000 handles open and close operations.
With !htrace you can enable to keep a much higher history by doing the following:
Type “!htrace –enable 0x20000” without the quotation marks and then press Enter.
In this example, we are increasing the handle history to 128000 (decimal, 0x20000 hexadecimal).
Then go thru
Note: Reproduce the leak, or let the app leak