[PowerShell Script] Finding Out the Managed Objects that "Leaked"

Here in PFE most of my teammates use the same approach to identify managed objects that “leak”. The idea is to run !DumpHeap –stat once, wait several minutes to allow the “leak” to make itself evident, then run the command a second time.

After that you have to select those objects that appear in both outputs, and then find the instances which have the biggest delta between the outputs. That is, find the amount of increase/decrease of each type from the first time you executed the command compared to the second time.

Notice that you don’t take those objects using more memory, the fact they’re using more memory doesn’t mean they are “leaking”, you’re interested on those specific objects that have their instances increasing at higher rate than other objects.

As you can see you need to create a tool to do that if you don’t want to use Excel and lots of error prone manual work. Thinking about it, I conferred with my teammate Brad Linscott to create a simple PowerShell script that does the work for you.

The beauty of this approach is that it’s simple, you can change the code whenever you want and there’s integration with LogParser, so you can see a beautiful chart. (think about it when sending a report to your users or upper level managers J)

The script below is a standalone script, meaning it doesn’t require the PowerDbg library or any interaction with WinDbg.

To use it you have to get the output from !DumpHeap –stat whether you’re doing live debugging or post-mort debugging. It doesn’t matter; you just have to copy the output of this command and save it into a text file. You need two text files, so get them using at least 15 minutes of interval time to give some time for the “leak” to manifest itself.

 

Upon running the script you’ll see the objects leaked and a number that is the Delta. As higher the Delta, higher is the probability the object leaked. You may also wish to run a Perfmon log during the data gathering, paying special attention to the GC counters in the .NET CLR Memory object to ensure that GC is occurring. Because if memory growth occurs with no GC, this is not an indication of a leak: either the GC hasn’t felt enough memory pressure to perform a GC or there is some other reason GC hasn’t run. Either way, you need to ensure that memory growth occurs in spite of GC occurring.

The chart will show you the same information in a graphical way, but just the top 20 objects leaked.

Note: The script does not check for leaks in the loader heaps (e.g., dynamic assemblies created due to, for example, compilation debug=”true”).

Note 2: This is a standalone script, it does not require the PowerDbg library.

Screenshots:

 

 

Source code for ShowManagedObjectsLeaked.ps1:

########################################################################################################

# Script: ShowManagedObjectsLeaked.ps1

#

# Parameters: [string] <$firstFile>

# Text file that has the output of !DumpHeap -stat

#

# [string] <$secondFile>

# Text file that has the output of !DumpHeap -stat. It should be collected minutes after the first file.

#

# Purpose: Shows the objects that are or may be causing a memory leak sorted by leak probability. It also creates

# a LogParser script to display a chart.

# To identify objects leaking the script gets the objects and number of instances from the two files, then

# it gets the objects that appear in these two files and have more instances coming

# from the second file. After that it calculates the delta, that is a number that represents the rate of

# increasing for each object. After that it displays the objects leaking sorted by this delta.

#

# Note: This is a standalone script, so it doesn't require PowerDbg.

#

# Changes History:

#

# Roberto Alexis Farah

# All my functions are provided "AS IS" with no warranties, and confer no rights.

########################################################################################################

param(

      [string] $firstFile  = $(throw "Error! You must provide the first text file that has the !DumpHeap -stat output."),

      [string] $secondFile = $(throw "Error! You must provide the second text file that has the !DumpHeap -stat output.")

     )

set-psdebug -strict

$ErrorActionPreference = "stop"

trap {"Error message: $_"}

# Do the file exist? Let's verify it.

if(((test-path $firstFile) -eq $false) -or ((test-Path $secondFile) -eq $false))

{

    throw "Error! You must provide the text file that has the !DumpHeap -stat output."

}

########################################################################################################

# Function: Parse-DumpHeapStat

#

# Parameters: [string] <fileName> - File that has the output of !DumpHeap -stat

#

# Return: Hash Table where the key is the object name and the value is the number of instances.

#

# Purpose: Extracts all objects and instances from !DumpHeap -stat and save it into a hash table.

# Note: It'll become a cmdlet forr PowerDbg soon. :)

#

# Changes History:

#

# All my functions are provided "AS IS" with no warranties, and confer no rights.

########################################################################################################

function Parse-DumpHeapStat([string] $fileName = $(throw "Error! You must provide the path and file name."))

{

    $output = @{}

    # Expression: \S+\s+(\d+)\s+\S+\s+([0-9a-zA-Z\.]+)

    # \S+ --> Scans one or more chars/digits/numbers without spaces.

    # \s+ --> Scans for one or more spaces.

    # (\d+) --> Gets one or more digits.

    # \s+ --> Scans for one or more spaces.

    # \S+ --> Scans one or more chars/digits/numbers without spaces.

    # \s+ --> Scans for one or more spaces.

    # ([0-9a-zA-Z\.]+) --> Gets all chars, numbers and '.'. Discard comma and others. The idea is to get just the namespace.

    foreach($line in $(get-content $fileName))

    {

        # Checks if the pattern matches.

        if($line -match "(\S+\s+(?<value>(\d+))\s+\S+\s+(?<key>([0-9a-zA-Z\.]+)))")

        {

            # Save the key and value in our hash table.

            $output[$matches["key"]] = $matches["value"]

        }

    }

   

    return $output

}

########################################################################################################

# Function: Save-ObjectsLeaking

#

# Parameters: <firstHashTable> - Hash table containing objects and instances.

# <secondHashTable> - Hash table containing objects and instances.

# [string] <csvFileName> - Name of the CSV file to be created.

#

# Return: CSV file that has just the objects leaked.

#

# Purpose: Compare the two hash tables with objects and instances, considering only objects that:

# 1- Appear in both hash tables.

# 2- The number of instances from the second hash table is bigger than the first hash table.

#

# After that it calculates the delta, that is the rate of increasing of objects of the same type.

# The output is a CSV file which is not sorted. The CSV file has the object as key and delta as value.

#

# Changes History:

#

# All my functions are provided "AS IS" with no warranties, and confer no rights.

########################################################################################################

function Save-ObjectsLeaking(

                               $firstHashTable  = $(throw "Error! You must provide the hash table."),

                               $secondHashTable = $(throw "Error! You must provide the hash table."),

                               $csvFileName     = $(throw "Error! You must provide the name of the CSV file that will be created.")

                            )

{

    # Prepare string to create CSV file.

    $builder = New-Object System.Text.StringBuilder

    # Title for the CSV fields.

    $builder = $builder.AppendLine("ObjectLeaked,Delta")

    

   # Scans the 'firstHashTable' and tries to locate the same object in 'secondHashTable'.

    # After that, checks if the number of instances is bigger in the second hash table.

    foreach($object in $firstHashTable.keys)

    {

        # Is the object present in the second hash table, too?

        # Notice that we want to ignore Free objects since we're not interested on them.

        if($secondHashTable[$object] -and ([string]::Compare($object, "Free", $true) -ne 0))

        {

            # Is the number of instances bigger in the second hash table?

            if([System.Convert]::ToInt32($secondHashTable[$object]) -gt [System.Convert]::ToInt32($firstHashTable[$object]))

            {

                # Calculates delta and save it.

                $builder = $builder.AppendLine($object + "," + [int] ([System.Convert]::ToInt32($secondHashTable[$object]) / [System.Convert]::ToInt32($firstHashTable[$object])))

            }

        }

    }

   

    # Send output to the CSV file.

    out-file -filepath $csvFileName -inputobject "$builder"

   

    return

}

write-Host "Extracting information from files..." -foreground Green -background Black

# Gets objects and instances from the first file.

$outputFirstFile  = Parse-DumpHeapStat $firstFile

# Gets objects and instances from the second file.

$outputSecondFile = Parse-DumpHeapStat $secondFile

$csvFile       = "ManagedObjectsLeaked.CSV"

$csvFileSorted = "SortedManagedObjectsLeaked.CSV"

# Gets just the objects leaking, ignoring Free objects.

Save-ObjectsLeaking $outputFirstFile $outputSecondFile $csvFile

write-Host "Done!" -foreground Green -background Black

# Sort CSV file and select the top 20 items based on occurrences.

import-csv $csvFile | sort-object {[int] $_.Delta} -desc | select-Object -first 20 | export-csv $csvFileSorted -notypeinformation

# Creates LogParser file.

[string] $logParser = "logparser.exe `"SELECT ObjectLeaked, Delta INTO Top20ManagedObjectsLeaked.gif FROM " + [system.io.path]::GetFullPath($csvFileSorted) + " ORDER BY Delta DESC`" -i:CSV -o:CHART -charttype:ColumnClustered -groupsize:512x480 -chartTitle:`"Top 20 Managed Objects Leaked`" -e:1"

$batFile = "Top20ManagedObjectsLeakedChart.bat"

out-File -filepath $batFile -inputobject $logParser -encoding "ASCII"

write-Host "========================================================================" -foreground Green -background Black

write-Host "`nMANAGED OBJECTS LEAKED FROM MEMORY`n" -foreground Green -background Black

import-Csv $csvFileSorted | format-Table -autosize

write-Host "`nTo see the chart copy the file " -foreground Green -background Black -nonewline

write-Host $batFile -foreground Red -background Black -nonewline

write-host " into the LogParser folder and run it to create`na GIF file that has the chart."  -foreground Green -background Black

write-Host "`n========================================================================" -foreground Green -background Black