OpsMgr: Monitoring Closed Monitor Alerts with a Powershell-based Monitor

"HEY!!    You have monitors in a degraded state but their alerts MAY have been closed incorrectly ! If these monitors are not reset back to Healthy, the same condition that generated their alerts may happen again but no alerts will be generated because the health states have not changed."

This solution uses the Powershell-based monitor type and wizard featured in my first post (can now be downloaded from the TechNet Gallery), for self-monitoring - to find and alert on monitor alerts that may have been incorrectly closed.

 image

 

For this Powershell-based unit monitor, the following values were used in General Properties.
Note: This monitor targets the RMS Emulator class, and hence ONLY one instance of the workflows will run on the Management Server assigned with the RMS Emulator role.

        imagel

 

The polling frequency should be set to 1 day or more and to run during non-business hours as the Powershell script used will search ALL alerts still available in the Operations Console including their owner (source) and can affect the overall performance of the RMS emulator if there is a large number of alerts and managed objects to search through.

 image

 

The Powershell script used by this monitor to find and alert on monitor alerts that may have been incorrectly closed, has been modified to return its output in a Propertybag.

Notice that the ”import-module operationsmanager” and the ”New-SCOMManagementGroupConnection” cmdlets were used to load the OperationsManager module and connect to the management group.

image

 

Here is the script used:

This script searches for closed Monitor Alerts that are not resolved by “System” and alerts if their corresponding monitor are still in Warning or Critical state:

import-module operationsmanager
New-SCOMManagementGroupConnection

$FullList=""
$newline = "`r`n"

# SCOM PropertyBag
$API = new-object -comObject "MOM.ScriptAPI"
$PropertyBag = $API.CreatePropertyBag()

$monitorAlerts = get-SCOMalert | where-object {$_.IsMonitorAlert -eq "True" -and $_.ResolutionState -eq "255" -and $_.ResolvedBy -ne "System"}
foreach ($monitorAlert in $monitorAlerts)
{
$locatedmonitor = get-SCOMmonitor | where-object {$_.ID -eq $monitorAlert.MonitoringRuleID}
$targettedclass = get-SCOMClass | where-object {$_.ID -eq $monitorAlert.MonitoringClassId}

$monitoredobject = $targettedclass | get-SCOMmonitoringObject | where-object {$_.ID -eq $monitorAlert.MonitoringObjectId}

$monitorslist = $null
$monitorslist = new-object "System.Collections.Generic.List[Microsoft.EnterpriseManagement.Configuration.ManagementPackMonitor]"
$locatedmonitor | foreach{ $monitorslist.add($_)}

if($monitoredobject.GetMonitoringStates($monitorslist) | where-object { $_.HealthState -eq "Warning" -or $_.HealthState -eq "Error"})
{
$output1 = $monitoredobject.GetMonitoringStates($monitorslist) | select MonitorDisplayName
$output1 = $output1 -replace "@{MonitorDisplayName=", ""
$output1 = $output1 -replace "}", ""
$output2 = $monitoredobject.FullName

$FullList = $FullList + "Monitor: " + $output1 + " Targetting: " + $output2 + $newline
}
}

if($FullList -eq "")
{
$PropertyBag.AddValue("State","OK")
$PropertyBag.AddValue("Description", "You are good !")

}
else
{
$PropertyBag.AddValue("State","ERROR")
$outputLongLine = "HEY!! You have monitors in a degraded state but their alerts MAY have been closed incorrectly ! They are: " + $newline + $FullList + "Please Note: " + $newline + "If these monitors are not reset back to Healthy, the same condition that generated their alerts may happen again but no alerts will be generated because the health states have not changed."

$PropertyBag.AddValue("Description", $outputLongLine)
}

$PropertyBag

 

Building expressions based on the value in the Propertybag and mapping monitor conditions to health states were very straight forward as follows. The Expression Builder Pages builds expression that looks for a particular value from the Propertybag that the data source outputs (Property[@Name='State'] ).

image 

 image

 image

 

The name of the value in the Propertybag was specified in the alert context variable: $Data/Context/Property[@Name='Description']$

image

 

 

To force an alert, close any monitor alerts in the Active Alerts view. Here is an example of the active alert generated and the state change recorded:

 image

 image

 

Attached with this post are both the self-monitoring - monitor alerts incorrectly closed management pack (TakeAWei.SM.Monitor.Alerts.Incorrectly.ClosedMP.xml) and the sealed library management pack being referenced. Both management packs will need to be imported together via the Operations Console.

 

 

Disclaimer:  

All information on this blog is provided on an as-is basis with no warranties and for informational purposes only. Use at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of my employer.

MonitorAlertIncorrectlyClosed.zip