A Very PowerShell Thanksgiving
I had the week of Thanksgiving off. So, what better time to learn a new technology. And, since I spend more time in the command prompt than anywhere else on my system, I figured I should get with the times and learn PowerShell. So, I grabbed Bruce Payette's 'Windows PowerShell in Action' and got to work.
Let me first say that this is a really well written book. Bruce explains not only the what, but also the why. I find it easier to understand a technology when I understand why the decisions were made by the team in the first place. It didn't take long before I was up and running in a PowerShell prompt. It turns out that this was the perfect technology to learn on my vacation. I really didn't want to spend much time on it, as I have been neglecting my family trying to get our Beta shipped and traveling. PowerShell and Bruce's book were the perfect combination for learning something in short bursts. I was able to play with my daughter, spend time with my wife and fit in a little learning in the odd moments I had to myself.
I'm very impressed with PowerShell. It is a simple, logical language and shell environment. My test for a new language is whether it is cohesive. In other words, does it make sense. If it makes sense, I won't have to keep referring to documentation to get stuff done. PowerShell makes sense. The commands are small, simple and consistent. The PowerShell team did a great job.
Whenever I pick up a new language, I always choose a project that I want to do and see how well I can do it with the language in question. For example, when I learned Ruby, I decided to create a Sudoku solver. Ruby was so easy to learn and consistent that I finished the solver in two hours after reading only part of a book. That is still a record for me.
For PowerShell, I decided to choose a project that fell into the systems management area. Rather than doing some boring file processing project, I figured I would try to replicate a utility I had previously written in C++. Our Test team has some automated tests that exercise Microsoft Office on a Terminal Server to test the scalability overhead of SoftGrid (ahem, Microsoft Application Virtualization). Occasionally, they run into an issue where an Office process will just start chewing up 100% cpu on a processor. It is unclear what is going on with Office, but since it happens without SoftGrid, it isn't something that they worry about. The one problem is that this issue can skew their results. So, they were constantly monitoring their test rigs so they could kill any processes that had "run away". Since I figured these guys had better things to do with their time, I wrote them a small utility that would monitor processes and kill any that had "run away". It allowed them to configure the parameters that defined what it meant to run away. The utility worked out great. It saved them from having to manually monitor these systems and made their results much more predictable.
This sounded like the perfect project for learning PowerShell. It was a pretty narrow problem. And, I figured PowerShell would be good at solving it. Boy, was I right. After reading half of Bruce's book (I have to admit, I still haven't completely finished it, my daughter took precedence J) I was able to solve the problem quickly and easily. Now, I’m not a PowerShell Ninja yet. But, I was able to code together a reasonable solution in about 40 minutes with 20+ lines of PowerShell code. This included time for getting the syntax right and figuring out one weird issue with the Process object.
To solve this particular problem, I just created a hashtable of Process objects retrieved from the Get-Process command, indexed by their process id. Then, I took another snapshot of processes after a configurable period and calculate how much CPU they had used. Simple. But, when I first tried this, it didn’t work. The reason is that the CPU property on the Process object is dynamic. It is not a snapshot in time. Each time you query it, it gives you the latest amount of CPU used for the process in question. So, I had to store the current value somewhere. Luckily, PowerShell allows you to attach properties (it calls them ‘notes’) to an existing object. This made my life much easier, since it meant I didn’t have to invent a new object to store the Process object and my custom property. Also, since I wanted to have multiple samples before I killed a process, I needed to keep track of how long the process had been “in violation” of the policies handed into the script. Again, PowerShell’s notes came to the rescue. I could store the number of times a process was in violation and only kill it after it had violated the threshold number of cycles in violation.
In the end, I added a couple of other small features to the script. For one, I decided to make it configurable whether it would kill the process or just lower its priority to Idle. This seemed more useful for regular situations where you might not want to kill the process, but still want your system to be responsive. I also added an exclusion for processes running in Session 0. This made sense, since only Services and System processes should be running in Session 0 (at least, on Vista).
Here is the output of the tool, with an example process that was taking up too much CPU (given the name of it, no wonder). One thing you'll note is the initial error message that I get when I launch PowerShell. Every time I launch PowerShell, I get the following error: "Attempting to perform the InitializeDefaultDrives operation on the 'FileSystem' provider failed." It turns out that this error occurs when PowerShell attempts to access a drive which it doesn't have permission to. Now, this is a very atypical case. In my case, it is because I have Microsoft Application Virtualization installed and the drive is not accessible from this particular PowerShell process. So, I get the error. PowerShell is still functional. And, I have a feeling it will be fixed in the future.
Anyway, below is the script I finally came up with. Now, as I said, I still haven’t reach Ninja status in PowerShell. I’m sure that it looks too much like a C++ code and there are probably tons of things that a seasoned PowerShell programmer would do differently. Still, it solved my problem quickly and efficiently. Isn’t that the point of scripting languages? J I considered commenting the code to help you understand what is going on. But, I decided to leave it as is. In its own way, I think it is beautifully simple. It is surely much simpler than the C++ code that it replace. Happy PowerShelling.
param([int] $period=5, [int] $maxCpu=$period*0.9, [int] $threshold=5, [bool] $kill=$false)
$procs = @{}
while($true) {
# Create hash table of processes, hashed by process id
$new_procs = @{}
foreach ($proc in Get-Process) {
add-member -in $proc noteproperty periodsAboveMaxCpu 0
add-member -in $proc noteproperty cpuUsageSnapshot $proc.cpu
$old_proc = $procs[$proc.id]
if($old_proc -ne $null) {
$proc.periodsAboveMaxCpu = $old_proc.periodsAboveMaxCpu
if(($proc.cpuUsageSnapshot - $old_proc.cpuUsageSnapshot) -gt $maxCpu){
$proc.periodsAboveMaxCpu++
} else {
$proc.periodsAboveMaxCpu = 0
}
}
$new_procs[$proc.id] = $proc
}
$procs = $new_procs
$bad_procs = $procs.keys | foreach{$procs[$_]} | where {$_.periodsAboveMaxCpu -gt 0}
$bad_procs
foreach ($proc in $bad_procs) {
if(($proc.periodsAboveMaxCpu -gt $threshold) -and ($proc.SessionId -gt 0)) {
if($kill) {
write-output "Killing $proc"
$proc.Kill()
} else {
write-output "Dropping priority of $proc to idle"
$proc.PriorityClass = [System.Diagnostics.ProcessPriorityClass]::Idle
}
}
}
sleep($period)
}