Being an Evil Genius with F# and .NET

A couple weeks ago I was in glorious Sandusky, Ohio presenting at CodeMash. CodeMash is a community driven conference that’s less about product announcements and typical “rah rah” and more about getting smart people together to talk about what they are passionate about – coding.

Rather than doing the prototypical “Intro to F# Talk” I figured I go with something a bit more fun and relevant to the every day developer. Sure F# is neat and everything – but why bother to learn a new programming language unless you can use it to do something meaningful. Well, in addition excelling at both functional and object-oriented programming, F# is ideal for world domination.

------------------------------------------------------

Slide1

To be honest, this may be the best talk abstract I’ll ever write.

Being an Evil Genius with F# and .NET

In today's competitive global economy it's increasingly difficult to find quality henchmen to aid you in taking over the Earth. Fortunately the .NET platform has plenty of technological gems which can help you

in your quest.

Speech recognition APIs to automate your demands for ransom. The Parallel Extensions to .NET for distributing control of your nanobots. Even image recognition to identify CIA spies and kill them on sight!

This code-focused talk will show you the libraries you can use to make your next high-powered plot for world domination a success. (F# knowledge is not required, though will help any aspiring evil genius grep the demos.)

Slide2

Someone once told me that at the beginning of a talk you should introduce yourself. Well, I’m much like you – except that my ambition isn’t to build a house or write the great American novel. It’s to take over the world!

Slide3

I’ve been very fortunate in my career as an evil genius and 2007 was a hallmark year. I had just leased a new volcano lair, had a growing army of minions, and even got a hold of real nuclear missile. The movies these days make it look like just anybody with three billion dollars can get a hold of one of these babies – not true! I had to get my minions to stage a raid on an ex-Soviet munitions dump and then kidnap a scientist to rebuild the flight systems. Hard work, the work was well worth it.

foreclosed

But the global economic disaster struck me just like everyone else. As embarrassing as it is to admit this, I couldn’t keep up the payments on my volcano and it got foreclosed on. Then my minions were all ambushed and killed by some CIA commandos. And worst of all I had to give up my nuclear missile.

Saddened, I trudged on.

Slide5

Then it dawned on me. Maybe being an effective evil genius doesn’t mean volcano lairs or orbiting space lasers, perhaps you can still terrorize the Earth with software.

Slide6

And that brings us to the meat of my CodeMash talk, being an evil genius with F# and .NET. The talk isn’t so much about F# as it is about being an evil genius. But we all know F# is the premier .NET language for aspiring villains, so along the way I’ll highlight some language constructs that are especially helpful when plotting to take over the world.

Slide7

The first problem you’ll encounter is how to actually strike your foes? What if they are behind a wall, in a bunker, or inside of a tank?

Slide8

As it turns out the terrorist training organization ‘Nerf’ had it right all along. Foam missiles are the ideal weapon.

  • Ever hear of an evil genius getting caught while trying to sneak foam missiles onto an airplane? There’s a reason for that…
  • Since they are sponges after all, why not soak them in poison to make them more lethal?

Slide9

In order to get your hands on a foam-missile-WMD you could enslave a few engineers to build you one, but if you are more in a time crunch you can pick one up at Amazon for about $40.

Dream Cheeky USB Missile Launcher

Slide10

However, this piece of specialized military hardware isn’t quite ideal for terror operations. The software it comes with is a little clunky and doesn’t offer the degree of precision required for world domination. What would any evil genius do in this case? Haxor, of course!

Slide11

A great magician never reveals his tricks, however that rule didn’t stop Pedram Amini from detailing the process of reverse engineering how the USB Rocket Launcher works.

https://dvlabs.tippingpoint.com/blog/2009/02/12/python-interfacing-a-usb-missile-launcher

Once you know the protocol for communicating with the missile launcher, it’s just a matter of reading and writing to the underlying USB device. Which you can do using P/Invoke in .NET. The code provided is in C# and was written by level-9 ninja master Matt Ellis.

 [DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern unsafe bool ReadFile(SafeFileHandle hHandle, byte* lpBuffer, uint nNumberOfBytesToRead, ref uint lpNumberOfBytesRead, IntPtr lpOverlapped);

[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern unsafe bool WriteFile(SafeFileHandle hHandle, byte* lpBuffer, uint nNumberOfBytesToWrite, ref uint lpNumberOfBytesWritten, IntPtr lpOverlapped);

/// <summary>
/// That is, call directly into the drivers for the Rocket Launcher
/// </summary>
/// On a 64-bit OS: [DllImport(@"C:\Program Files (x86)\USb Missile Launcher\USBHID.dll")]
[DllImport(@"C:\Program Files\USb Missile Launcher\USBHID.dll")]
public static extern SafeFileHandle OpenHID(ushort VendorId, ushort ProductId, ushort ReveisionId);

Once you have a wrapper on top of the rocket launcher, then it’s just a matter of writing some code to interact with it. For RocketLauncherOfDoom v1.0 we’ll just have a console interface. Something like this:

 let ckInfo = Console.ReadKey(true)
let t = 
    if (ckInfo.Modifiers = ConsoleModifiers.Shift) then 
        15
    else 
        5

let action =
    match ckInfo.Key with
    | ConsoleKey.UpArrow    -> MoveUp(t)
    | ConsoleKey.DownArrow  -> MoveDown(t)
    | ConsoleKey.LeftArrow  -> MoveLeft(t)
    | ConsoleKey.RightArrow -> MoveRight(t)
    
    | ConsoleKey.Spacebar -> Fire

    | ConsoleKey.Q  -> (loop <- false)
                       NoOp
  
    | _ -> NoOp

performAction rocketLauncher action |> ignore

However, we are far from done.

Slide12

Once you have control over your USB rocket launcher, the problem is that you spend the rest of your time either making demands for ransom or listening to people beg for their lives. Both are kind of annoying and tedious. Fortunately we can automate this.

Slide13

Build into Windows are the Microsoft Speech APIs (SAPI) which can do both speech recognition – transcribing feeble cries for help – and speech synthesis – giving your robo-army a verbal presence.

Slide14

Getting setup is simple. First, you need a quality microphone. I’ve been very pleased with my purchase of:

Plantronics Audio 995 Wireless Stereo Headset

Next, you can dramatically improve your computer’s accuracy at recognizing your voice by training your computer. Simply open up Control Panel and click ‘Speech Recognition’ and then ‘Train your computer to better understand you’. Essentially you will dictate to your computer for 10 minutes but the results are worth it.

Slide15

Once you dictate Windows into submission, it’s time for code! The System.Speech APIs come with .NET 3.0 and are quite straight forward. The following snippet prints all installed voices on your system and says “Hello, World!”

 #r "System.Speech.dll"

open System
open System.Speech.Synthesis

// Say "Hello, World!"
let prompt = new PromptBuilder()
prompt.StartParagraph() 
prompt.AppendText("Hello", PromptEmphasis.Strong)
prompt.AppendText("World", PromptEmphasis.Reduced)
prompt.EndParagraph()

// List installed voices
let synthesizer = new SpeechSynthesizer()
let installedVoices = synthesizer.GetInstalledVoices()
for installedVoice in installedVoices do
    printfn 
        "Enabled:%b, Name:%s, Age:%A, Description:%s" 
        installedVoice.Enabled 
        installedVoice.VoiceInfo.Name 
        installedVoice.VoiceInfo.Age 
        installedVoice.VoiceInfo.Description

synthesizer.Speak(prompt)

Speech recognition is even easier. Once you instantiate the recognizer object, simply add an event handler to the ‘SpeechRecognized’ event. For more information about the inner workings of the System.Speech namespace, check out the documentation on MSDN.

 #r "System.Speech.dll"

open System
open System.Speech.Recognition

let sp = new SpeechRecognitionEngine()
let defaultDictationGrammar = new DictationGrammar()

defaultDictationGrammar.Name <- "Default Dictation"
defaultDictationGrammar.Enabled <- true

sp.LoadGrammar(defaultDictationGrammar)
sp.SetInputToDefaultAudioDevice()

sp.RecognizeAsync(RecognizeMode.Multiple)

//sp.SpeechHypothesized.Add(fun args -> printfn "Hypothesized [%f] '%s'" args.Result.Confidence args.Result.Text)
sp.SpeechRecognized.Add(fun args -> printfn "Recognized [%f] '%s'" args.Result.Confidence args.Result.Text)

With a mastery of speech and sound, we can now upgrade our missile launcher software to be controlled via voice activation. The following snippet shows F#’s first-class events in action. Rather than interacting with the System.Speech APIs directly, the getRecognizer function returns an event object, to which you can add event handlers later. The result is a pretty clean implementation IMHO.

 // Returns an event handler which fires when spoken text is
// recognized
let getWordRecognizer() =
    let sp = new SpeechRecognitionEngine()
    let defaultDictationGrammar = new DictationGrammar()

    defaultDictationGrammar.Name <- "Default Dictation"
    defaultDictationGrammar.Enabled <- true

    sp.LoadGrammar(defaultDictationGrammar)
    sp.SetInputToDefaultAudioDevice()

    sp.RecognizeAsync(RecognizeMode.Multiple)

    // Return an instance of an event which fires when words are recognized.
    // Callers can then subscribe to that event.
    sp.SpeechRecognized
    |> Event.map(fun args -> args.Result.Text.ToLower())

With the speech recognized event, all we need to do is write two event handlers. One to simply interact with the rocket and another to listen for when the user wants to exit the program.

 let recognizerEvent = getWordRecognizer()

// Main handler - convert spoken text into RL commands
let handleWord spokenText =
    printfn "Recognized Word: %s" spokenText
    let action = 
        match spokenText with
        | "up"  // Has a hard time recognizing this :(
        | "north" -> MoveUp(20)
        | "down"  -> MoveDown(20)
        | "left"  -> MoveLeft(20)
        | "right" -> MoveRight(20)

        | "fire"  -> Fire

        | _ -> NoOp
        
    performAction rocketLauncher action |> ignore

// Exit handler - specifically look for exit/quit
let terminateLoop = ref false
let terminateLoopHandler = function | "exit" 
                                    | "quit" -> terminateLoop := true
                                    | _      -> ()

// Hook up event handlers
recognizerEvent.Add(handleWord)
recognizerEvent.Add(terminateLoopHandler)

while terminateLoop.Value = false do
    System.Threading.Thread.Yield() |> ignore

()

Slide16

Of course, the biggest challenge isn’t actually controlling your rocket launcher, but determining what to shoot.

Slide17

Next we’ll try to add in some computer vision to identify faces. This is where things get serious, computer vision truly is the rocket science of software. Think about it, I give you an image, which is a matrix of pixels, which are a tuple of color intensities, which are just values, and you tell me if there is a ‘face’ somewhere in that sea of numbers? The sheer idea is ludicrous, but somehow people much smarter than me have figured out how to do it.

Currently the best computer vision library is Open CV, which continues to evolve with active research. If you are interested in learning more, check out:

Book cover of Learning OpenCV

Learning OpenCV, O’Reilly

However, OpenCV is written in C/C++ and we all know that true evil geniuses use .NET. While you could just write a million P/Invoke calls and use the OpenCV library as-is, fortunately the Egmu CV is a .NET wrapper on top of OpenCV.

Slide18

 

Slide19

For the purposes of this blog post I’ll avoid the specifics of how face detection works in OpenCV. In general, OpenCV uses a technique developed by Paul Viola and Michael Jones to quickly identify ‘features’ of an image. If a region shares many ‘features’ of a human face, then a face is detected. However, the Viola-Jones method can be used for detecting more than just faces. You can actually train the computer to identify any custom ‘features’ as long as you have an ample set of example images.

For more information check out the following resources:

https://www.cognotics.com/opencv/servo_2007_series/index.html

https://www.cognotics.com/opencv/servo_2007_series/part_2/sidebar.html

https://en.wikipedia.org/wiki/Viola-Jones_object_detection_framework

Now let’s get back to sewing evil!

Slide20

To get setup detecting faces you’ll need to install OpenCV, EmguCV, and optionally get a webcam. For a cheap and viable option, try:

Microsoft LifeCam VX-1000

Slide21

Using OpenCV to detect faces is very straightforward. The following code opens up a HaarCascade (or set of features to identify a specific thing in an image) and puts a square around the provided image file.

 let detectFaces (filePath : string) =

    // Load the image file
    let img = new Image<Bgr, byte>(filePath)
    
    // Resize to something resonable
    let img = img.Resize(float 300 / float img.Width, CvEnum.INTER.CV_INTER_CUBIC)
    
    // Load the object detector
    let haarCascades = @"C:\OpenCV2.0\data\haarcascades\"
    let objectToDetect = new HaarCascade(haarCascades + "haarcascade_frontalface_default.xml")
    
    // Convert to grayscale
    let imgGray : Image<Bgr, byte> = img.Convert()
    
    let faces : MCvAvgComp[] = imgGray.DetectHaarCascade(objectToDetect).[0]
    for face in faces do
        img.Draw(face.rect, new Bgr(Color.White), 1)
        
    // Display the image
    let viewer = new ImageViewer()
    viewer.Image <- img
    viewer.Show() |> ignore

OpenCV comes with a slew of these HaarCascades for detecting faces from different angles. I' wrote a simple WinForms application where you can open all the images in a directory and see what the HaarCascades identify.

The results are hit and miss, but overall I’m very impressed with the results.

image

Slide22

While automating USB missile launchers and striking terror into the heart of men is fun, there are other worthwhile uses of these libraries. Using the System.Speech APIs you can write software to go hands free. For example, have a program dictate new F# questions on StackOverflow as they are posted. Or, training OpenCV to detect image features in real-time, such as defects in industrial manufacturing.

Slide23

If you are interested in learning more about F#, you should consider buying my book. Have fun conquering!

Disclaimer #1

All code samples are provided "AS IS" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. So in other words, if the attached source files cause you any grief, up to and including destroying your USB rocket launcher, you can’t blame me or Microsoft.

Disclaimer #2

Theme of evil genius-ery aside, consider the following. If you have above average programming skills, such as knowledge of reverse engineering debuggers or decompilers, then just like Spiderman you have amazing powers. Please consider the legal and ethical ramifications of your actions. Pedram Amini showing how to reverse engineer a USB rocket launcher is innocent enough. But you can apply the same techniques to cause real evil in the world. (For example spying on ordinary citizens.) 

The point is, like Uncle Ben said, “with great power comes great responsibility.” This applies to super heroes and evil geniuses alike.

RocketLauncher_v1.0.zip