Using the new MailSim tool to test an Outlook-compatible email server

Article
08/27/2015

In this post, we will use the new MailSim (Mail Simulator) traffic-generator tool to test the email and folder functions of an Outlook-compatible email server.

There are three actors in a MailSim test environment:

The MailSim tool (MailSim.exe), which reads user-defined XML command files and then controls
An Outlook 2013 client on the same machine as MailSim, configured with a default email profile, which then interacts with
An Outlook-compatible email server with a set of test user accounts and distribution lists.

Note: MailSim is an open source project on GitHub, so you're able to extend it with your own code.

There can be many MailSim/Outlook clients for your email server, to better approximate the actual production traffic. Each client runs independently, creating its own predefined test traffic. Clients are only aware of each other through emails.

All test traffic is specified using MailSim input XML command files, which come in two flavors: operations files and sequence files. An operation file defines a set of, well, operations, such as sending replies to all emails in a given folder. In turn, a sequence file defines groups of operations from one or more operations files. Most operations and sequences have optional repetition and delay values.

Each combination of a sequence file and its operation file(s) is called a test profile, and each MailSim/Outlook client processes one test profile at a time.

To make things more interesting, MailSim has some built-in randomness. For example, email recipients and message attachments can be specified individually, or chosen at random from a distribution list or attachments directory. In addition, you can move, reply to, forward, or delete either a specified or random number of emails. The folder creation and deletion operations can also pick their own numbers of folders.

With that explanation finally out of the way, here’s what we’re going to do:

Set up an Outlook 2013 test client and connect it to your email server.
Write a MailSim test profile that does something demonstrative with emails and folders.
Run MailSim, and see the live-action results!

Setting up the MailSim test environment

Note: There can be a lot of MailSim-created traffic, so it’s best to have a separate email server environment.

On your email server, add some test users and one or more fantasy distribution lists.
On an Outlook 2013 client machine, set up a default email profile for one of the test user names.
Note: The client machine must have .NET 4.5 installed.
Compile the GitHub source (I used Visual Studio 2013) and put the executable on the same Outlook 2013 client machine. For this post, MailSim.exe is in a directory "C:\MailSimDemo". This directory must also contain the sequence and operation schema definition (XSD) files and the sample XML files, from the MailSim/MailSim source directory and its Sample subdirectory respectively.

Writing the test profile

In this initial release, MailSim provides seven operation templates (XML elements):

· MailSend

· MailReply

· MailForward

· MailMove

· MailDelete

· FolderCreate

· FolderDelete

For this demo we’ll use each in turn: MailSend to send a few emails, MailReply to a random subset of those, MailForward one email with a random attachment, and then MailMove all those sent emails into the Junk folder. For good measure, we’ll also create six subfolders (in sets of three) with FolderCreate. We’ll use MailDelete and FolderDelete to clean up; all this debris will end up in the Deleted Items folder. Emptying that folder is left as an exercise for the reader.

Note: For this demo, we're only using one MailSim/Outlook client, so the only emails we can reply to or forward are those in our own Sent Items folder. In a typical test environment, clients will reply to and forward each other's emails, creating a more complex traffic flow.

To create our test profile, we’ll need a new operations file, and a new sequence file. These files can be created in either order, and must go in the MailSim executable's directory – in our case, "C:\MailSimDemo".

Operations file

Operation definitions are wrapped in a MailSimOperations element:

<?xml version="1.0" encoding="utf-8"?>

. . .

</MailSimOperations>

Let’s have MailSend send a simple email to three specified recipients, with one clearly spelled out attachment – in turn, the sequence file (it's coming soon!) will repeat this four times:

<MailSend OperationName="OpMailSendTo3Recipients"

Count="1"

Sleep="3">

<Recipients>TestAcct41</Recipients>

<Recipients>TestAcct42</Recipients>

<Recipients>TestAcct43</Recipients>

<Subject>Hello from the MailSim Demo</Subject>

<Body>Welcome aboard!</Body>

<Attachment>C:\MailSimDemo\Attachments\attachment3.txt</Attachment>

</MailSend>

Now we'll Reply All to a random number (signified by Count="0") of the emails in the Sent folder:

<MailReply OperationName="OpMailReplyToSentAtRandom"

ReplyAll="true"

Count="0">

<Folder>olFolderSentMail</Folder>

<ReplyBody>Remember me?</ReplyBody>

</MailReply>

Next, let's forward a single email whose subject contains "MailSim Demo" to a solitary recipient. For the recipient's entertainment, we'll toss in a random number (again, signified by Count="0") of attachments from the, say, Attachments directory:

<MailForward OperationName="OpMailForwardWithRandomAttachments"

Count="1">

<Folder>olFolderSentMail</Folder>

<Recipients>TestAcct41</Recipients>

<MailSubjectToForward>MailSim Demo</MailSubjectToForward>

C:\MailSimDemo\Attachments

</RandomAttachments>

</MailForward>

Next, have MailMove move up to one thousand of our recently sent emails to the Junk folder. The emails to move are identified by having “MailSim Demo” in the subject:

<SourceFolder>olFolderSentMail</SourceFolder>

<DestinationFolder>olFolderJunk</DestinationFolder>

<Subject>MailSim Demo</Subject>

</MailMove>

Note: In the likely case where we have fewer than 1,000 emails in the Sent folder, MailSim will reduce the Count to the current number available, and generate a warning message:

8/19/2016 2:54:36 PM Warning : OpMailMoveSentToJunk : Only 5 email(s) are in the folder, so the number of emails to move is adjusted from 1000 to 5

Finally, we’ll MailDelete up to 1,000 “MailSim Demo” emails from the Junk folder (we'll probably see the Count warning message again):

<Folder>olFolderJunk</Folder>

<Subject>MailSim Demo</Subject>

</MailDelete>

Moving on to folders! Let's use FolderCreate to make three new "Demo Folder" folders under Inbox, waiting 5 seconds after each one – the sequence file (almost there!) will call this twice, for a total of 6 new subfolders:

<FolderCreate OperationName="OpFolderCreate3Folders"

Count="3"

Sleep="5">

<FolderPath>olFolderInbox</FolderPath>

<FolderName>Demo Folder</FolderName>

</FolderCreate>

And then use FolderDelete to remove them all, or at least the first 1,000:

<FolderPath>olFolderInbox</FolderPath>

<FolderName>Demo Folder</FolderName>

</FolderDelete>

Note: All the emails we send will have "MailSim Demo" in their subject, so MailSim will only match those emails when moving, deleting, etc. Likewise for our new folders, all of which have "Demo Folder" in their names.

Sequence file

Sequence definitions are wrapped in a MailSimSequence element:

<?xml version="1.0" encoding="utf-8"?>

< MailSimSequence>

. . .

</MailSimSequence>

Once we have the operations, it’s straightforward to create the sequence file. For this demo, we'll wrap all the previous operations into a group, and go through them three times. Since we get a new log file with each test profile execution the logs really pile up, so let's put them in their own directory:

<?xml version="1.0" encoding="utf-8"?>

<MailSimSequence

LogFileLocation="C:\MailSimDemo\Logs\">

<OperationGroup Name="Demonstration times 3"

OperationFile="Ops.xml"

Iterations="3"

Sleep="20">

</OperationGroup>

</MailSimSequence>

Running MailSim

Open a Windows command window, and:

C:\MailSimTest> MailSim Seq.xml

To see the immediate results, you’ll want to have both the MailSim command window and its Outlook client window open. In our test profile, the sequence waits 20 seconds after most tasks so we can see their results live. Remember, the MailSim console messages are also saved to each run's log file.

We're starting with a completely empty Outlook client that has its default email profile set to a test user.

Caution: Be sure the Outlook client is running with a test email profile. If Outlook is currently using your own email profile, test messages may be sent all over your real network, and your personal set of email folders may be scrambled.

The first messages from MailSim show the Outlook connection data, and indicate that the magical Inbox monitor is running:

c:\MailSimDemo> MailSim Seq.xml

8/20/2015 10:38:04 AM Info : Connection : Connecting to an existing Outlook instance

8/20/2015 10:38:04 AM Info : DefaultInboxMonitor : Registered event to \\williamtest@mytestserver.com\Inbox

And now we start the group run:

8/20/2015 10:38:04 AM Info : Demonstration times 3 : Starting group run 1

Email operations

Finally we're executing the first task, OpMailSendTo3Recipients. Note that the messages show each part of the task, in this case the Subject, Body, each Recipient, and each Attachment:

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Running task 1

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Subject: 8/20/2015 10:38:04 AM - Hello from the MailSim Demo

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Body: 8/20/2015 10:38:04 AM -

Welcome aboard!

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Recipient: TestAcct41

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Recipient: TestAcct42

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Recipient: TestAcct43

8/20/2015 10:38:04 AM Info : OpMailSendTo3Recipients : Attachment: C:\MailSimDemo\Attachments\attachment3.txt

8/20/2015 10:38:05 AM Info : OpMailSendTo3Recipients : Sleeping for 3 seconds

8/20/2015 10:38:08 AM Info : OpMailSendTo3Recipients : Completed task run 1

8/20/2015 10:38:08 AM Info : OpMailSendTo3Recipients : Sleeping for 1 seconds

Since our sequence file iterates this task four times, you'll see four sets of the above messages, culminating in "Completed task run 4" and the task's specified one-second nap. Here's a snapshot of the results in the Outlook client:

Next up is our MailReply entertainment – in this case, MailSim has randomly chosen the number 3 as the number of replies to send, and so iterates itself three times:

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Running task 1

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Randomly replying 3 emails

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Starting iteration 1

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Subject: RE: 8/20/2015 10:38:17 AM - Hello from the MailSim Demo

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Body: 8/20/2015 10:38:21 AM - Remember me?

From: William Rohm

Sent: Thursday, August 20, 2015 10:38 AM

To: TestAcct41 Rohm; TestAcct42 Rohm; TestAcct43 Rohm

Subject: 8/20/2015 10:38:17 AM - Hello from the MailSim Demo

8/20/2015 10:38:17 AM -

Welcome aboard!

8/20/2015 10:38:21 AM Info : OpMailReplyToSentAtRandom : Finished iteration 1

. . .

Finished iteration 3

Completed task run 1

Sleeping for 20 seconds

And sure enough there are now four MailSend + three MailReply sent messages:

The next task is one MailForward with a random number of attachments:

Running task 1

Starting iteration 1

Randomly selecting 2 attachments

. . .

And again you can see in the Outlook client that there's another Sent message.

Next comes MailMove, and our operation will move the first 1,000 Sent messages to the Junk folder – here, the number of emails is just 8, so we see a warning message about adjusting the Count down accordingly:

8/20/2015 11:27:55 AM Warning : OpMailMoveSentToJunk : Only 8 email(s) are in the folder, so the number of emails to move is adjusted from 1000 to 8

As there are eight emails to move, we get an iteration for each one:

8/20/2015 11:27:55 AM Info : OpMailMoveSentToJunk : Starting iteration 1

8/20/2015 11:27:55 AM Info : OpMailMoveSentToJunk : Moving to olFolderJunk: FW: 8/20/2015 11:10:33 AM - Hello from the MailSim Demo

. . .

Finished iteration 8

Completed task run 1

Sleeping for 20 seconds

At this point, our Junk folder has some junk in it, so our sequence will MailDelete them. Like MailMove above, since we've specified 1,000 as the operation's Count we'll get another adjustment message. Also like MailMove, each email deleted is its own iteration:

8/20/2015 11:33:59 AM Info : OpMailDeleteFromJunk : Running task 1

8/20/2015 11:33:59 AM Warning : OpMailDeleteFromJunk : Only 8 email(s) are in the folder, so the number of emails to delete is adjusted from 1000 to 8

8/20/2015 11:33:59 AM Info : OpMailDeleteFromJunk : Starting iteration 1

8/20/2015 11:33:59 AM Info : OpMailDeleteFromJunk : Deleting email with subject: 8/20/2015 11:10:33 AM - Hello from the MailSim Demo

8/20/2015 11:33:59 AM Info : OpMailDeleteFromJunk : Finished iteration 1

8/20/2015 11:33:59 AM Info : OpMailDeleteFromJunk : Starting iteration 2

. . .

Finished iteration 8

Completed task run 1

Sleeping for 20 seconds

If you check the Outlook folders again, you'll see that there are no longer any Sent emails.

Folder operations

On to the folders – the sequence file calls the FolderCreate operation as specified, resulting in six subfolders (after all iterations are done):

Running task 1

Starting iteration 1

Creating folder: 08/20/2015 11:53:46.959 - Demo Folder

Sleeping for 5 seconds

Finished iteration 1

. . .

Lastly, we're calling FolderDelete to send those fresh folders to Deleted Items. You may recognize the execution format:

Running task 1

Only 6 folders available, adjusting delete from 1000 folders to 6

Starting iteration 1

Deleting folder: 08/20/2015 11:54:17.054 - Demo Folder

. . .

Starting iteration 6

Deleting folder: 08/20/2015 11:53:46.959 - Demo Folder

Completed task run 1

As promised at the start of this example, the Deleted Items folder is now populated with leftovers:

Our sequence file repeats this whole example three times, ending with:

Demonstration times 3 : Completed group task run 3

Demonstration times 3 : Sleeping for 20 seconds

DefaultInboxMonitor : Unregistered event to \\williamtest@mytestserver\Inbox

Interpreting the log messages

The log messages are identical to the console messages, just wrapped in XML tags:

<Detail>Registered event to \\williamtest@mytestserver\Inbox</Detail>

</Info>

<Detail>Starting group run 1</Detail>

</Info>

<Detail>Running task 1</Detail>

</Info>

. . .

Conclusion

We've seen all the initially available operations, and how they're used in a sequence file.

Note: MailSim is open source, and you're very welcome to extend or modify it!

In this example, the numbers of repetitions (Count) and Iterations are tiny, and the Sleep times are just fractions of a minute. In a more realistic scenario, massive amounts of traffic can be generated by several clients over the course of hours or days.

I hope this article has sparked your interest in creating your own test profiles, and in watching Outlook and your server(s) dance to your beat.

Using the new MailSim tool to test an Outlook-compatible email server

Setting up the MailSim test environment

Writing the test profile

Operations file

Sequence file

Running MailSim

Email operations

Folder operations

Interpreting the log messages

Conclusion

Additional resources