Working inefficiently in the test team

  • Article

I've spent the last two weeks being very inefficient. Obviously, I had a reason for this and it goes something like this.

We have some test tools that we run every day to help find bugs in OneNote. The tools are great and generate TONS of information about how current iterations of OneNote are working, with most of this information being kept in log files or memory dumps (if certain conditions are hit). The log files can be 3-30MB in size, and you may have 0-2 memory dumps to investigate. And it has been in the area of investigation I have been inefficient.

My role for this tool is to set up some background tasks beforehand, such as creating a notebook for the tool to use. At first, I wanted to create a tool to create a new notebook, but I held off on this. It takes under a minute to create a clean notebook - it's just a simple drag and drop file copy of an empty "canned" notebook - so at most I would save 10 minutes per month with this tool. It would probably take me about an hour or two to create, debug and check this tool into our repository. Even though that is a minimal hit, it will take at least a year or so two pay off.

Moving on, I investigated several of the tool results by digging through log files. My inefficient routing went something like this:

  1. The log files took a little time to open, then parsing through them by hand looking for error could take up to half an hour.
  2. Once that was done, trying to trace the root cause of any errors I had found could take up to another half hour to (in rare cases) a few days to isolate. Let's call this 10 minutes on average.
  3. On days I had memory dumps to investigate, I spent about 20 minutes for each.
  4. Other items to check, such as size of the OneNote cache, number of embedded files the machine had and other information like this took a minute or two at most.

So obviously the bulk of my time was being spent here. It totals to just over an hour each day.

Now consider that up to 20 other testers may be running the tool each day and have to go through the same steps. That's 20 hours spent by the team per day just investigating the results of this tool. Per week, that is 2 1/2 testers spent full time on this - a huge cost.

And by working inefficiently and costing this out, I realized where I could add some tools to help the entire team work more efficiently. I've said this before - this is not earth shattering information by any means. I just wanted to be able to know, as opposed to guess or estimate, how much time was being spent here.

More on that on Monday.

Questions, comments, concerns and criticisms always welcome,

John