Disadvantages of Ruby for Test Automation

  • Article

In the past few years there has been a push to use Ruby as a programming language for test automation. Ruby as a programming language has some benefits compared to other scripting languages. However, for test automation or any other serious application development the disadvantages of Ruby certainly outweigh any of the recent sensational hype promoting the language. I may be a bit biased, but I honestly can’t fathom why a tester wanting to learn a programming language would spend time learning Ruby, especially with the ease of use, availability of resources, and broad adoption of C#.

1. Ruby lacks informational resources. A search on Barnes & Noble, or Amazon reveals about 50 or so books on Ruby programming. But, that is barely a drop in the bucket compared to more than 400 books written about C#. These numbers certainly don’t inspire a lot of confidence in Ruby as a broadly accepted programming language in the industry at large. Web searches for available online resources also reflect this tremendous disparity. Sure, the Ruby zealots support the few websites and respond to requests for assistance. But, there are a greater number of C# forums with greater numbers of registered members who frequently participate and provide solutions to questions. Additionally, you won't find too many universities or community colleges offering courses in Ruby programming. If Ruby is so good then why are there such limited resources? The answer is because there simply isn't the business demand, or other compelling reasons for adoption.

2. Ruby is not a high demand skill among employers. Take a look at any of the technical job sites such as Dice, Monster, etc. and you will not find a plethora of jobs asking for Ruby programming skills. For example, of the approximately 16,500 software testing jobs on Dice only 65 contain the keyword Ruby as compared to 1,668 job listings containing the keyword C#. That means there are 25 times more employers desiring C# as compared to Ruby. IT Jobs Watch in the UK has an interesting site with lots of statistics relative to software testing positions in the UK. Looking at software testing jobs C# is listed in the top 5 desired programming languages. (Ruby doesn’t even make the top 20 list anywhere on this site.) The job trends on Indeed provides a visual perspective comparing jobs with C#, Ruby, Perl, and VB.NET keywords.

3. Ruby has performance problems. Scripting languages are notoriously slower than compiled languages, but it seems that Ruby is often slower (CPU time) and requires a larger memory footprint as compared to other scripting languages. The Computer Language Shootout site provides some very interesting benchmark results for various languages. One benefit of automation is the decreased time to execute a specific test. Now, you may be thinking that it only takes a few more seconds to execute a test in Ruby as compared to Java or C#. So, let’s arbitrarily say that each automated script in Ruby takes 5 seconds longer to execute as compared to the same automated test in C#. That may not seem like a big deal. But, instead of running one or two automated tests you want to run a test library of 200,000 tests. That’s about 12 hours of additional time needed to run the test automation written in Ruby. Now, I am sure Ruby advocates will discuss the reduced cost of development time, but time to develop an automated test is a one time expense (this does not include sustained maintenance which is a variable cost compounded over time for all automation libraries). Depending on your product’s shelf-life, you may need to rerun your test automation suites for the next 7 to 10 years for sustained engineering and product maintenance.

 

4. Ruby has not been widely adopted in application development. So, you may ask why this is a weakness for testers writing test automation? The simple fact is that if the development team is programming in say C/C++ or Java, and the test automation is in Ruby you probably won’t get a lot of support from the development team to help reviewing or debugging test automation. Also, it is very likely the developers may not want to install the Ruby interpreter on their Windows machine to use test automation to reproduce a defect, and instead ask for the manual steps. The test libraries the development team creates will require porting to Ruby which increases the cost and effort. Since many developers are familiar with at least the basic syntax of C/C++ and Java it is easier for them to pick up C# syntax and understand automated test code.

 

5. Ruby is just as cryptic as any other programming language. All programming languages use unique syntax, and users must learn the language’s syntax to code effectively. Now, I am no expert in Ruby but let’s compare a Ruby script to launch Windows calc.exe as compared to a C# program.

 

--------------------------------------------------------------

 

# Launch calc.exe in Ruby

require “win32/guitest”

include win32::GuiTest

 

run_cmd “calc.exe”

 

-----------------------------------------------------------------

 

// Launch calc.exe in C#

using System;

using System.Diagnostics;

 

namespace MyNameSpace

{

    class MyClass

    {

        static void main ()

        {

            Process.Start(“calc.exe”);

        }

    }

}

 

-----------------------------------------------------------------

 

Obviously there are more lines of code in the C# program as compared to the Ruby script. But, considering the fact the template in Visual Studio auto-generates the framework for a console application (the primary method of writing an automated test case) the only thing I need to add to the .cs file are the ‘using System.Diagnostics’ namespace declaration, and the ‘Process.Start(“calc.exe”);’ statement. Additionally, the IntelliSense feature of the Visual Studio IDE references language elements, and even inserts the selected element into the code. Also, perhaps it is a matter of personal taste, but Process.Start() seems a lot more ‘readable’ than run_cmd.

 

Ruby activists boast how quickly they can teach Ruby scripting to non-programming testers. I have been teaching C# to testers for more than 3 years. I have been very successful at teaching testers with no previous programming skills to write automated GUI tests in C# that will launch a Windows application, manipulate the application, generate and send data, verify and log results, and clean up the test environment within a day.

 

There may be some interesting features of Ruby, but don’t get sucked in by all the fanatical propaganda. Ruby has been around for more than 10 years and hasn't replaced any language or garnered a significant following! The simple fact is that Ruby simply doesn't offer anything revolutionary, and thus hasn't compelled the development community to rush to adopt or support it. All programming languages have strengths and weaknesses depending on the application. But, for test automation Ruby is not the best choice as a programming language.

 

In my (biased) opinion, I think C# is a much better choice, and in a later post I will outline the benefits of C# as a programming language for test automation.