Acceptance Testing Made Easy with F# and Canopy

  • Article

Editor’s note: The following post was written by Visual Studio ALM MVP Amir Barylko 

Acceptance Testing Made Easy with F# and Canopy

Why acceptance testing?

Writing tests is part of our development cycle. Most teams I work with have some kind of testing in place.

In most cases I found that the tests are a mix of unit tests and integration tests.

A unit test is a test for one class or, more specifically, one method. All the collaborators should be faked in order to avoid coupling and to make sure the test is for the class and not for the dependencies.

An integration test is when two or more classes are involved. Integration is usually used when there are doubts about how the classes will work together (though a unit test should take care of that) and because writing an acceptance test is very difficult.

So far, so good—however, it is not enough.

We have tested the classes and we have figured out that some of them work fine together. Does that imply that each feature is working as expected? Unfortunately it doesn’t so we need to go a bit further.

The missing test

To ensure that the functionality the user sees is the expected one, we need a test that works the same way the user does: end to end, with no clue about classes, methods, etc.

This is known as acceptance testing, and because we do not know what is involved it is also called black box testing.

Acceptance can be done automatically, manually, or through a combination of both. Clearly, the automated version, if available, will bring lots of benefits and save us lots of time.

Furthermore, we can use acceptance testing to drive the feature in that  we can build code until the acceptance tests passes.

As tempting as it sounds, the question is then why do most projects not usually write acceptance tests? The answer is simple: because it is hard! How can we make it easier?

Enter Canopy

Canopy is an F# library that provides a DSL on top of Selenium to make writing acceptance tests for web applications quite easy. Add to the mix F#’s descriptive power and we get a winning team that can save you time and effort.

To demonstrate how to use Canopy I am going to use a project that I wrote some time ago. This project uses Knockout.js and Coffeescript to build a rich website that shows a collection of movies and lets us add movies to the collection.

You can download the code and try it for yourself from Github.

Baby steps

The MVC application shows a welcome page first.

 

On the left you can see the menu with two options. The link that says “Demo binding” leads us to the actual demo, and the list of movies loaded from the database is displayed. I am using Mongodb in this case but you can run the example with an in-memory collection if you wish.

 

I am going to write a series of acceptance tests that will test:

  • Listing movies from the database
  • Listing the default movies when the database is empty
  • Adding movies to the list

The acceptance project

In order to create an acceptance test project we need to create an F# console application.

We need to install the Canopy nuget package into the project.

Then on Program.fs we can enter the basic skeleton (that you can read about on the Canopy website).

The basics

If we were to test the site manually, the steps to do so would be something like:

  • Launch a browser
  • Open the website
  • Click the link to get to the demo
  • Do the testing
  • Check the results
  • Close the browser

And we are going to do exactly that, but by automating every step.

The first part is the same for all the tests. Here we can see the basic structure.

start firefox

url "https://localhost:1419/"

click "Demo binding"

 

// Your test goes here

 

//run all tests

run()

 

printfn "press [enter] to exit"

System.Console.ReadLine() |> ignore

 

quit()

It is almost the same as writing in English! Isn’t that nice?

Listing the default movies

Now that we have the basic structure I want to test that when my database is empty the default movies are loaded and shown on the page.

I am going to write what I want first and then we will add the support methods to make it happen.

I see the scenario using the following steps:

  • Clear the database (to make sure it is empty)
  • Go to the movie list page
  • Check that the movies listed are the ones defined by default

Below you can see the steps written in F#.

context "When the database is empty"

 

// Given I have no movies

// When I open the list of movies

before(clearMovies >> openMainPage)

 

"the default movies are listed" &&& fun _ ->

    actualMovies() |> should equal defaultMovies

The keyword context creates a new scenario that describes our test. The text for  the context will be printed to the console.

The keyword before runs a function passed as a parameter before each test.

My test is about ensuring the listed movies (actual movies) are the same as the expected ones (default movies). I am using FsUnit for the assertions.

Let’s look at the missing bits we need to complete the test.

let repo = MovieRepository()

 

let clearMovies = repo.Clear

clearMovies is a function that uses the MovieRepository class instance and calls the method Clear.

let openMainPage = fun _ ->

    url "https://localhost:1419/"

    click "Demo binding"

openMainPage is a function that navigates to the URL running my website and then follows the link to enter the demo.

let actualMovies = fun _ -> (elements ".movie .name") |> Seq.map(elementText)

 

let defaultMovies = MovieRepository.DefaultMovies |> Seq.map(fun m -> m.Title)

actualMovies is a function that uses the elements function, passing a CSS selector to obtain all the HTML elements that have a class name and a parent element with a class movie, and takes the text of each element. That should return the movie title.

defaultMovies is a function that invokes the DefaultMovies method on the MovieRepository class and selects the title from each movie.

When tests are run the console output shows the results.

 

Adding new movies

To test adding a new movie the scenario would look something like:

  • Click the add movie button
  • Enter the title and release date
  • Press the save button
  • Check that the movie is listed

And that’s exactly what I am going to write in my F# Canopy test:

context "When adding a movie to the list and is saved"

 

before(clearMovies >> openMainPage >> addNewMovie >> (fun _ -> click ".icon-ok-sign"))

"the new movie is listed" &&& fun _ ->

    let expected = Seq.append defaultMovies ["High Anxiety"]

    actualMovies() |> should equal expected

The before function, similar to the previous test, first does the cleaning and opens the main page with the movie list. The difference is that after showing the list a new movie is added by entering the information and saving the new entry to the list.

let addNewMovie = fun _ ->

    click ".your-movies button"

    ".new-movie input" << "High Anxiety"

    (last ".new-movie input") << "Sep 1, 1978"

 

addNewMovie is a function that allows clicking on a button to add a new movie and then enters the value for the title and release date field. To do so the function uses the << operator function that takes a CSS selector class and a value.

The cherry on top

Please go ahead a check the code to see all the tests in action and play with them. There is also an example using NUnit with Canopy.

In the code on GitHub in the startup process I used PhantomJS as a browser.

That way I can have a headless browser without having to launch a window and run my tests on a CI server.

Using Canopy plus F# is a killer combination for writing acceptance tests that are easy to understand and maintain and that are quick to produce. All the tools come out of the box— we just need to plug in the code and enjoy.

 

About the author

Amir Barylko started his career in 1994 working for IBM as a senior developer while he was finishing his Master’s degree in computer science. Since then he worked as team leader and architect for the past 15 years. Having started with languages like C++ and Java he spent many years coding in C# and training other developers in topics such domain modelling, abstractions, patterns, automation, dependency injection, testing, etc. Being an incurable geek, always thirsty for knowledge, his passion for technology moved him towards Ruby on Rails (RoR) a few years ago, becoming an advocate of RoR web development.

Looking for new ways of sharing his knowledge and helping others to achieve their goals motivated Amir to become an owner and build a lean management tool called SmartView (https://smartviewapp.com) that was released in 2014. Amir is a rare combination of high technical skills, experience in a wide range of platforms, exceptional presentation skills and great sense of humour. His presentations are always rich in content and fun to attend.  Follow Amir on Twitter @abarylko

 

About MVP Mondays

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager, formerly known as MVP Lead, for Messaging and Collaboration (Exchange, Lync, Office 365 and SharePoint) and Microsoft Dynamics in the US. She began her career at Microsoft as an Exchange Support Engineer and has been working with the technical community in some capacity for almost a decade. In her spare time she enjoys going to the gym, shopping for handbags, watching period and fantasy dramas, and spending time with her children and miniature Dachshund. Melissa lives in North Carolina and works out of the Microsoft Charlotte office.