*Words of Wisdom From The Elder

When writing unit tests there are a couple of ways to test a bunch of scenarios.  One way is to write a test for each and every case.  That can take forever and as developers we always like to work smarter instead of harder right?  Another way is to create a data file with known expected results and then run a test on the data file.  For example, let’s say you write a regular expression, something a little complex like a regular expression to validate URLs.  How many unit tests would you have to write to cover all the possible scenarios?  One?  Two? Ten?  I know I can think of ten tests I’d want to test easily.  Using the DataSource attribute on tests with MSTest is a great way to play out an unlimited number of test scenarios.  Here’s how to accomplish this.  It is easier than you think!

Sample Solution

To showcase how this is done I created a sample solution in Visual Studio.  I created a C# library with a static class called Validator.cs.  This class will hold my static method to validate a URL with a regular expression.  My favorite place to find regular expressions is http://ww.regexlib.com.  I did a quick search for a URL regular expression and found one written by Brian Bothwell that seemed interesting.

It didn’t look too bad and seemed to account for the various things I think a regular expression like this should account for. 

I took Brian’s regular expression and added it to my Validator.cs file as seen here.

        public static bool ValidateURL(string url)
        {
            return Regex.IsMatch(url, @"^(http|https|ftp)\://([a-zA-Z0-9\.\-]+(\:[a-zA" 
                +@"-Z0-9\.&%\$\-]+)*@)*((25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]"
            +@"{1}[0-9]{1}|[1-9])\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]"
            +@"{1}|[1-9]|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9]"
            +@"|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[0-9])"
            +@"|localhost|([a-zA-Z0-9\-]+\.)*[a-zA-Z0-9\-]+\.(com|edu|gov|int|mil|net"
                +@"|org|biz|arpa|info|name|pro|aero|coop|museum|[a-zA-Z]{2}))(\:[0-9]+)*"
                +@"(/($|[a-zA-Z0-9\.\,\?\'\\\+&%\$#\=~_\-]+))*$");
        }

With my method in place I added a test project to the solution and generated a test for this method.  My initial test looked like this.

        /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [TestMethod()]
        public void ValidateURLTest()
        {
            string url = "http://keithelder.net"; 
            bool expected = true;
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

Running the test we can see it passed.

This is great to test one scenario but I want to test this regular expression with a bunch of URLs to make sure this test covers all of my possible scenarios.  Here’s how to convert this one simple test into a data driven test.

Convert Simple Test Into Data Driven Test in Visual Studio

The first thing we need to do to power our test case with data is to create a data file.  Visual Studio supports several data sources:  Database, CSV, and XML.  In my experience I have always found CSV to be the easiest and my recommended choice for creating these types of test.  The data is easy to edit with Excel and can also be easily edited with just a plain text editor.  To add a  CSV file to Visual Studio right click your test project and select add new item.  In the new window select the general category and then the Text File.  Enter the name of your file with a .csv extension as shown here.

Double click the .CSV file and add two columns.  One called URL and another called Valid.  We’ll use these two columns to store the urls we want to test with our method and the expected outcome we expect of the url in the Valid column.

Here’s a sample file:

URL,Valid
http://www.keithelder.net,true

Now that we have data let’s setup our test to use this file.  To do this is open the Test List Editor or the Test View as seen here.

Once the Test List Editor is open you should see all of your tests. 

After this screen opens, select the test in the list and press F4 or right click and click on properties.

In the properties panel select the ellipses in the “Data Connection String” field.

In the next screen select the type of data source you want to use.  Since we setup a .CSV file, the choice should be pretty obvious!

Browse to your project location and select your .CSV file.

You’ll notice that it will parse the file and display the data in the screen.   Click finish and several attributes will be placed onto the ValidateURLTest().  Here’s what the method looks like with the new attributes.

        /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
            "|DataDirectory|\\URLs.csv", 
            "URLs#csv", DataAccessMethod.Sequential), 
        DeploymentItem("TestProject1\\TestProject1\\URLs.csv"), 
        TestMethod()]
        public void ValidateURLTest()
        {
            string url = "http://keithelder.net"; 
            bool expected = true;
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

There are two new attributes besides the TestMethod attribute.  The DataSource and the DeploymentItem.   The DeploymentItem attribute is what places the file in the output directory where the tests are run from so it can be found.  The DataSource attribute specifies the type of data source (in this case csv), the name of the file and how it is to be processed.  In this example the data is to be processed sequentially.  Another enumeration option possible is to randomize the data.

Now that we’ve got the ability to pump data into our test to play multiple scenarios we only have to change a few lines to use this data.  In order to do this we’ll use the TestContext class to access the data.  Here is the change.

       /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
            "|DataDirectory|\\URLs.csv", 
            "URLs#csv", DataAccessMethod.Sequential), 
        DeploymentItem("TestProject1\\TestProject1\\URLs.csv"), 
        TestMethod()]
        public void ValidateURLTest()
        {
            string url = TestContext.DataRow["URL"] as string;
            bool expected = Boolean.Parse(TestContext.DataRow["Valid"].ToString());
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

The data is accessible via TestContext.DataRow.  Notice we use the same names as the names in the columns to access our data row.  Pretty simple.  Running this our test passes.  Great!  Now, let’s add some more URLs to the test data. We’ll add some that we know shouldn’t pass and some that should pass.  Here’s the newly added data below.  Note:  Since you are using a CSV file, I suggest adding another column to the file called “Notes”.  This will help you later on to keep track of why a specific scenario was added and what you are testing for. 

Here’s my initial tests I created along with the notes that go with them.

Running our ValidateURLTest now runs all of these scenarios at one time in one test.  You might wonder where the foreach loop or for loop is to loop through all the data, but you don’t need it.  The testing framework automatically will play through all of the test scenarios in the file.  Here’s what it looks like after you view the test.

As you can see, each row in the CSV file is played through the test.  If a particular row is incorrect you’ll know exactly which row it is.  If one row fails, the whole test fails as seen here.

That’s it!  From this point forward just use your imagination and easily convert plain boring tests into more dynamic tests to get better code coverage.  The more pieces of data you throw at your tests the better.  Happy testing!