*Words of Wisdom From The Elder

The Freedom of Virtualization

For awhile now I have been enjoying the freedom of developing with virtual machines.  The biggest advantage it has given me lately is the freedom to do whatever I want to the laptop I use as a workstation.  What I mean by that is I don’t even blink twice if I want to try a new operating system anymore.  Maybe I want to run a version of Debian Linux or Ubuntu to keep my Unix skills polished up.   Or maybe I want to try running a 64bit version of Windows Server as my main operating system to see if all of my hardware works.  To consider doing this previously without virtualization would be almost unthinkable because of the amount of time it would take to get everything back up and running. 

All of this freedom of choice is made possible through virtualization, specifically with VMWare Workstation.  VMware runs on a variety of operating systems including Windows, Linux and Mac.  Once my operating system is installed along with the needed vitalization tools I can literally go back to work by just starting up a virtual machine where all of my tools are stored.  Compared to setting up a full blown development machine this is huge.  Not only that but virtualization allows developers like myself to try out alphas, CTPs, betas and more without harming the main development operating system.  This ability to test software is made possible through cloning and what is called snapshots within VMWare. 

I’m writing this article about virtualization for two reasons.  One because I’ve spent a lot of time tweaking my setup and I wanted to document it.  And two, I know there are others that can benefit from this information as well.  In this article I’m going to cover setting up a VM (virtual machine) from scratch.  Then how to configure the operating system specifically for development in VMWare.  The purpose of this setup is to configure Windows Server 2008 as a development platform using VMWare.  Let’s get going!

Creating Your Base Virtual Machine

When starting from scratch I always create what I call a base virtual machine (VM).  This VM serves as my base image for all of my needs and will be re-used for various setups.  There are few things you need to keep in mind when you create a base image.  I’ll mark these as we go along in the screen shots below.  For posterity sakes, note that I’m using VMWare Workstation 6. 

Once VMWare Workstation is installed click File->New Virtual Machine.

Although VMWare supports Server 2008 as “experimental” I’ve found that choosing Windows Vista works just as well.  Once 2008 support is fully supported in a future release of Workstation this will not be hard to change.

You’ll notice below that I’m storing this VM on the F:\ drive not the C:\ drive.  To really get performance out of your virtual machines you need to separate them onto a different spindle or disk.  I use an external Western Digital 500GB MyBook which has a 7200RPM hard drive.  When I travel I sync my virtual machines with my portable 250GB passport drive.  This allows me to use a faster disk when I’m in the office which provides more speed and also provides a backup of the virtual machines. 

I have 4GBs of memory in my laptop of which 3.3GB is really accessible.  While I could give the VM 2GB of memory, I find that sticking between 1-1.5GB of memory seems to be the sweet spot.  If you use too much it slows the host machine down.

For networking I use bridge networking for some VMs and NAT for others.  As you’ll see later on I will create two different setups from this base image.  One will be for work, the other for presentations and speaking.  For work I use a bridge network since this allows me to separate the VPN connection from my host machine completely.  While I’m at home this allows me to access my local area network from my host machine (printers, home server, file storage, backups, etc).  For the other VM I use NAT.

Doing some reading and research the LSI Logic I/O adapter is supposedly the faster choice for SCSI adapters.

Disk capacity is something you really need to think about before setting this option.  I made the mistake of setting up a complete VM environment only to realize I needed more space.  While virtual machines can be resized using the disk manager utility, they cannot be resized if there are clones or snapshots tied to the VM.  For me this meant I had to blow away a lot of my work and go back to my base images (which is another reason why I do full cloning to start with as you’ll see later on). 

You’ll notice I didn’t choose “Allocate all disk space now”.  Essentially what this does is it reserves all of your hard drive space for your VM.  Think of it like formatting your VM image files.  As noted there is a performance gain.  However, this also means you are going to take up as much disk space as you enter in the field below.  In my case if this was checked, I’d eat up 50GB of disk space on the drive.  Not fun, especially when cloning full images (50GB + 50GB + 50GB).  Thus I do not check this option. I do however split the files into 2GB files.  The main reason is for defragmentation.  If you have one large image file you need twice the amount of space to defrag it.  With 2GB files you’ll only need an additional 2GB of space.

That’s my notes or tips for setting up a base virtual machine.  After the virtual machine is setup be sure to check your settings and configure your monitors and other peripherals.

Setting Up Windows Server 2008 for VMWare Workstation

Once you get your settings tweaked launch the VM and get the Windows startup started.  I like using server as my development operating system (OS) because it is tuned for speed unlike Vista.  There is a big difference in developing on Vista and Server 2008.  Even comparing XP and Server 2003 there was a huge speed difference.  A lot of things are turned off in server that are enabled in Vista and this is what contributes to its speed.  This setup will walk through how to configure Server as a workstation along with some specific VMWare things you need to do as well. 

Once the operating system is installed, follow these simple steps to prepare the image.

Step 1 – Tools

The very first thing you need to do after the OS boots up is install VMWare Tools.  This will provide drivers for the OS to run smoother and help you keep your sanity.  Things like the mouse jumping around randomly for no apparent reason will be fixed once this is installed.  After installed, reboot.

Step 2 – Updates

Check for updates.  This is pretty much something you need to do on any operating system you install.   Install the updates and reboot. Note: at the time of this article there are 9 updates for Server 2008.

Step 3 – Time Zone

 

Using the “Initial Configuration Tasks” screen which should be on the screen after the reboot there are a few things you need to do.  For starters check your time zone.  Review the other options while you are in there and make sure your computer name is setup the way you want and other things.

Step 4 – Shutdown Tracker

If you are familiar with Server you know it annoyingly asks you why it was restarted after it reboots.  While this might be handy for *real* servers for a workstation used for development it is just annoying.  To disable this open Start->Run->gpedit.msc.  Find the “Display Shutdown Event Tracker” as shown below, right click properties and disable it. 

Step 5 – IE Security

In order to use Server 2008 as a workstation within your VM you will need to remove the baked in security settings for IE.  From within Server Manager as show here select “Configure IE ESC”.

Set both settings to OFF.

Reboot.

Step 6 – Features

Within Server Manager you will also need to enable all the features you want. 

Here are the options I enable since I know this image is going to be primarily used for .Net development.

• .NET Framework 3.0
• Desktop Experience – This includes eye candy features from Vista as well as allows you to support the Vista Themes (we’ll enable this later on).
• Remote Assistance
• Powershell
• Windows Process Activation Service

In the next screen you’ll be asked to configure IIS.  I always select a  minimal setup but feel free to select what you know you will need.

Restart to complete installation.

Step 7 – Roles

In order to get fast file searching and other features you’ll need to turn on the role for File Services as shown here.

Since this is running in a VM depending on your usage you *could* leave this off.  I am basically going to be working day in and day out from this image so having the ability to search and find emails and files quickly is important.

Step 7 – Registry Settings (optional)

Open the registry editor by doing Start->Run->regedit

Add the following keys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters

Add a new DWORD by the name of EnablePrefetcher and assign is the value of 3
Add a new DWORD by the name of EnableSuperfetch and assign is the value of 3

What does this do?  It enables Windows to learn which programs you use so it can pre-load those applications before you run then.  It is a feature that was added in Vista.  You can also skip this step if you feel there is no value in this feature but it is *suppose* to speed things up down the road.

Step 8 – Services

There are several services we need to enable to get Server 2008 to behave more like a workstation.  These are disabled by default.

Within Server Manager select Configuration->Services.  Or you can do Start->Run-Services.msc. 

Enable the following services:

1. Themes
2. SuperFetch (only if you enabled the registry setting above)
3. Windows Audio
4. Windows Audio Endpoint Builder

Step 9 (optional) – Make it pretty

Totally optional and something you may or may not want to enable since this is running in a VM.  I’ve found that things are still pretty snappy but you may want to see how your mileage varies.  Right click on the desktop and select Theme.  Choose the Vista Theme in the drop down to get a Vista look and feel for your virtual machine.

Before:

After:

Step 10 – Setup Sound

When you install Server 2008 you’ll notice that sound doesn’t work.  Vista does the same thing so this will work in either case.  In order to get sound to work you have to install the Creative Labs PCI sound driver.  It is a pita to find so I am attaching it to this article so I can easily find it down the road as well as to ease your setup pains.

Step 11 – Activate Windows

We are going to be in the control panel tweaking some things next.  While we are here, go ahead and activate Windows.  Nothing is worse than forgetting to do this, then having to do it on every clone or snapshot down the line as you build up your VMs.

Note:  If you are using a MSDN License, you will get a DNS error code while trying to activate after installing Windows.  This is because you need to change the product key and enter your *real* product key.

Step 12 – Performance Settings

Adjust the processor scheduling which is in Control Panel->System->Advanced Settings->Advanced.

In the next tab, turn on DEP.

Step 13 – Defragment

In order to make this the fastest VM we can, we need to defrag the drive.  Defragmenting the VM is the key to keeping things fast. 

 

After I install a big product like Office, or Visual Studio I defrag the drive.  Also remember to defrag the drive the virtual machine is running from!  The more you defrag, the more performance you’ll get out of your VMs.

Step 14 – VM Optimizations

I’m not a VMWare expert but there are a few things I have found out that work well.  For one since I/O is important because the VMs will be run from an external drive, I disable page sharing.  Add this line to your .VMX configuration file.

sched.mem.pshare.enable = “FALSE”

Another option is memory trimming which doesn’t eat up host memory and then give it back when it needs it.

MemTrimRate = 0

Step 15 – Setup Your User Account

We’ve been logged into the VM as Administrator up until this point.  We need to create an account that we will use to login with and do work.  We want to create this account now because we want to set our desktop settings once and then each VM we clone from this base image we’ll already have everything setup.  Create your account in the control panel, I suggest setting the account as an administrator.  Once the account is created, log out of the administrator account and then setup that account.

Step 16 – Shrink Disk

Before moving forward we need to shrink our VM disk.  This reclaims unused space on the drive.  Note, we cannot shrink a disk if it has been cloned or has a snapshot.  We also can’t shrink a disk that has been pre-allocated.  If you followed the steps up to this point, none of this should be a concern.  There are several steps.  The first is VMWare tools prepares the image and then the second is where Workstation performs the actual shrinking.  Remember to defrag everything first!

To shrink the disk go into Control Panel->VMWare Tools.  Select the shrink tab and press the “Prepare to shrink” button.  Then wait.

Once this process finishes you will be prompted with this dialog.  Click yes to begin the process.

At this point we have a clean VM image to build other virtual machines from.  It has all the updates applied to it, is configured for workstation use, is defragged and has been shrunk.  The only thing in this virtual machine is the operating system.  This is what I call the base image.  Note:  My base image is 6.1GB after it has been shrunk.

Step 17 – Create Full Clone

Now that you have this image setup.  Create a full clone of it.  Using the clone, add your development tools like Visual Studio, SQL Server, etc.

UPnP Tip

If you are running Server 2008 on a private network and want to connect to UPnP devices and see them via network discovery then see this post which enables that.

 

Strategy For Cloning Virtual Machines

Once I get through the process above I split this base image off into two full clones.  It takes a long time to get this base image setup thus it isn’t something one wants to repeat often.  By using  a feature of VMWare Workstation called cloning we can create an exact clone of this base image to build other images. 

What I do is create two full clones of this base image.  I use one for work since it has to be bound to the domain.  The other I use for installing various pieces of software for personal use and for speaking engagements. 

For the VM at work I create one called work base.  This image will serve as my base image that I will install VPN software, Office 2007 and a few other utilities needed for work.  Once I get this image setup I create another clone of it that is a linked clone.  In this clone I install all of my development tools.  This gives me a hard restore / starting point where I can install or test software without development tools.  The reason I do all of this is it allows me to go back to points in time and try something different out without having to build everything from scratch.  For example let’s say a new version of Visual Studio comes out I need to test or something like that.  To play with that type of software it is important to start from a fairly clean machine.  Here is a chart I put together to show how I layout my virtual machines.

You’ll notice there are no snapshots listed above (another feature VMWare supports).  Snapshots are really meant for temporary use, not to run for long periods of time.  This setup gives me really good flexibility.  At least to the point to where I haven’t needed to alter it much at all.  That’s pretty much it.  Once this setup is done I don’t touch the “Server Base” image except to start it up occasionally to apply updates.  The only thing I use it for moving forward is for cloning so I have an optimized image to start with no matter the scenario.

I hope this helps someone out.  I know I wish I’d had something like this to refer to when I started out playing with virtualization images for development.  As I find new tricks or things I’m sure I’ll update this over time thus I consider this article a moving target.  One thing that is for sure is the way you do things today will not be the way you do things tomorrow.  If you haven’t explored virtualizing your development give it a try.  Here are some Words of Wisdom ‘ll leave you with:

“When you virtualize your development or main body of work, it doesn’t matter which operating system you run to host it.” – The Elder

When writing unit tests there are a couple of ways to test a bunch of scenarios.  One way is to write a test for each and every case.  That can take forever and as developers we always like to work smarter instead of harder right?  Another way is to create a data file with known expected results and then run a test on the data file.  For example, let’s say you write a regular expression, something a little complex like a regular expression to validate URLs.  How many unit tests would you have to write to cover all the possible scenarios?  One?  Two? Ten?  I know I can think of ten tests I’d want to test easily.  Using the DataSource attribute on tests with MSTest is a great way to play out an unlimited number of test scenarios.  Here’s how to accomplish this.  It is easier than you think!

Sample Solution

To showcase how this is done I created a sample solution in Visual Studio.  I created a C# library with a static class called Validator.cs.  This class will hold my static method to validate a URL with a regular expression.  My favorite place to find regular expressions is http://ww.regexlib.com.  I did a quick search for a URL regular expression and found one written by Brian Bothwell that seemed interesting.

It didn’t look too bad and seemed to account for the various things I think a regular expression like this should account for. 

I took Brian’s regular expression and added it to my Validator.cs file as seen here.

        public static bool ValidateURL(string url)
        {
            return Regex.IsMatch(url, @"^(http|https|ftp)\://([a-zA-Z0-9\.\-]+(\:[a-zA" 
                +@"-Z0-9\.&%\$\-]+)*@)*((25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]"
            +@"{1}[0-9]{1}|[1-9])\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]"
            +@"{1}|[1-9]|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9]"
            +@"|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[0-9])"
            +@"|localhost|([a-zA-Z0-9\-]+\.)*[a-zA-Z0-9\-]+\.(com|edu|gov|int|mil|net"
                +@"|org|biz|arpa|info|name|pro|aero|coop|museum|[a-zA-Z]{2}))(\:[0-9]+)*"
                +@"(/($|[a-zA-Z0-9\.\,\?\'\\\+&%\$#\=~_\-]+))*$");
        }

With my method in place I added a test project to the solution and generated a test for this method.  My initial test looked like this.

        /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [TestMethod()]
        public void ValidateURLTest()
        {
            string url = "http://keithelder.net"; 
            bool expected = true;
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

Running the test we can see it passed.

This is great to test one scenario but I want to test this regular expression with a bunch of URLs to make sure this test covers all of my possible scenarios.  Here’s how to convert this one simple test into a data driven test.

Convert Simple Test Into Data Driven Test in Visual Studio

The first thing we need to do to power our test case with data is to create a data file.  Visual Studio supports several data sources:  Database, CSV, and XML.  In my experience I have always found CSV to be the easiest and my recommended choice for creating these types of test.  The data is easy to edit with Excel and can also be easily edited with just a plain text editor.  To add a  CSV file to Visual Studio right click your test project and select add new item.  In the new window select the general category and then the Text File.  Enter the name of your file with a .csv extension as shown here.

Double click the .CSV file and add two columns.  One called URL and another called Valid.  We’ll use these two columns to store the urls we want to test with our method and the expected outcome we expect of the url in the Valid column.

Here’s a sample file:

URL,Valid
http://www.keithelder.net,true

Now that we have data let’s setup our test to use this file.  To do this is open the Test List Editor or the Test View as seen here.

Once the Test List Editor is open you should see all of your tests. 

After this screen opens, select the test in the list and press F4 or right click and click on properties.

In the properties panel select the ellipses in the “Data Connection String” field.

In the next screen select the type of data source you want to use.  Since we setup a .CSV file, the choice should be pretty obvious!

Browse to your project location and select your .CSV file.

You’ll notice that it will parse the file and display the data in the screen.   Click finish and several attributes will be placed onto the ValidateURLTest().  Here’s what the method looks like with the new attributes.

        /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
            "|DataDirectory|\\URLs.csv", 
            "URLs#csv", DataAccessMethod.Sequential), 
        DeploymentItem("TestProject1\\TestProject1\\URLs.csv"), 
        TestMethod()]
        public void ValidateURLTest()
        {
            string url = "http://keithelder.net"; 
            bool expected = true;
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

There are two new attributes besides the TestMethod attribute.  The DataSource and the DeploymentItem.   The DeploymentItem attribute is what places the file in the output directory where the tests are run from so it can be found.  The DataSource attribute specifies the type of data source (in this case csv), the name of the file and how it is to be processed.  In this example the data is to be processed sequentially.  Another enumeration option possible is to randomize the data.

Now that we’ve got the ability to pump data into our test to play multiple scenarios we only have to change a few lines to use this data.  In order to do this we’ll use the TestContext class to access the data.  Here is the change.

       /// <summary>
        ///A test for ValidateURL
        ///</summary>
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
            "|DataDirectory|\\URLs.csv", 
            "URLs#csv", DataAccessMethod.Sequential), 
        DeploymentItem("TestProject1\\TestProject1\\URLs.csv"), 
        TestMethod()]
        public void ValidateURLTest()
        {
            string url = TestContext.DataRow["URL"] as string;
            bool expected = Boolean.Parse(TestContext.DataRow["Valid"].ToString());
            bool actual;
            actual = Validator.ValidateURL(url);
            Assert.AreEqual(expected, actual);
        }

The data is accessible via TestContext.DataRow.  Notice we use the same names as the names in the columns to access our data row.  Pretty simple.  Running this our test passes.  Great!  Now, let’s add some more URLs to the test data. We’ll add some that we know shouldn’t pass and some that should pass.  Here’s the newly added data below.  Note:  Since you are using a CSV file, I suggest adding another column to the file called “Notes”.  This will help you later on to keep track of why a specific scenario was added and what you are testing for. 

Here’s my initial tests I created along with the notes that go with them.

Running our ValidateURLTest now runs all of these scenarios at one time in one test.  You might wonder where the foreach loop or for loop is to loop through all the data, but you don’t need it.  The testing framework automatically will play through all of the test scenarios in the file.  Here’s what it looks like after you view the test.

As you can see, each row in the CSV file is played through the test.  If a particular row is incorrect you’ll know exactly which row it is.  If one row fails, the whole test fails as seen here.

That’s it!  From this point forward just use your imagination and easily convert plain boring tests into more dynamic tests to get better code coverage.  The more pieces of data you throw at your tests the better.  Happy testing!

Visual Studio has long supported the concept of opening more than one file.  Most all IDE tools and text editors support this concept so it is nothing new.  In Visual Studio as files are opened they are displayed along the top in tabs (also a standard convention).  This allows developers to switch between files they are working on easily.  However there are times when developers need to compare two files at the same time.  Using Tabbed Groups developers can view files vertically or horizontally.  There is also the ctrl-tab feature which allows developers to quickly switch between files they have open.  Here is a video that shows how this is done.  (the video is below, requires Silverlight)