Introduction to setting up automated unit testing in Silverlight with NUnit

Ok, so MVVM is obviously about Unit testing right? Well, I don’t really agree, but it is definitely a part of why you chose MVVM, even if it is only a small part of the reason for me. I have been using the MVVM pattern for a while now, but I still haven’t started unit testing my code properly. I know I should, but for different reasons I never get around to it. Mostly due to time constraints.

And for all of you that tell me that writing unit tests will not take more time as there will be less bugs to fix, bug off! It does take time. It does include mocking or stubbing services. It does take time to figure out how to write useful tests. And first and foremost, it takes time to get the experience needed to do it fast… So argument ignored!

What I do do though though, is keeping it in my mind when I design my VMs. I always consider whether or no the VM is testable. If it is, then I know that I haven’t introduced any dependencies that I shouldn’t have. And even if it isn’t a fool proof way of limiting dependencies, it does help me…

I am currently working on a larger OOB application that might live on for several years and is built in a modular way using PRISM. So even after I am done with it, the application will evolve and progress, and for something like that, I find that having unit tests for the VMs seems like a very good idea. It makes you think a little extra, and verifies that you aren’t breaking things down the track.

So how do you do unit testing in Silverlight? Well, luckily there are solutions to this in the “Unit Test Framework for Silverlight”, which has now merged into the Silverlight Toolkit. So if you have downloaded the Toolkit, you are all set to go!

If you start a new project in VS2010 after the install, you have the option to create a “Silverlight Unit Test Application”. This is a small application that is prepped and ready to test your code.

So why is it an application instead of just a class library? Well, all Silverlight code must run inside the Silverlight runtime, which means it has to be run in a Silverlight application. You cannot run the unit tests as you normally would by using a “harness” inside CS, as it needs to spin up a browser and run the plug-in to be able to run the tests as they are Silverlight tests. This has some major drawbacks out of the box, but I will show you how to get around that… The biggest drawback is the fact that you get the results displayed visually in the browser, making it hard to integrate with CI for example.

But let’s hook everything up and have a look at it…

When you create the application, you get a simple App.xaml file as well as a test class. In the Application_Startup method in App.xaml.cs, you can see that it hooks up everything that you need to run unit tests, which isn’t really that much. It sets the RootVisual to the result from a single static method call called UnitTestSystem.CreateTestPage(). That’s it… That will hook everything up for you. It will create a control that hosts the test and run them for you.

The test class in turn looks like any other test class more or less. The class is adorned with a [TestClass] attribute, and the test method with a [TestMethod]. These are standard attributes for unit tests from Microsoft.

If you run the application, you will see that it launches a new Silverlight application in a browser window, and then asks you if you want to filter the tests to run or if you want to run them all. If you click “No, run all tests”, or wait for a couple of seconds, it will run through the tests and give you the result in the left hand panel. And since the currently empty test doesn’t do anything, it won’t complain or find anything wrong.

So from here, you can start writing your tests, or you can keep reading… Cause this didn’t quite suit my needs, so I needed to make some changes…

First of all, due to other parts of the solution, I should really be using NUnit for my tests. Unfortunately, NUnit doesn’t support Silverlight. But where there is a will, there is a way. So some very nice guys have created an unofficial port that runs on Silverlight. It can be downloaded at http://code.google.com/p/nunit-silverlight/.

After NUnit is downloaded, it is just a matter of adding a reference to the Silverlight NUnit assemblies (NUnit.Silverlight.Compatibility, NUnit.Silverlight.Framework, NUnit.Silverlight.MetaData), and then change the App.xaml.cs file to use NUnit instead of the default Microsoft test stuff. This change is really simple though. The UnitTestSystem class has a static method called RegisterUnitTestProvider(), which can be used to switch out the provider used. So by calling that method, passing in a new NUnitProvider, I can easily change the test system to run NUnit instead.

private void Application_Startup(object sender, StartupEventArgs e)
{
UnitTestSystem.RegisterUnitTestProvider(new Microsoft.Silverlight.Testing.UnitTesting.Metadata.NUnit.NUnitProvider());
RootVisual = UnitTestSystem.CreateTestPage();
}

That’s all there is to it…

The only thing left to get the system to use NUnit is to change the test class attributes to the corresponding NUnit attributes. Instead of [TestClass] it should be [TestFixture] and instead of [TestMethod] it should be [Test]. And if we want to set up the class before and after the tests are run, we just need to add a void parameterless method and add a [TestFixtureSetUp] attribute. This will be called each time the tests inside the class is about to run. And if you add another method with the same signature but with an attribute called [SetUp], it will be run before each test is run. All of this is documented on the NUnit site…

If we run the app now, you will see the same application and the same result. No difference whatsoever on the surface…

But, will this really work in a larger solution? Do I really want to add all of my tests as class files in the same application? Don’t I want to be able to have the tests separated depending on what they test? Well…I do! So I decided to change my implementation a bit.

First of all, I created a new class library project and added a reference to it from the test application project. In the new project, I added a reference to the project containing the classes I want to test as well as one class file for each of the classes I am testing. Nicely organized so I know where to find the tests for each class.

Honestly, I have one class library project for each assembly I am writing tests for, and then one test class for each of the classes I am testing… The only thing is that as soon as the test classes move to another assembly, the tests stop running… Oops!

So to make this work, I apparently have to tell the test system what classes to run. Lucky for me, this is really simple. The CreateTestPage() method has an overload that takes a UnitTestSettings class, which in turn has a list of assemblies to reflect while looking for tests to run. So all I need to do is to create a new UnitTestSettings class and populate that list…Sweet!

My first attempt was a miserable failure though. I created a new UnitTestSettings class, populated the list and ran the application, which failed with the message: “Test harness was not specified on the test harness settings object…”.

Ok…so do I really need to learn every setting on that class and populate them? Well…no… There is another static method on the UnitTestSystem class called CreateDefaultProvider(). This will return a new UnitTestSettings class with all the default values set…Sweet!

So all I had to do was the following

...
var settings = UnitTestSystem.CreateDefaultSettings();
foreach (var test in Tests)
{
var ass = test.GetType().Assembly;
if (settings.TestAssemblies.Contains(ass))
continue;
settings.TestAssemblies.Add(ass);
}
RootVisual = UnitTestSystem.CreateTestPage(settings);
...

But before I go any further, I also want to remove that annoying pop-up window that asks about what tests to run. This is a trivial thing, just set the UnitTestSettings property StartRunImmediately to true. And that’s where I failed again… This property apparently has to be set after the call to CreateTestPage(). But as soon as it was moved, I was good to go and my tests were running fine.

But if you were paying attention, you probably realized that I used a property called Tests in the code snippet above. So where does that come from? Well, as this thing needed to have test classes added dynamically, I though MEF would be able to help out. So what I did was that I added an [Export] attribute to my class that contained the tests. Currently it is an ugly hack that exports like this

[Export("Test")]
[TestFixture]
public class MyTestes
{
...
}

Next I declared a property of type List<object> called Tests, which I adorned with an [ImportMany] attribute like this

[ImportMany("Test")]
public List<object> Tests
{
get;
set,
}

And in the Application_Startup method I added a call to CompositionInitializer.SatisfyImports(this), which will nicely enough hook everything up for me. So after that call, I can just read all the classes that exports tests.

The last thing on my list is to get this going with the continuous integration stuff we are using. Unfortunately (or luckily) this doesn’t really support loading up a browser and then looking at the elements in the browser to see if it worked or no, so I need something else, and in this case, something else is called StatLight. StatLight is an open source project on Codeplex that makes it possible to run our Silverlight unit tests using a command line application instead of a browser. It also has TeamCity integration as well as Xml reports and a few other bits and pieces.

In this post, I just want to show that you can get it going, and not how to integrate it with TeamCity. Without a CI environment, you could still add it as a post build thing in VS.

The StatLight application consists of an exe file, some Xaps and dlls, and running it is just as simple. Open a command prompt window and navigate to the location of the StatLight executable and type in

Statlight -x=<path to xap file>

And as you press enter, it runs your tests and tells you how many tests were successful and how many that failed, which is exactly what I wanted. This can then be integrated as I said earlier into an CI environment or just set up as post build event.

And that’s it! Unit testing our Silverlight applications isn’t hard. It is actually quite simple, but it can be very helpful! (At least in larger/long running projects)

Comments (2) -

I enjoy your blog, keep up the good work!

Thanks, this was a very useful post, and well written. Thanks.

Pingbacks and trackbacks (1)+

Comments are closed