Archive for February, 2009
An Exercise in Managing the Measurements
Feb 19th
I’m a firm believer in the phrase:
“What gets measured gets managed.”
When it comes to unit testing code coverage is something we can use to measure how much of our code is under test. By no means does this mean that the code that is under test is actually covered by valid tests that mean anything to the product at hand – it just means at some point during the test execution a path of test code passes through the given line of code. However, it is still important to know what assemblies and code paths are not covered during the testing process so teams can analyze areas that might need to be covered a bit more.
When I set up a CC.NET server I always try to convince the client that NCover (a Code Coverage Tool) is a requirement just as much as the NUnit is. As long as I can set it up rather quickly the client is happy to oblige because from a management perspective they can now see via the CC.NET Dashboard what code is covered and what code is not covered.
That is … if you understand how NCover works.
The Problem
NCover will create a profile for any assembly that is loaded into memory at runtime of the test suite. Lets think about that….
“… any assembly loaded into memory at runtime … ”
There’s a problem here. What if one of my assemblies have NO tests whatsoever? The end result is that the assembly _does_not_ show up on the CC.NET report. But… to NCover’s defense this is expected. How can NCover know _exactly_ what assemblies to profile? It can’t. The best it can do is to load the one’s it see’s during testing and profile those.
Unfortunately if your project has 10 assemblies and only 2 of them are under test you coverage percentage might be 75% – because those two assemblies might have a decent test suite. Since the other 8 assemblies were not loaded during the execution process they do not get profiled. Hmmm. problem…
So how do we get NCover to profile _all_ the assemblies that _we_ want?
Hacking NCover
I’ll be the first to admit – this is a hack. But… it works.
In order to get NCover to profile the assemblies I want covered I created a class file called “NCoverHelper.cs” with one test inside of it, shown below.
[Test]
public void This_will_load_all_Foo_assemblies_for_ncover_auto_discovery()
{
// HACK: This is a SUPER NCover hack. Read below for more info.
//
// This test is purely here to force NCover to LOAD the assembly into memory so that
// we can profile the ENTIRE stack of DLL's that we want. If we do not do this, NCover will
// only load those assemblies that are "touched" by tests. Therefore, if you have an
// assembly that DOES NOT have a test suite it would NOT show up. This test FORCES it
// to show up in the coverage report.
//
var files = Directory.GetFiles(AppDomain.CurrentDomain.BaseDirectory, "Foo.*",SearchOption.AllDirectories);
var dlls = files.Where(x => Path.GetExtension(x) == ".dll" && x.Contains(@"\build\bin\"));
// Uncomment line below to see all files the app is loading.
//File.WriteAllText(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "dlls.txt"), String.Join(Environment.NewLine, dlls.ToArray()));
dlls.ToList().ForEach(x => Assembly.LoadFile(x));
}
How it works:
The code looks in the currently executing directory, then searches for all “Foo*” files. This would return Foo.DataAccess.dll, Foo.Business.dll, Foo.Repository.dll, you get the point. The code searches all folders under the currently executing root folder. In the end, it could possibly find 10 of the same Foo.DataAccess.dll if it was referenced throughout the project.
After the dll’s are found, I filter them based upon a location. In the example above, I filter them based upon the path “\build\bin\” which is where my build is dropped.
After I have all the Foo dll’s I need, I load them via Assembly.Load(…). Yes, its kind of slow. BUT… it works.
After loading the assemblies, NCover will then profile them and find that some of them are never tested. Therefore, the report that is returned from NCover is a 0.0% coverage for any assembly that has no tests.
Please note – I’m sure there is a better LINQ query to get the info I need, but this one works for me.
Conclusion
A hack? Yes. Does it work? Yes.
In the end we are able to load our assemblies via runtime of the tests which enables NCover to profile them. The end result is that we can now see our true code coverage for our entire project, not just the projects which were “touched” by a unit test.
Build Report Will Not Show in CCNET
Feb 12th
Ok, stupid … but time consuming.
If you edit your dashboard.config in CCNET to incldue NCoverExplorer details and for some reason the report NEVER SHOWS you need to do the following:
- Restart the CCNET Service
- Reset IIS (iisreset command)
I forgot to do #2 and for an hour and a half I banged my head against the wall trying to figure out what was going on.
Oops.
Fluent Func/Expression Reflector
Feb 12th
Lou commented on the last post about using Expression and Func as an expression tool and came up with a much better solution than I had.
I’m going to repost it here for everyone else to see in the RSS reader.
This is the test class of whom we’d like to get the property (and method) names.
public class Thing
{
public int Height { get; set; }
public int Zone(string foo) { return 5; }
public void Restore(string bar, string baaz) { }
}
This is the updated reflection class that is now static and represents a more fluid/fluent syntax:
public class NameOf<T>
{
public static string Property<TResult>(Expression<Func<T, TResult>> func)
{
return (func.Body as MemberExpression).Member.Name;
}
public static string Method<TResult>(Expression<Func<T, TResult>> func)
{
return (func.Body as MethodCallExpression).Method.Name;
}
public static string Method(Expression<Action<T>> func)
{
return (func.Body as MethodCallExpression).Method.Name;
}
}
And a simple example:
class Program
{
static void Main(string[] args)
{
var name1 = NameOf<Thing>.Property(x => x.Height);
var name2 = NameOf<Thing>.Method(x => x.Zone(null));
var name3 = NameOf<Thing>.Method(x => x.Restore(null, null));
}
}
A big thanks goes out to Lou!
Using Expressions and Func as a Reflection Tool
Feb 10th
I’m using Castle Active Record for a project that I’m on and I constantly find myself having to provide column names when creating queries through the ActiveRecordMediator<T> object. Here’s an example:
ActiveRecordMediator<Customer>.FindOne(Expression.Eq("FirstName", "Bob"));
The column name “FirstName” is not strongly typed. If I change that name (or anyone else does) and they don’t use a tool like ReSharper they might run into some issues.
So I poked around a bit and found a few examples of what I was trying to do and created my own strongly typed version with this class:
/// <summary>
/// Returns the name of a property via an expression.
/// </summary>
/// <typeparam name="TypeToParsePropertiesFrom">The type to parse properties from.</typeparam>
public interface IPropertyNameResolver<TypeToParsePropertiesFrom>
{
/// <summary>
/// Parses a property from the given type.
/// </summary>
/// <typeparam name="PropertyToParse">The property to parse.</typeparam>
/// <param name="property">The property.</param>
/// <returns>The property name.</returns>
string ResolvePropertyName<PropertyToParse>(Expression<Func<TypeToParsePropertiesFrom, PropertyToParse>> property);
}
/// <summary>
/// Class responsible for parsing a property name.
/// </summary>
/// <typeparam name="TypeToParsePropertiesFrom">The type to parse properties from.</typeparam>
public class PropertyNameResolver<TypeToParsePropertiesFrom> : IPropertyNameResolver<TypeToParsePropertiesFrom>
{
public string ResolvePropertyName<PropertyToParse>(Expression<Func<TypeToParsePropertiesFrom, PropertyToParse>> property)
{
//
// If the expression body was x.FirstName, it would return a string "x.FirstName".
//
var fullPropertyName = property.Body.ToString();
//
// Parse the index of the period and get the propertyName after that.
// therefore x.FirstName would return "FirstName"
//
return fullPropertyName.Substring(fullPropertyName.IndexOf(".") + 1);
}
}
Usage:
To get a property name using this expression you will need to use a Lambda. Check it out:
IPropertyNameResolver<Customer> customerResolver =
new PropertyNameResolver<Customer>();
var propertyName = customerResolver.ResolvePropertyName(x => x.FirstName);
Assert.That(propertyName, Is.EqualTo("FirstName"));
The only real con is that I have to create an instance of that generic class to do this. But … the good thing is that its strongly typed.
Excluding Integration Tests from your Automated Build
Feb 9th
A simple tip to exclude certain tests from running during your automated build.
In this scenario we have a bunch of tests inside of a particular DLL. Inside of these tests we have a folder that is for “integration tests”. These tests are run usually once a day in a nightly build and are also run on a developers machine to ensure they didn’t break anything from an integration standpoint before checking in the code.
What we want to do is exclude the tests from running in our CC.NET continuous integration build environment. This is real simple to do by just following a couple of simple steps.
- Categorize your tests as “Integration” (or whatever you want to call them). I prefer “integration” because it signifies exactly what they are.
- Exclude them via the NUnit2 task in the NAnt build script.
Example code:
[TestFixture, Category("Integration")]
public class OrderServiceIntegraitonTests
{
// a slew of tests go here ...
}
Then in your NAnt file do something similar to the following to ensure these tests are excluded for the continuous integration build:
<nunit2 failonerror="true" >
<formatter type="Xml" outputdir="${results.dir}" usefile="true" extension=".xml"/>
<formatter type="${outputType}" usefile="false"/>
<test assemblyname="${out.dir}/DF.Example.Tests.dll">
<categories>
<exclude name="Integration" />
</categories>
</test>
</nunit2>
That’s it. By using the exclude element in the NUnit2 task in NAnt you can exclude them.
This is also a command line switch as well, so if you’re not running them in NAnt you can also exclude them via the command line switch, /exclude.