Friday, May 29, 2009

Announcing Gallio and MbUnit v3.0.6 Update 2.

Today we are releasing Gallio and MbUnit v3.0.6 Update 2.  This is mainly a bug fix release.

Download here: http://www.gallio.org/Downloads.aspx

Documentation here: http://www.gallio.org/Docs.aspx

Earlier release notes: v3.0.6, v3.0.6 update 1, all versions

Changes

MbUnit:
  • [New!] Added [Parallelizable(TestScope.All)] and [Parallelizable(TestScope.Descendants)] to recursively mark tests as parallelizable such as all tests within a fixture or all fixtures in an assembly.
  • Fixed "test was orphaned" errors involving Parallelizable tests.
xUnit.net:
  • Upgraded adpater to xUnit.net v1.1 RTM. 
NCover:
  • Added support for running NCover v1.5.8 as a 32-bit process on 64-bit machines.
Echo:
  • Fixed a regression that was causing test reports not to be generated when canceled.
ReSharper:
  • Fixed ClassCastExceptions in the ReSharper runner.
  • Fixed missing hotfix lightbulbs in the ReSharper within test files.
  • Fixed red strikethrough that would appear in the Unit Test Session window when running MbUnit tests declared in subclassed test fixtures.
AutoCAD:
  • Fixed a regression in the AutoCAD test driver.
TeamCity:
  • Improved the handling of parallel tests so that TeamCity test statistics make more sense.
Miscellaneous:
  • Improved the installer to support in-place upgrades.
  • Fixed a regression that occasionally would prevent dynamic tests from running.
Technorati Tags: ,

Thursday, May 28, 2009

Registry Weirdness, UAC and the Vista Virtual Store

It appears that two different can see a different view of the registry content.  I seem to recall seeing mention about registry values potentially being seen differently by elevated and non-elevated applications when UAC is active.

In this case, ProcessInvocation.exe sees the following value for a particular key:  (which is wrong!)

C:\Source\MbUnit\v3\src\Extensions\TDNet\Gallio.TDNetRunner\bin\Gallio.TDNetRunner.dll

But regedit.exe sees the following value:  (which is correct!)

C:\Program Files\Gallio\bin\TDNet\Gallio.TDNetRunner.dll

The first value was used for local development of Gallio v3.0.6 but I deleted that value!  In fact I deleted it a week ago and I don't think I've recreated it since (because Gallio v3.0.7 uses different keys).

The second value was set by the Gallio v3.0.6 installer (uses the old key).  This is the value I expect to see globally.  All interesting applications were closed at the the time of installation.

So why are some applications seeing the first value and other applications seeing the second value?

No, this is not a simple question of refreshing or shutting down applications.  I have shut them both down and tried again and the weird view remains persistent!

Next up, reboot.  Argh.

After the Reboot...

Same behavior.

Why?

Here's what I found when I searched the registry for the incorrect value:  (SID masked)

HKEY_USERS\S-1-5-21-xxxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxx\Software\Classes\VirtualStore\MACHINE\SOFTWARE\MutantDesign\TestDriven.NET\TestRunners\Gallio_MbUnit

Whoa.  So at least I'm not dreaming.

The UAC Virtual Store

On Vista, normal users are forbidden from writing to certain folders (like %ProgramFiles% and %SystemRoot%) and into the HKEY_LOCAL_MACHINE part of the registry.  Even local administrator users are forbidden from writing into these folders, unless elevated.

Well, not quite.  As a compatibility feature, instead of generating an error, a write by a non-elevated application running as a local administrator may be redirected to a Virtual Store on the file system or in the registry as appropriate.  The Virtual Store functions as a shallow per-user overlay of protected parts of the file system and registry.  Each user has a separate Virtual Store from all other users so misbehaving old-style applications can keep pretending they have write access to places they don't but the impact remains local to the user.

The file system Virtual Store is in %LOCALAPPDATA%\VirtualStore.  Here are a few things that mine contains:

  • _vimrc files
  • *.pyc files created by GNU Solfege
  • Reflector.cfg
  • Trillian crash dumps and user data (this explains a few oddities I've seen!)

The registry Virtual Store is in HKEY_CURRENT_USER\Software\Classes\VirtualStore.  Here are a few things that mine contains:

  • Old obsolete registry keys I created for debugging that got me into this mess.
  • A EULA key for Acrobat Reader.  (Nowadays I use FoxIt Reader.)
  • Configuration for a few audio codecs.
  • An MRU list for DirectDraw (weird).
  • A flag set by my 3G wireless software.
  • Configuration for OpenVPN.

What Actually Happened

Once upon a time, I ran a little script without elevation to create some registry keys for local development.  Since the script was not elevated, the new registry keys were created within the Virtual Store.

Later on, I ran a script to delete these keys.  But in the interim I had changed the script to run with elevation.  Because it ran with elevation it did not see the keys in the Virtual Store so it left them there.

Then I installed Gallio v3.0.6 which, being an installer, created some new registry keys with elevation in the expected location.

When I tried to run a test, the ProcessInvocation.exe component of TestDriven.Net enumerated these keys in the registry, but without elevation.  Consequently it saw the old keys in the Virtual Store and got extremely confused.

Crash!

Moral

Normally the Virtual Store on Vista is not a problem.  Actually it's a pretty clever solution to a serious compatibility problem.  However, if you ever run into issues like mine you might just need to learn about it so that you can delete or repair the magic virtualized content if needed.

More info: http://support.microsoft.com/kb/927387

Technorati Tags:

Vista UAC and Dog Food

A few days ago I re-enabled Vista UAC on my development laptop.

Why?

I was developing a few features for Gallio that required being able to perform privilege elevations on Vista.  So unless I got my act together to play nice with UAC, those features were just not going to work!

With reluctance I re-enabled UAC.  Sometimes dog food, while nutritious, contains unpleasant additives.

Teaser

Here's one of the new features from Gallio v3.0.7 in trunk:

image

When certain settings are changed, we show shield buttons:

image 

And of course when clicking on one of those buttons we'll get the standard UAC elevation dialog.  I can't show you that because by design UAC blocks out other applications including SnagIt.

How it works (very briefly)

To display shield buttons, set the Button's FlatStyle to System and then send the BCM_SETSHIELD message to the button using SendMessage.  (Or borrow my ShieldButton class.  Ignore the Mono bits.)

Before performing privilege elevation, check whether it is even needed for the current user.  Here's one way.  A better way might be to examine the Win32 ProcessToken directly.

public static bool CurrentUserHasElevatedPrivileges()
{
    WindowsPrincipal principal = new WindowsPrincipal(WindowsIdentity.GetCurrent());
    return principal.IsInRole(WindowsBuiltInRole.Administrator);
}

To perform privilege elevation, first split the code into two parts: non-privileged and privileged.  The privileged part will have to run in a separate process one way or another.  Then decide how to perform the elevation using one of these options.

  1. Use the COM elevation moniker as in the CoCreateInstanceAsAdmin example to instantiate a COM object within an elevated process provided by the system.  This method is convenient but it requires that you have registered your privileged code as a COMVisible managed class ahead of time.  Consequently, we do not use this method in Gallio which is expected to be xcopy deployable with no prior installation of components required (besides copying files).
  2. Create a separate executable process that contains the privileged code.  Be sure to embed an appropriate application manifest like the following either using the Visual Studio manifest build

    option or mt.exe.

    Our incantation of mt.exe looks a bit like this: "mt.exe -nologo -manifest Elevated.manifest -managedassemblyname:Gallio.Host.Elevated.exe -nodependency -outputresource:Gallio.Host.Elevated.exe;#1".  Keep in mind that if you use mt.exe on a signed assembly that the signature will be invalidated and the assembly will need to be re-signed with "sn.exe -R ...".  You might consider delay-signing the assembly at first, then merging the manifest, then re-signing the assembly.  This way you can tell whether you're working with an assembly with a proper manifest because the original delay-signed copy will fail validation (normally) but the patched and re-signed copy will be good.

    Here is the Elevated.manifest file that we use for Gallio.Host.Elevated.exe:

  3. <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
      <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
      <description>Provides out of process hosting for Gallio components that require privilege elevation.</description>
      <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
        <security>
          <requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
            <requestedExecutionLevel level="requireAdministrator" uiAccess="false"/>
          </requestedPrivileges>
        </security>
      </trustInfo>
    </assembly>

    Launch the privileged process directly being sure to set a few extra properties:

    var processStartInfo = new ProcessStartInfo("PrivilegedApp.exe");
    processStartInfo.Verb = "runas";   // optional if you embedded a manifest
    processStartInfo.UseShellExecute = true;  // mandatory
    processStartInfo.ErrorDialog = true;  // mandatory
    processStartInfo.ErrorDialogOwnerHandle = ownerForm.Handle;  // recommended

Ok, I just glossed over a million little details...

If you are ever in a position to have to perform privilege elevation in a managed application, I highly recommend reading the relevant MSDN articles three times over.

Technorati Tags:

Tuesday, May 19, 2009

Gallio on Visual Studio 2010, Redux

After the Microsoft PDC of 2008, I released a variant of Gallio v3.0.5 with support for the Visual Studio 2010 pre-release bits.

So today I have been working on getting Gallio v3.0.7 running on Visual Studio 2010 Beta 1.

The migration has been pretty easy for the most part.  With the PDC bits, I had a hard time producing a build because I had to install all sorts of tools inside the PDC VM and deal with rather unstable bits.  With the Beta 1 release, the process is much smoother since I can just install the framework on the build server.  Visual Studio 2010 Beta 1 is working ok although I have filed a couple of bugs already.

Unfortunately since not all contributors to Gallio will be running VS 2010 anytime soon I will have to maintain a parallel set of project files and solutions.  I guess I will be maintaining *.vs2008.csproj and *.vs2010.csproj files side-by-side for a little while.   I wish Microsoft would invest some effort in providing a smooth backward-compatible upgrade path for projects to enable developers on newer tools to work with projects created using older tools without having to actually migrate them over.  (This is my 3rd Visual Studio upgrade and I'm getting really tired of this story.)

 

The first pre-release installer should be available later this week.

Looks promising so far...

image

Technorati Tags: ,

Friday, May 8, 2009

NDepend v2.12 Rocks.

I have used NDepend on and off for the past couple of years.  It has never been an everyday thing, but whenever I have an itch to find out just how convoluted my code dependency graph has become, NDepend has been there for me.

Full disclosure: Patrick Smacchia sent me a free professional license of NDepend to play with some time ago.  To be honest, I didn't really need the license because I was already using the OSS edition of NDepend at the time.  I just think this is a cool tool.  :-)

Download and Installation

Installing NDepend is pretty easy.  Just visit the NDepend download page, type in your email address for the OSS or trial version or your license code for the progressional addition, then download the NDepend ZIP file.  Extract the ZIP someplace useful (like C:\Program Files) then pin a shortcut to VisualNDepend.exe to your Quick Launch bar.

Tools Integration

Now launch VisualNDepend.  To enable Visual Studio and Reflector integration click on the button in the UI.  A couple more clicks and you're done.

image

image

I have to say, unboxing NDepend is very slick!  There are a bunch of links to little screencast demos to explain how stuff works.  I have to admit I have tool envy here.  I wish my tools could help the user along like this.

That said, I do wish NDepend (and Reflector also) had an MSI installer flavor.  It's harder to explain to others how to extract ZIPs and create shortcuts than to just ask them to run through a standard installer process.

Dissecting Gallio with NDepend

After installing the NDepend add-in for Visual Studio, I opened up the Gallio solution.  There's a lot of stuff in here, so for this exploration I'm going to start by looking at the dependencies within Gallio.dll.

image

It's been a few months since I looked at this so I'm cringing here...  What will I discover?

Up pops an instance of Visual NDepend with this dialog listing all of the assemblies in my solution.  Pretty slick.  I don't have to create a new NDepend project from scratch.  On the other hand, I'm only interested in Gallio.dll and I did right-click specifically on that project in Visual Studio so I will remove everything else.

image

NDepend Report

The next thing that pops up is an NDepend report.  It shows up in my default web browser (FireFox).  I think it might be cooler if it appeared inside of the Visual NDepend application itself using an embedded web browser (eg. System.Windows.Forms.WebBrowser) but this report is pretty useful anyways.

Bunch of metrics regarding both the source code and the compiled binaries.  Looks like Gallio.dll consists 66% of comments.  Heh.  There's all of the XML documentation!

image

One thing to keep in mind when reading the report is that not all of the information is equally useful.  NDepend computes lots of different metrics to help you find hot spots.  Of course, if there's nothing really wrong then what it finds might not be very interesting.  You need to read the report intelligently.

Top 10 Methods to Refactor?

For example, here are the top 10 methods in Gallio.dll that I should refactor, according to NDepend.

image

Whoa, so the top 2 are compiler generated and most of the others are pretty simple methods.  Based on the criteria used (see CQL below).

image

The main thing I'm hitting up against is the IL nesting depth and number of variables.  None of these methods have a particularly troubling cyclomatic complexity except for CreateFilteredClosure which has a CC of 20 but can't really be improved much more given what it does.

So in sum, I guess I'm doing ok!  Cool.  However I bet if I threw this at some production code of my employer, we'd find some interesting things to talk about.

Let's keep digging.

Finding Large Methods

Out of the box, the NDepend report shows a summary of large methods.  This is worth paying close attention to.

Here are the 10 biggest methods in Gallio.dll.

image

I'm amused that FindMiddleSnake made the list.  That is one monster method!  Unfortunately I can't really do much to improve it because of the nature of the algorithm.  Calculating diffs is hard stuff!  The topological sort is in a similar category.

However, the PluginCatalog.ApplyTo and DefaultRuntime.VerifyInstallation methods are long for no good reason.  So with a little help from ReSharper's Extract Method refactoring here's what ApplyTo now looks like:

image

Overlap with FxCop

Some of the metrics that NDepend computes out of the box produce similar results to those that could be obtained using FxCop.  This is rather interesting since NDepend's CQL language actually has the potential to be much more expressive and configurable than the FxCop verifications, if you like.  However there can be some quirks.

Here are some methods and constructors that appear in abstract classes that were marked "public" but that NDepend thinks could be "protected".

image

The NativeCodeElementWrapper, NativeMemberWrapper and NativeFunctionWrapper constructors I understand (fixed!), the others not so much.  TestTypePatternAttribute is a public attribute class whose constructor really does need to be public because it is public API although NDepend doesn't know that.  The NativeCodeElementWrapper.Target property is only used internally in Gallio.dll so it could be made protected or internal if desired.

What it comes down to is that you still have to use your head when you interpret the data.  It's worth the effort though!

Finding Inadvertent Coupling

Of all of the many metrics and views provided by NDepend, this one is by far my favorite.  It shows the afferent coupling (incoming dependencies) in blue, efferent coupling (outgoing dependencies) in green, and mutual coupling (both incoming and outgoing dependencies) in black.  The following chart is displaying coupling among namespaces.

image

Oh no!  There's some black in there.  The black means that some code in one namespace is using code in another namespace which also happens to use code in the former.

Namespaces are frequently used to separate subsystems from one another.  Coupling among namespaces can indicate one of the following problems:

  1. One or more classes are in the wrong namespace and should be moved.
  2. One or more classes have dependencies on the wrong things and should be refactored to use dependency injection or use inversion of control to access foreign services.
  3. The namespaces are not very cohesive and the subsystems they represent should be restructured to be more tightly encapsulated or have fewer responsibilities.
  4. There are common services partially defined in both namespaces that should be extracted and moved elsewhere.

To take a deeper look I just double-click on one of the black squares to drill down.

image

Ah ha!  It looks like the Gallio.Model.Diagnostics.ExceptionData class is being used by several of the classes in the Gallio.Runtime.Logging namespace.  This is bad.

The Runtime subsystem is the lowest tier of Gallio and provides the core foundational services needed to get Gallio up and running.   The Model subsystem is one of the tiers above the Runtime and its job is to define the test object model.  What we have here is a cyclic dependency that crosses tiers.

In a tiered architecture, dependencies should always flow from one tier down to the ones below it and never the other way around.  If you let dependencies flow both ways then you may quickly find yourself enmeshed in a bowl of spaghetti!

It looks like in order to fix this problem, I'm going to have to rearrange a few of these dependencies.  The upside is that the structure of the system will be cleaner when I am done.

CQL: A Recipe for Analysis

NDepend provides a code query language called CQL which enables all sorts of fancy analysis.  In addition to answering various questions interactively, you can incorporate NDepend CQL constraints into your build process such that the build will fail if a constraint is violated.  By using CQL as part of a continuous integration process you can validate system design conventions early and often.  For example, you can add a CQL constraint to detect when cyclic dependencies are accidentally created across tiers.

To be honest, I have not delved into CQL very much myself.  Using it as part of the build process looks like it could be a very useful tool for managing large teams of developers.  (I just need to find more time to do architecture work...)

Summary

NDepend is a good tool that repays careful analysis.  It won't fix your system design defects for you but it will help you find the trouble spots.

As a result of writing this post, I fixed a few big methods and performed a major refactoring of Gallio's primary namespaces guided mainly by NDepend's dependency matrix view.  This is not the first time I have used NDepend to improve Gallio.  Here is how that dependency matrix is looking now:

image

All in all, I feel much better now!