XmlSerializers, ModuleVersionId, ILMerge, and You

I solved a minor mystery today.

Gallio internally uses the .Net XmlSerializer class to load plug-in metadata and save reports.  You probably know all about XmlSerializer already, but maybe you don’t know just how badly using it can impact application start-up performance.

About XmlSerializer Code Generation

The heart of the issue is that XmlSerializer uses code generation to perform serialization and deserialization.  Specifically, XmlSerializer generates some fresh C# code, compiles it out of process with csc.exe, then loads the resulting assembly.  This work takes time.  Seconds… feels like forever to a user.  The generated assemblies are not cached across runs so there is a significant performance penalty on every launch.

There is a way to avoid this cost: pre-generate serializers assemblies and distribute them with the application.

To pre-generate a serializers assembly, you are supposed to use SGen.exe a bit like this:

SGen.exe /assembly:MyAssembly /type:MyRootXmlType

This command will emit MyAssembly.XmlSerializers.dll.  All you need to do then is copy it next to MyAssembly.dll and you’re done.  The cost of code generation is gone from every launch.

Two problems:

1. SGen only lets you specify one root type.  If you use multiple Xml document types in your assembly then you will need to write a custom tool like this: Gallio.BuildTools.SGen.
2. Make absolutely sure that you keep MyAssembly.dll and MyAssembly.XmlSerializers.dll in sync.  If you change MyAssembly.dll in any way then you must regenerate the serializers assembly.  This is easy to set up once in your build scripts and forget about.

Troubleshooting

So let’s say you do all of this work and you run your program and you still see csc.exe starting up while your program runs.

To find out why, add the following to your application’s config file.  (eg. MyApp.exe.config)

<configuration>

    <system.diagnostics>
        <switches>
            <add name=”XmlSerialization.PregenEventLog” value=”1” />
        </switches>
    </system.diagnostics>

</configuration>

Then run your program again and look at the Windows Event Log.  There should be some information in there to help you out.  (Another thing to try is fuslogvw.exe.)

Here’s the message that I saw:

Pre-generated serializer ‘Gallio.XmlSerializers’ has expired. You need to re-generate serializer for ‘Gallio.Runtime.Extensibility.Schema.Cache’.

This message is telling me that I violated rule #2 above: the serializers assembly must always be kept in sync with its parent assembly!

Why?  I’ve got fancy build scripts…

ModuleVersionId

XmlSerializer verifies that the serializer assembly is in sync with its parent assembly by comparing the module version id of the parent assembly with an id that it previously cached when it generated the serializer assembly.

Let’s check whether the assemblies are in sync using Reflector.

Gallio.dll is the parent assembly, it has a module version id of “fbe34432-817a-46dd-832f-3e5bc679ecff”.

Gallio.XmlSerializers.dll is the serializers assembly, it has a special assembly-level attribute called XmlSerializerVersion that was emitted during code generation.  It expects the parent assembly id to be “3c420916-3f3c-45cb-a79c-9a4bbc81014a".

Pedantic note: An assembly can have multiple modules each and each module has its own id.  So the XmlSerializerVersion attribute actually contains a comma-delimited list of all module version ids of the parent asssembly sorted in increasing order.

Ok, so XmlSerializer thinks the pre-generated serializers assembly is out of sync because the module id is out of sync.  That seems bananas because my build scripts always regenerate Gallio.XmlSerializers.dll whenever Gallio.dll is recompiled.

ILMerge

Well, my build scripts do one more thing: they run ILMerge on Gallio.dll to internalize certain dependencies that might otherwise get in the way of running unit tests.  (Bad things would happen if your unit testing tool exposed a dependency on one version of Mono.Cecil.dll whereas your tests depended on a different version.)

When ILMerge internalizes dependencies, it has to regenerate the assembly.  As a result, the new freshly merged Gallio.dll gets a brand new ModuleVersionId.  Unfortunately we pre-generated Gallio.XmlSerializers.dll before running ILMerge so it is now out of sync.

All we need to do is regenerate Gallio.XmlSerializers.dll after the ILMerge.  Problem solved.