How to unit test WIX merge modules?

Question

I am building merge modules with WIX. The batch files which calls the WIX tools to generate the merge modules from *.wxs files are run by my daily build.

I am trying to figure out how could I automate the testing of these merge modules. Things I would like to test are, whether the merge module installs required files, whether the versions of the files are correct etc.

One idea I have is to write a script (may be VB Script) to install the merge module at a temporary location and check if it has installed everything correctly. However, I am not sure if this is a correct way to do it.

Are there any standard ways of writing unit tests for merge modules? Any ideas around how to go about this are welcome.

Answer 1

When you test an installer, the primary goals are to verify that

1. When installing the msi file, msiexec reports success (i.e. return code 0).
2. After running the installer, your application can be started and works as expected.

The first point should be easy enough to do, though if you want to keep the test automated you can only test the non-interactive install. Here is an example how to do that from a batch file or on the command line:

msiexec /i myinstaller.msi /passive || echo ERROR: non-zero return code!

The second point is a bit more complicated. I think the best way to achieve this is to build some integration tests into your application, and invoke those tests after the install:

"c:\program files\mystuff\app.exe" /selftest || echo ERROR: non-zero return code!

In your case you're testing a merge module rather than an entire installer. The same approach can be used. You just will have to do the additional work of building a "self test" application and its installer which consumes your merge module.

Answer 2

I've often thought about this but haven't come up with anything that I like. First I would call this integration testing not strictly unit testing. Second the problem of "right files" and "right versions" is difficult to define.

I'm often tempted to say WiX/MSI is just data that defines what the installer is to do. It's declarative in nature and therefore by definition correct. It's tempting to want to create yet another set of data that cross checks the implementation of the installer but what exactly does that accomplish that the first data set didn't already represent? It's hard enough sometimes to maintain what files go into an application yet alone to maintain a second list of files.

I continue to think about this and wonder if there's an approach that would make sense but at this point I just do my normal MSI validation.

Answer 3

You could try to use scripts or other small console program that will do the job, just like you suggested.

With your build process you could also build a basic setup that just uses the merge module. Your script could just install this, run the other script or console app that will check if all the files are in place, that they have the correct version, that all the registry keys are installed, etc. After all the output is gathered your main script would just uninstall everything. You could also run the check program after uninstalling to be sure that everything is gone and that the uninstall works correctly. I would recommend this if, for example, you have custom actions set for install and uninstall.

Ideally this whole install / uninstall process should be done on a separate machine, or a virtual one, in order to avoid messing up the build server.

You'll have some work to do with all this scripts but once you have it, you'll be able to use it with little changes for any other future merge module projects or just plain setup projects.

Hope this would help,

Adrian.