Registering a CPP dll into COM after installation

I am trying to register a CPP library into COM during the Msi installation.

I have searched a lot and found many solutions in here, but nothing is working in my code. I don't know is there any direct method for this. I have tried using Custom Action with direct ExeCommand and with a batch script.

Here is the code with batch script.

<SetProperty Id="Register" Value="&quot;[INSTALLDIR]Scripts\Register.bat&quot;" After="CostFinalize"/>
<CustomAction Id="Register" BinaryKey="WixCA" DllEntry="CAQuietExec" Execute="deferred" Return="check" Impersonate="no"/> 

<SetProperty Id="Unregister" Value="&quot;[INSTALLDIR]Scripts\UnRegister.bat&quot;" After="CostFinalize"/>
<CustomAction Id="Unregister" BinaryKey="WixCA" DllEntry="CAQuietExec" Execute="deferred" Return="check" Impersonate="no"/>

Using this code, installation does not show any error, but dll is not getting registered. After installation if I ran the batch script separately then it gets registered.

Register.bat

cd "C:\Windows\System32"

regsvr32 "C:\Program Files (x86)\ABC\Abc.dll"

ping -n 15 127.0.0.1 >nul:

Unregister.bat

cd "C:\Windows\System32"

regsvr32 /u "C:\Program Files (x86)\ABC\Abc.dll"

ping -n 15 127.0.0.1 >nul:

With Custom Action with ExeCommand it shows error like some dll dependency missing. ExeCommand code is given below.

<CustomAction Id="Register" Directory="INSTALLDIR" Execute="deferred" Impersonate="no"
              ExeCommand="[WindowsFolder]System32\regsvr32 &quot;[INSTALLDIR]Abc.dll&quot;" Return="check" />
<CustomAction Id="Unregister" Directory="INSTALLDIR" Execute="deferred" Impersonate="no"
             ExeCommand="[WindowsFolder]System32\regsvr32 /u &quot;[INSTALLDIR]Abc.dll&quot;" Return="check" />

And InstallSequencefor both the cases is given below.

 <InstallExecuteSequence> 
    <Custom Action="Register" Before="InstallFinalize" >NOT Installed</Custom>
    <Custom Action="Unregister" Before="RemoveFiles">Installed AND NOT UPGRADINGPRODUCTCODE</Custom>
 </InstallExecuteSequence>

In both the case I think it is running with elevated privileges.

This is the error I get most of the time.

Edit

Dependency walker view of the dll is shown below.

Also I am adding the heat command I used. I have added this to prebuild event to generate the component. After that added this component in product file.

call "$(WIX)bin\heat.exe" file "dllPath\Abc.dll" -dr "INSTALLDIR" -srd -gg -sfrag -suid -out "$(SolutionDir)Installer\ComRegisterComponent.wxs"

And the generated file is look like this.

 <Fragment>
    <DirectoryRef Id="INSTALLDIR">
        <Component Id="Abc.dll" Guid="*">
            <File Id="Abc.dll" KeyPath="yes" Source="SourceDir\Abc.dll" />
        </Component>
    </DirectoryRef>
</Fragment>

Here this SourceDir path is confusing me.I have added the exact path in heat command even then it generates this SourceDir.

回答1:

Short, Summary Answer

You need to stop using batch files and custom actions for COM registration (unreliable) and rather extract the COM registration information using the heat.exe tool from the WiX toolkit to add the COM registration to your MSI database at compile time.

There are some complications for 64-bit binaries, see details below. Fortunately it looks like you are dealing with a 32-bit component based on your installation directory shown above.

In this particular case it helped to run heat.exe on the COM file after deployment when all dependencies were "in place" for the COM file to load properly. There is a lot of "debugging communication" in these answers - I'll leave it all in place for the future, but try this simple solution first. And perhaps try the new dependency tool "Dependencies.exe" described below.

Long, Detailed Answer

Before I try to answer the question (which seems to revolve around missing dependencies or something weird being done in your batch file), I want to clear up a few things for you with regards to best practice for COM registration.

Note: the screenshot seems to indicate something weird going on in your batch file, but missing dependencies could still be an issue.

Self-registration Considered Harmful

Self-registration should not be used to register COM files. Here is a description of why this is the case: MSI register dll - Self-Registration considered harmful. There is good news, however, doing things as intended via built-in MSI mechanisms will be both easier and more reliable once you set it up properly.

There is a way to self-register files during installation without using custom actions like you are trying to do (the SelfReg table). Custom actions are very hard to get working correctly, but you should not use the built-in mechanism to run self-registration either (as explained in detail in the linked answer above)

Rather than using custom actions or the SelfReg table, COM registration information should be extracted from your COM files at compilation time - in other words when you compile your MSI file from your WiX source files. The extracted registry data should be used to populate the family of MSI data tables designed to reliably register and unregister the COM file during installation and uninstallation respectively.

WiX: The "heat.exe" Command Line Tool

Understanding the intricate details of this process is not necessary - all you need to know is what tools to use. WiX provides the "heat.exe" tool for this purpose. It is essentially a "harvester" tool capable of generating valid WiX XML source files for several purposes - one of which is COM extraction. It also supports traversing directories in general - generating WiX source files which can install the files encountered during traversal. It is essentially a very quick way to make an MSI package once you know how to use it.

Dependency Walker

So we have established that you should take the time to learn how to use heat.exe to generate the WiX source necessary to register the COM file properly. however, there is one more problem: the missing dependencies.

For a COM file to be able to self-register - or for you to be able to successfully extract the COM registry data using heat.exe - the COM file must be able to load correctly. For this to be possible all dll dependencies must be available on the system in question in an accessible location.

Get yourself a copy of Dependency Walker and use it to scan your COM file for what files it depends on. Here is an example of a COM file which fails to load because it cannot find MMUtilities.dll:

You will most likely find something similar wrong with your dll (or whatever file type it is, for example OCX) when it is run from your setup's installation location. The required dependency files can not be found by regsvr32.exe and the registration process fails.

There are some reported missing dependencies that are not important - I guess this has to do with the age of the Dependency Walker tool - it hasn't been updated recently as far as I know. Look for a file you recognize as either your own dependency file or a core system file as opposed to very long dll names of files you have never heard of. Keep in mind that some dlls have dependency language dlls that are required for loading. For example MMUtilities.dll needs MmUtilitiesEnglish.dll or another language dll present in the same folder to be able to load correctly.

Some sample false positive dependencies for the above file: API-MS-WIN-CORE-RTLSUPPORT-L1-1-0.DLL, API-MS-WIN-CORE-PROCESSTHREADS-L1-1-0.DLL, API-MS-WIN-CORE-REGISTRY-L1-1-0.DLL, etc... There were many. I believe, but I am not sure, that the root cause of these false positives centers around problems with side-by-side components installed to the WinSxS folder, but that is a whole other discussion - just mentioning it.

UPDATE: I just checked this again and most of the false positive dependencies seen above are apparently API-sets - a Windows feature introduced long after Dependency Walker was created, and hence not handled correctly by the tool.

As the linked answer indicates (do read it, link above), there is now also a rewrite of Dependency Walker in C# called "Dependencies", available here: https://github.com/lucasg/Dependencies (untested by me at time of writing, will test shortly).

Also quick mention: if you have an executable to check (as opposed to a dll, ocx, etc...), you can launch it via the Profile => Start Profiling... menu entry and you will then also see "hidden run-time dependencies" not specified in the binary import table (where dependencies / imports are specified). You need to really exercise the application using all features in all dialogs to be sure to get all such dependencies.

The dependency walker web page calls these hidden dependencies explicit dependencies (a dynamic or run-time dependency) and system hook dependencies (injected dependencies). See the link just above for more details.

Heat.exe Extraction

Once you have determined what files are missing for making your WiX heat.exe extraction work, you can put the files in question "next to" your COM dll so they are found during its loading. You can test run using regsvr32.exe to see if registration completes correctly. There should be no error messages when you run registration manually like this. Remember to run registration from an elevated command prompt.

Several other stackoverflow answers explain how to use heat.exe - I haven't used it in a long time: How to run heat.exe and register a dll in wix. Here is the official documentation for heat.exe from the WiX guys themselves. It can be a somewhat intimidating tool - it has a lot of features.

In its simplest form (for normal 32-bit COM files with all dependencies available in the path or local folder), you can run this heat.exe command line to generate an output WiX source file called YourFileName.wxs with all the required COM registry data.

heat.exe file YourFileName.ocx -o YourFileName.wxs

I wrote an answer several years ago showing how to incorporate the exported WiX registry data into your main WiX source: How to Reference a Heat Output(wxs) in Wix (Command Line). I believe this is an accurate description of the procedure, but for some reason someone down-voted the answer. Please give it a go and see if it works for you.

Important!: heat.exe does not yet correctly process 64-bit COM binaries (December, 2017).

I have been informed that the WiX Expansion Pack (not free) handles 64-bit binaries and provides several other features. I suppose I can link to it (I am not affiliated with FireGiant): https://www.firegiant.com/wix/wep-documentation/harvesting/. Somehow people need to know about these things, but I am not sure about the the stackoverflow etiquette of linking.

Are your binaries 64-bit? (it doesn't look that way from your installation folder, but I add this for others who might find it). For a 64-bit component I guess we have come full circle and have to advise you to either use the above expansion pack feature or just try to set the file to self-register as described here. I hate this self-register "solution", but I can't think of any other quick fixes at the moment (none that I would recommend anyway). I will check again. Make sure to check the latest WiX release to see if the 64-bit problem has been fixed before going for this "self-registration fix". It is at least better than trying to register using custom actions and batch files (which should never be attempted - there are so many potential problems relating to MSI's complex custom action sequencing, impersonation / elevation, conditioning,installation modes with silent and interactive, not to mention the potential interference from security software, and the list goes on).

Some links:

How do you register a Win32 COM DLL file in WiX 3?
Cannot register DLL using WiX
Use wix to perform function of REGSVR32
Heat.exe Cannot Harvest TypeLib Info from a 64-bit COM DLL
Heat.exe: 64-bit .dll fails to be converted to a 64-bit .msi

回答2:

Just adding another answer with information on how to use procmon.exe in an efficient manner to debug missing dependencies during self-registration using regsvr32.exe.

Essentially you should set an include filter for what events you want to monitor and record, otherwise you get an almighty list of irrellevant events listed and it is hard to find what you need in the sea of useless information:

To limited the captured process information to what you need, just go to Filter => Filter...
Set the left drop down to Process Name and then the second column to is and finally type in regsvr32.exe in the right box as illustrated above in the picture.
Crucially set the rightmost box to Include. Then press OK.
All unnecessary events should now be suppressed and only regsvr32.exe events are displayed (when you get to running it).
Run regsvr32.exe the normal way and look for "NAME NOT FOUND" entries (or similar) in the list.
In the illustration below MMUtilities.dll can not be found. In other words it is a missing dependency.

The other way to debug missing dependencies during self-registration is to use Dependency Walker as illustrated in my other answer in this "thread" or question or whatever to call it. Generally I find Dependency Walker to be the quicker option, but ProcMon is perhaps better overall - unless you are debugging an EXE file, then Dependency Walker has the superior Profile feature (Profile => Start Profiling...).

回答3:

The idea is not to use a bat file for registering the dll. Let Windows installer/WiX do it for you. This has been answered in the past also here.