replace nsICertificateDialogs from firefox extensi

I'm trying to replace the nsICertificateDialogs dialog with my own implementation. I already have an extension that handles smime mails from webmail. I want to be able to use the Mozilla cert store with my extension. The only possibility I've found to export smime certs/keys (to use it in my extension) is via exporting a pfx file.

I unregistered the original certifica tedialog and registered mine as described here Failure while calling nsIX509CertDB.nsIX509CertDB from command line (unregister the old factory and register mine with the correct contractID). When I call Cc[@mozilla.org/nsCertificateDialogs;1].getService(Ci.nsICertificateDialogs); my class gets instantiated. When calling certDB.importPKCS12File(null, certfile) I get an NS_ERROR_FAILURE and my dialog class is not instantiated.

What is the difference when FF instantiates the certificate dialog. How can I test it? What am I missing?

edit: my certificate dialog class does not get instanciated by ff - even when called with a valid token certDB.importPKCS12File(token, certfile); In change calling var certdialogInterface = Cc[sMimeCertificateDialog.mozillaContractID] .getService(Ci.nsICertificateDialogs); correctly instanciates my class (followed by some QueryInterface calls)

edit: I uploaded a (hopefully) simple reproducer to http://www.sodgeit.de/dialog_reproducer.zip Look at src/chrome/components/smime_certificate_dialog.js and at src/tests/smime_certificate_dialog_test.js

What is the difference when FF instantiates the certificate dialog.

No difference actually. When Firefox need the file password it will call getNSSDialogs() function and that one will do the moral equivalent of your getService() call. However, getService() only instantiates the component the first time, subsequent calls would not cause instantiation again - this might be the explanation why you don't see any coming from Firefox code.

The other potential issue is that you are not passing any token to the function. This means that Firefox will try to determine the token itself and call GetSlotWithMechanism(). In case of multiple available tokens it will display a selection dialog, it will basically call Cc["@mozilla.org/nsTokenDialogs;1"].getService(Ci.nsITokenDialogs).ChooseToken(). So maybe it errors out because this component isn't usable in your xpcshell - meaning that you would have to either replace it as well or specify a token explicitly.