How to compile C# DLL on the fly, Load, and Use

Question

A) compiling C# EXE's and DLL's on the fly are relatively easy.
B) Executing an EXE means that a new application is run. Loading a DLL means that methods and functions can be used in cases that may be shared between applications or projects.

Now, the quickest and easiest way to compile your EXE (or with mild modifications, DLL) can be found from the MSDN or for your convenience:

private bool CompileCSharpCode(string script)
{
lvErrors.Items.Clear();
    try
    {
        CSharpCodeProvider provider = new CSharpCodeProvider();
        // Build the parameters for source compilation.
        CompilerParameters cp = new CompilerParameters
        {
            GenerateInMemory = false,
            GenerateExecutable = false, // True = EXE, False = DLL
            IncludeDebugInformation = true,
            OutputAssembly = "eventHandler.dll", // Compilation name
        };

        // Add in our included libs.
        cp.ReferencedAssemblies.Add("System.dll");
        cp.ReferencedAssemblies.Add("System.Windows.Forms.dll");
        cp.ReferencedAssemblies.Add("Microsoft.VisualBasic.dll");

        // Invoke compilation. This works from a string, but you can also load from a file using FromFile()
        CompilerResults cr = provider.CompileAssemblyFromSource(cp, script);
        if (cr.Errors.Count > 0)
        {
            // Display compilation errors.
            foreach (CompilerError ce in cr.Errors)
            {
                //I have a listview to display errors.
                lvErrors.Items.Add(ce.ToString());
            }
            return false;
        }
        else
        {
            lvErrors.Items.Add("Compiled Successfully.");
        }
        provider.Dispose();
    }
    catch (Exception e)
    {
        // never really reached, but better safe than sorry?
        lvErrors.Items.Add("SEVERE! "+e.Message + e.StackTrace.ToString());
        return false;
    }
    return true;
}

Now that you can compile on the fly, there are a few variances between how to load the DLL. Typically speaking, you would add it as a reference in Visual Studios to be compiled into the project. This is rather easy and you have probably done it many times over, but we want to use it in our current project, and we can't very well require the user to recompile the entire project every time they want to test out their new DLL. Therefor, I will simply discuss how one loads a library 'on the fly'. Another term here would by "programmatically". To do this, after a successful compile, we load up an Assembly as follows:

Assembly assembly = Assembly.LoadFrom("yourfilenamehere.dll");

If you have an AppDomain, you can try this:

Assembly assembly = domain.Load(AssemblyName.GetAssemblyName("yourfilenamehere.dll"));

Now that the lib is "referenced", we can open it up and use it. There are 2 ways to do this. One requires you to know if the method has parameters, another will check for you. I'll do the later, you can check the MSDN for the other.

// replace with your namespace.class
Type type = assembly.GetType("company.project");
if (type != null)
{
    // replace with your function's name
    MethodInfo method = type.GetMethod("method");

    if (method != null)
    {
        object result = null;
        ParameterInfo[] parameters = method.GetParameters();
        object classInstance = Activator.CreateInstance(type, null);
        if (parameters.Length == 0) // takes no parameters
        {
                // method A:
            result = method.Invoke(classInstance, null);
                // method B:
            //result = type.InvokeMember("method", BindingFlags.InvokeMethod, null, classInstance, null);
        }
        else // takes 1+ parameters
        {
            object[] parametersArray = new object[] { }; // add parameters here

                // method A:
            result = method.Invoke(classInstance, parametersArray);
                // method B:
            //result = type.InvokeMember("method", BindingFlags.InvokeMethod, null, classInstance, parametersArray);
        }
    }
}

PROBLEM: First compile works fine. First execution works fine. However, the recompile attempt will error, saying that your *.PDP (debugger database) is in use. I've heard some hints about marshaling, and AppDomains, but I haven't quite cleared up the problem. Recompile will only fail after the DLL has been loaded.

---

Current attempt at Marshaling && AppDomain:

class ProxyDomain : MarshalByRefObject
    {
        private object _instance;
        public object Instance
        {
            get { return _instance; }
        }
        private AppDomain _domain;
        public AppDomain Domain
        {
            get
            {
                return _domain;
            }
        }
        public void CreateDomain(string friendlyName, System.Security.Policy.Evidence securityinfo)
        {
            _domain = AppDomain.CreateDomain(friendlyName, securityinfo);
        }
        public void UnloadDomain()
        {
            try
            {
                AppDomain.Unload(_domain);
            }
            catch (ArgumentNullException dne)
            {
                // ignore null exceptions
                return;
            }
        }
        private Assembly _assembly;
        public Assembly Assembly
        {
            get
            {
                return _assembly;
            }
        }
        private byte[] loadFile(string filename)
        {
            FileStream fs = new FileStream(filename, FileMode.Open);
            byte[] buffer = new byte[(int)fs.Length];
            fs.Read(buffer, 0, buffer.Length);
            fs.Close();

            return buffer;
        }
        public void LoadAssembly(string path, string typeName)
        {
            try
            {
                if (_domain == null)
                    throw new ArgumentNullException("_domain does not exist.");
                byte[] Assembly_data = loadFile(path);
                byte[] Symbol_data = loadFile(path.Replace(".dll", ".pdb"));

                _assembly = _domain.Load(Assembly_data, Symbol_data);
                //_assembly = _domain.Load(AssemblyName.GetAssemblyName(path));
                _type = _assembly.GetType(typeName);
            }
            catch (Exception ex)
            {
                throw new InvalidOperationException(ex.ToString());
            }
        }
        private Type _type;
        public Type Type
        {
            get
            {
                return _type;
            }
        }
        public void CreateInstanceAndUnwrap(string typeName)
        {
            _instance = _domain.CreateInstanceAndUnwrap(_assembly.FullName, typeName);
        }
    }

Errors on _instance = _domain.CreateInstanceAndUnwrap(_assembly.FullName, typeName); saying that my Assembly isn't serializable. Tried adding [Serializable] tag to my class with no luck. Still researching fixes.

Seems things can get a bit confusing when you can't see how they're being used, so here's making it easy?

private void pictureBox1_Click(object sender, EventArgs e)
    {
        pd.UnloadDomain();

        if (CompileCSharpCode(header + tScript.Text + footer))
        {
            try
            {
                pd.CreateDomain("DLLDomain", null);
                pd.LoadAssembly("eventHandler.dll", "Events.eventHandler");
                pd.CreateInstanceAndUnwrap("Events.eventHandler"); // Assembly not Serializable error!

                /*if (pd.type != null)
                {
                    MethodInfo onConnect = pd.type.GetMethod("onConnect");

                    if (onConnect != null)
                    {
                        object result = null;
                        ParameterInfo[] parameters = onConnect.GetParameters();
                        object classInstance = Activator.CreateInstance(pd.type, null);
                        if (parameters.Length == 0)
                        {
                            result = pd.type.InvokeMember("onConnect", BindingFlags.InvokeMethod, null, classInstance, null);
                            //result = onConnect.Invoke(classInstance, null);
                        }
                        else
                        {
                            object[] parametersArray = new object[] { };

                            //result = onConnect.Invoke(classInstance, parametersArray);
                            //result = type.InvokeMember("onConnect", BindingFlags.InvokeMethod, null, classInstance, parametersArray);
                        }
                    }
                }*/
                //assembly = Assembly.LoadFrom(null);

            }
            catch (Exception er)
            {
                MessageBox.Show("There was an error executing the script.\n>" + er.Message + "\n   - " + er.StackTrace.ToString());
            }
            finally
            {
            }
        }
    }

Answer 1

Once you have loaded a DLL into (the default appdomain of) a running process, the file on disk cannot be overwritten until the process is terminated. DLLs cannot be unloaded in managed code like they can be in unmanaged code.

You need to create a new appdomain in your host process and load the newly created DLL assembly into that appdomain. When you are ready to compile a new version of the DLL, you can dispose of the appdomain. This will unload the DLL from memory and release the lock on the DLL file, so that you can compile a new DLL to that same file. You can then construct a new appdomain to load the new DLL into.

The main hazard of using appdomains is that all calls across the appdomain boundary must be marshalled, much like an IPC or network RPC. Try to keep the interface of the objects you need to call across the appdomain boundary to a minimum.

You can also compile the assembly to memory, receiving a byte array or stream as the output, then load that assembly into the separate appdomain. This avoids creating debris on disk that will need to be deleted eventually.

Do not use compile to memory as a workaround for the file lock issue. The core issue is that assemblies cannot be removed from memory when they are loaded into the default appdomain of the process. You MUST create a new appdomain and load the DLL into that appdomain if you want to unload that assembly from memory later in the lifetime of the process.

Here's a rough outline of how to construct an object in the context of another appdomain:

    var appdomain = AppDomain.CreateDomain("scratch");
    byte[] assemblyBytes = // bytes of the compiled assembly
    var assembly = appdomain.Load(assemblyBytes);
    object obj = appdomain.CreateInstanceAndUnwrap(assembly.FullName, "mynamespace.myclass");

After this sequence, obj will contain a reference to a proxy that links to the actual object instance inside the appdomain. You can invoke methods on obj using reflection or typecast obj to a common interface type and call methods directly. Be prepared to make adjustments to support RPC marshalling of the method call parameters. (see .NET remoting)

When working with multiple appdomains, you have to be careful how you access types and assemblies because a lot of .NET functions default to operating in the current appdomain of the caller, which is usually not what you want when you have multiple appdomains. compilerResult.CompiledAssembly, for example, internally performs a Load of the generated assembly in the caller's appdomain. What you want is to load the assembly into your other appdomain. You have to do that explicitly.

Update: In your recently added code snippet showing how you load your appdomain, this line is your problem:

 _assembly = Assembly.LoadFrom(path);

That loads the DLL into the current appdomain (the caller's appdomain), not into the target appdomain (referenced by _domain in your example). You need to do use _domain.Load() to load the assembly into that appdomain.

Answer 2

if you have no need to debug, or don't mind to debug the "dynamic" code, with some missing information. you can generate the code in memory.. this will allow you to compile the code several times.. but will not generate a .pdb

cp.GenerateInMemory = true;

in alternative if you have no need to able to locate the assembly on disk you can ask the compiler to dump all the code in the temp directory and generate a temp name for the dll (wich will always be unique)

cp.TempFiles = new TempFileCollection(Path.GetTempPath(), false);
//cp.OutputAssembly = "eventHandler.dll";

in both this cases to access the dll and it's types you can get it from the compiler results

Assembly assembly = cr.CompiledAssembly; 

not explicitly loading is necessary

but if non of this situations apply and you must and a physical .dll with a .pdp in a known folder.. the only advice i can give you it to put a version number on the dll.. and in case you don't have a simple way to control the amount of times the dll was compiled you can always resort to a timestamp..

cp.OutputAssembly = "eventHandler"+DateTime.Now.ToString("yyyyMMddHHmmssfff")+".dll";

of course you must realize that every time you compile a new .dll will be loaded into memory and wont be unloaded unless you use separate app domains.. but that goes out of scope for this question..