C#: Writing extendable applications using on-the-fly compilation


Sometime back I had posted about writing applications that can load plugins using late binding. Users can drop assemblies in a specific folders which is scanned by an host application at startup. Using reflection the host can locate classes that implement a specific interface and instantiate those classes as plugins. The drawback is that the host processes plugin assemblies and you need to store/maintain the corresponding sources elsewhere.

This can be extended using on the fly compilation. In this technique the user drops source files in a specific folder and the host-application either at startup or on user request compiles the sources in-memory and loads the generated assembly. The sources can be in any of the .NET languages like C#, VB.NET or J#.  The whole of this can be trivially accomplished using the Microsoft.CSharp and the System.CodeDom.Compiler namespaces from System.dll.

We can call these sources as scripts as they are available as sources. However, in the truest sense they are not scripts because they are not interpreted but are compiled and executed. They are more of plugins where the step of compilation is done by the host application.

There are several security issues to be considered before using this technique and so you might want to read the drawbacks section at the end of this post before trying this out.

The Common Contract

The first and one of the most important things to decide is the Document Object Model or the DOM that the host-application exposes. The DOM is used by the script to manipulate the host-application. Similarly it's also important to decide the interface that the script exposes so that the host application can locate the script class in the script-source and instantiate the class. The DOM and the script interface together form the SW contract that both parties use to communicate with each other.

It's best to define the DOM and the script interface in a common extension dll and make both the host application and all scripts refer to it. The DOM is also defined in terms of an interface which the host application implements. For our purpose let's consider the simple DOM/script-interface combination.

namespace ScriptableApplication.Extensibility
{
public delegate void ClickEvent(int x, int y);

public interface IHost

{

string Title { get; set; }

Color BackgroundColor { get; set; }

event ClickEvent Click;

}

public interface IScript

{

string Name {get;}

string Description { get;}

void Initialize(IHost host);

void Close();

}

}

Here the host exposes a simple DOM using which the scripts can modify the host-applications title text and background color. It also exposes an event which is fired each time user clicks on the host-form. Scripts can subscribe to the event to get click notification along with the coordinates of the click.

Each script needs to have at least one class that implements the IScript interface. The host application compiles the script on the fly and using reflection looks for any class that implements this interface and instantiates that class. It then calls the Initialize method passing it the IHost pointer which the script can use to manipulate the host DOM.

On the Fly Compilation

This is what makes the whole technique work. .NET framework exposes all the compilers and associated framework in a very well designed namespace. This can be used to locate any compiler registered with a given file extension and use it to compile the sources.

private CompilerResults LoadScript(string filepath)
{
string language = CSharpCodeProvider.GetLanguageFromExtension(
Path.GetExtension(filepath));
CodeDomProvider codeDomProvider =
CSharpCodeProvider.CreateProvider(language);
CompilerParameters compilerParams = new CompilerParameters();
compilerParams.GenerateExecutable =
false;
compilerParams.GenerateInMemory =
true;
compilerParams.IncludeDebugInformation =
false;

string
extAssembly = Path.Combine(
Path.GetDirectoryName(Application.ExecutablePath),
"Extensibility.dll");
compilerParams.ReferencedAssemblies.Add(extAssembly);
compilerParams.ReferencedAssemblies.Add(
"System.dll");
compilerParams.ReferencedAssemblies.Add(
"System.Drawing.dll");
compilerParams.ReferencedAssemblies.Add(
"System.Windows.Forms.dll");

return codeDomProvider.CompileAssemblyFromFile(compilerParams,
filepath);
}

In the first part we use the .NET framework to locate the relevant compiler based on the extension of the script file. In case of unsupported language System.Configuration.ConfigurationErrorsException is thrown.

After we get the code-dom provider we create the compiler parameters. The GenerateInMemory flag is set to indicate the assembly is not generated on disk but in-memory.

Since the script needs to refer to the extensibility dll and some bare minimal .NET framework dlls, they are added to the ReferencedAssemblies collection. After that the compilation is done and the result of the compilation is returned as a CompilerResults object.

In case compilation fails the failures are available and are enumerated as follows

CompilerResults result = LoadScript(filePath);
if (result.Errors.HasErrors)
{
StringBuilder errors = new StringBuilder();
string filename = Path.GetFileName(filePath);
foreach (CompilerError err in result.Errors)
{
errors.Append(
string.Format("\r\n{0}({1},{2}): {3}: {4}",
filename, err.Line, err.Column,
err.ErrorNumber, err.ErrorText));
}
string str = "Error loading script\r\n" + errors.ToString();
throw new ApplicationException(str);
}

This gives detailed failure messages along with rom/column numbers of the failure.

Loading the plugins

If there are no compilation errors then the compiled in-memory assembly is available in result.CompiledAssembly. The host uses reflection to search through all the types in the assembly that implements IScript interface and loads that type and calls IScript methods.

GetPlugins(result.CompiledAssembly);

private
void GetPlugins(Assembly assembly)
{
foreach (Type type in assembly.GetTypes())
{
if (!type.IsClass || type.IsNotPublic) continue;
Type[] interfaces = type.GetInterfaces();
if (((IList<Type>)interfaces).Contains(typeof(IScript)))
{
IScript iScript = (IScript)Activator.CreateInstance(type);
iScript.Initialize(m_host);
// add the script details to a collection

ScriptDetails.Add(
string.Format("{0} ({1})\r\n",
iScript.Name, iScript.Description));
}
}
}

With this we are done initializing the plugins

Implementing the scripts

Implementing the scripts is simple and can be done in any .NET language. The following example is a script in C#.

using System;
using ScriptableApplication.Extensibility;
using System.Drawing;
public class Script : IScript
{
int i = 0;
IHost host =
null;
public string Name {get {return "CoolScript";}}
public string Description { get {return "Coolest script";}}
public void Initialize(IHost host)
{
this.host = host;
host.Click +=
delegate(int x, int y)
{
host.Title =
string.Format("Clicked on {0}, {1}",
x, y);
Random autoRand = new Random();

host.BackgroundColor =
Color.FromArgb(autoRand.Next(0, 255),
autoRand.Next(0, 255),
autoRand.Next(0, 255));
};

}

public void Close()
{
}
}

The script stores the IHost reference and subscribes to the IHost.Click event. In the event handler it sets the host dialogs title to the position of the click and changes the background-color to some random generated color.

Similarly scripts can be written in VB.NET as follows

Imports Microsoft.VisualBasic
Imports System
Imports
System.Drawing
Imports ScriptableApplication.Extensibility


Public
Class Class1
Implements IScript

Public
Sub Initialize(ByVal host As IHost)
Implements IScript.Initialize
host.Title =
"VB Script loaded"
End Sub
 
Public Sub Close() Implements IScript.Close
End Sub
 
Public ReadOnly Property Description() As String
Implements IScript.Description
Get
Description = "I'm a nice VB script"
End Get
End Property
 
Public ReadOnly Property Name() As String
Implements IScript.Name
Get
Name = "VB Script"
End Get
End Property
End
Class

Sample project

You can download the complete sample from here. It contains the following

Extensibility project: This contains only one source file Interfaces.cs which contains definition for the IScript and IHost interfaces. This project generates the Extensibility.dll.

ScriptableApplication project: This contains ScriptHandler.cs which has the complete implementation of on the fly compilation and loading of the dlls. MainForm.cs which implements the host application.

Scripts: Contains two sample scripts script.cs and script.vb

Drawbacks

There are two huge drawbacks to this technique, both of which related to security. If any user with lesser privilege has access to the plugins folder or can direct the host-application to pick up a script from any folder then the user can execute any code he wants using the credentials of the host application. In my case I used this to create motion detection algorithm plugins. This was a personal use test code and I ensured that only I and other admins of the box had write permission to the plugins folder, and so it is kind-of safe for use. However, if you want to use this in a program that's distributed outside you need to plug this security-flaw.

I had initially assumed that I can apply assembly level security restrictions on the generated assembly to disallow file and network access using something like [assembly: FileIOPermission(SecurityAction.RequestRefuse, Unrestricted = true)] and I'll be done. I am not an expert on security related issues and so I asked around on what is the best option in securing this scenario. Shawn Farkas suggested that the best option would be to use sand-boxed AppDomains. However since the plugin will be in an AppDomain different from the AppDomain in which the host application exists, there'll be performance cost due to cross appdomain calls that needs marshalling.

The other issue is error conditions in the script. If there is an unhandled exception in the script it'll bring the whole application down.

Comments (10)
  1. leppie says:

    It would probably to use Attributes for Name and Description. This way the IScript instance needs not to be available, as one relies on the the Type. 🙂

    The only problem is that you cant force the user to declare an attribute, but an exception should get his attention.

  2. I don’t think that attribute is the right way to go here. As you pointed out there is no way to enforce it. Moreover, the contract as in the interface should be able to complete tell the script writer what he is suppsed to use. Doing things differently for name/description does not add value and no one reads a comment which says he has to apply some attribute.

    The solution you suggested of throwing exception is just suicidal. I added a script without an attribute and the application I targetted crashed with a unhandled exception. Specially if the scripts are loaded on-demand as opposed to on-init then users unsaved work just gets lost for a faulty script….

  3. I know that the article ended asking about security, but this was a great demonstration of writing extendable applications using on-the-fly compilation!   Kudos!

    Keep up the great posts!

  4. Moo says:

    Why not just detect the folder change event and then process the assemblies then without user interaction or just at start up.

    I hope you threat model this attack vector, and every giblet is treated as rogue.

  5. Moo, as a demonstration I showed user interaction for loading plugins. You’d never want to do this in production code. Typically at startup plugins are loaded.

    Using FileSystem watcher you can get notification when an assembly is dropped into the plugins folder. Using that doesn’t make that much sense becuase plugins are installed rarely and its perfectly ok to ask users to restart application once plugins are installed.

  6. I’m currently working on a personal project that needs to spit out code after parsing some XML file….

  7. Seshagiri says:

    what happens if I want to change the script code?

    Using on-the-fly compilation I can compile my c# script code. This generates an assemmbly during run time which the host application uses to execute the script.

    Problem is once this new assembly (from script code) is loaded, I cannot recompile my script code as the assembly stays in memory.

    Do you know how to solve this?

  8. You need to load the assembly in a seperate AppDomain and tear it down next time. This way the assembly get unloaded.

    However, you pay marshaling perf penalties in case your assembly in that AppDomain needs to communitate across AppDomains

  9. Eric says:

    My friend Greg.  He’s learning from this code right now.   He told me so.

  10. readonly configuration. how to resolve this problem

    //snehal 23-4-2007  add connection string dynamically and culture wise.

                   ConnectionStringSettings css = null;

                   if (System.Globalization.CultureInfo.CurrentCulture.Name == "da-DK")

                   {

                        css = new ConnectionStringSettings("RecipiesGUI.Properties.Settings.ConnectionString", "data source=&quot;db\Danish\FoodAndDrink&quot;", "System.Data.SQLite");

                   }

                   else if(System.Globalization.CultureInfo.CurrentCulture.Name == "en-US")

                   {

                       css = new ConnectionStringSettings("RecipiesGUI.Properties.Settings.ConnectionString", "data source=&quot;db\English\FoodAndDrink&quot;", "System.Data.SQLite");

                   }

                   ConfigurationManager.ConnectionStrings.Add(css);

                   //end snehal

Comments are closed.

Skip to main content