IgorShare Thoughts and Ideas

Consulting and Training

CodeDom extensions and dynamic LINQ (string/script to LINQ emitting)

Posted by Igor Moochnick on 01/11/2008

Ah… Don’t you like these method extensions?!!!

Recently I’ve developed a certain taste for method extensions. They allow to create shorter code, but you should always keep in your head the dangers of such freedom – code can become complicated and unnatural. On the other hand, new set of extensions, it’s like to learn a new language, new classes, new techniques. So, currently, I have a mixed feeling – great thing, but not for free (some overhead attached).

In my previous post (Intellisense-like method selection pop-up window) I’ve shown how to create a LINQ editor (or any editor of your choice). You can ask – what’s the point of such script? What can we do with it?

.NET answer – the sky is your limit. The .NET framework allows you dynamically convert any string (correctly formatted) into a code and immediately execute it. You can do this in-memory, in-file, from-file, to-file, from-string, to-string, and any other combination of your choice.

Here are a couple of example of what you can do with a correctly formatted string:

  1. Generate a full code file
  2. Generate an assembly in-memory
  3. Create an assembly in file for future use
  4. etc …

As well you have a wide choice of code providers:

  1. C#
  2. VB.NET
  3. C++
  4. JavaScript
  5. and any other code provider that is installed on your machine and inherits from System.CodeDom.Compiler.CodeDomProvider class.

To simplify the use of CodeDom (it requires a wide knowledge of how the build process works), I’ve created a very basic library that extends CodeDom functionality and provides a nice and slick way of generating code-on-the-fly. Here are a couple of small examples, before we’ll dive into how it’s done (find them in the attached test project):

A) The following snippet prints Hello World! to the console:

   1: var c = new CodeDom();
   2: c.AddNamespace("Samples").Imports("System")
   3:     .AddClass(
   4:       c.Class("TestClass")
   5:         .AddMethod(c.Method("Print", "Console.WriteLine(\"Hello World\");")));
   7: var methodPrint = c.Compile().GetType("Samples.TestClass").GetMethod("Print");
   8: methodPrint.Invoke();

B) And here is the example of how to call a function with parametes (it’ll add 2 numbers):

   1: var c = new CodeDom();
   2: c.AddNamespace("Samples").Imports("System")
   3:     .AddClass(
   4:       c.Class("TestClass")
   5:         .AddMethod(c.Method("int", "AddNumbers", "int a, int b", "return a + b;")));
   7: var methodAddNumbers = c.Compile().GetType("Samples.TestClass").GetMethod("AddNumbers");
   8: int retVal = Converter.To<int>(methodAddNumbers.Invoke(2, 3), 0);
   9: // retVal = 5

C) and here is the BEST – dynamic LINQ (compiled and executed in memory):

   1: var c = new CodeDom();
   2: c.AddReference(@"System.Core.dll")
   3:     .AddNamespace("Samples")
   4:     .Imports("System.Collections.Generic")
   5:     .Imports("System.Linq")
   6:     .AddClass(
   7:       c.Class("TestClass")
   8:         .AddMethod(c.Method("IEnumerable<string>", "FilterCollection", "IEnumerable<string> items",
   9:              "return from item in items where item.StartsWith(\"a\") select item; ")));
  11: var method = c.Compile().GetType("Samples.TestClass").GetMethod("FilterCollection");
  13: var items = new List<string>() { "apple", "banana", "ananas", "mango" };
  14: IEnumerable<string> retVal = (IEnumerable<string>)method.Invoke(items);
  15: // retVal contains: "apple", "ananas"

Obviously you can start mix and match these extensions and create more namespaces, classes with more methods, etc … I don’t want to get too much into the details (see attached test project for more samples), but rather I want to tell how it works.

NOTE: Yes, I know about the Dynamic LINQ library and I’ve read ScottGu’s blog article, but, so far, I’ve seen only that you can create filters with this technique. It is using tokenized parser behind the scenes and reflects expression tree based on the results. This has somewhat limited functionality and can’t emit expression tree solely based on a string LINQ query. May be in the future it’ll be expanded, but, at this moment, my solution is the only way to convert LINQ string to an executable code.

Extension methods are the key. CodeDom class just provides a set of factories to create the CodeDom classes like CompileUnit, CodeTypeDeclaration, CodeSnippetTypeMember, etc… It collects all the namespaces, referenced assemblies, constructed code and, at the end, it allows you to decide what is a final product of your actions. The beauty of the code ensured by the extension methods.

I will not go too much into details of how to create extension methods themselves (a lot of information can be found on the net), but rather talk about the most important one: Method.Invoke.

   1: public static object Invoke(this MethodInfo methodInfo, params object[] parameters)
   2: {
   3:     return methodInfo.Invoke(null, parameters);
   4: }

This extension method makes the constructs like “method.Invoke()” and “method.Invoke(2, 3)” possible. It hides the complexity of the reflection from the users and allows him to use the newly constructed method as if it’s a real method of the application.

At the end I want to mention, yet another, beauty of .NET – dynamic type converter. Such converter allows you to  take control over your untyped objects without second-guessing your type conversion assumptions and skipping all the error check logic. I’ve used such technique very extensively with untyped DataSets in all my middle tier applications (I’ll be posting about it soon). For example, Method.Invoke always returns an object. If you want to use the returned value as a strongly typed object – you have to cast it to the required type either by “()” or “as” constructs. Then you need to check for a “null” or an exception. In my case you can do something like this:

   1: int retVal = Converter.To<int>(methodAddNumbers.Invoke(2, 3), 0);

Note the last “0” in the converter’s call – this is a default value that will be returned in case the conversion has failed. This is how it works:

   1: /// <summary>
   2: /// Converts from "object" general type to a concrete type
   3: /// </summary>
   4: /// <remarks>
   5: /// The class is working with .Net 2.0 and above
   6: /// </remarks>
   7: public class Converter
   8: {
   9:     /// <summary>
  10:     /// Returns True if the type can get Null as a value (is a reference type and not a value one)
  11:     /// </summary>
  12:     public static bool IsNullable(Type t)
  13:     {
  14:         if (!t.IsGenericType) return false;
  15:         Type g = t.GetGenericTypeDefinition();
  16:         return (g.Equals(typeof(Nullable<>)));
  17:     }
  19:     /// <summary>
  20:     /// Returns a real type of a first generic argument
  21:     /// </summary>
  22:     private static Type UnderlyingTypeOf(Type t)
  23:     {
  24:         return t.GetGenericArguments()[0];
  25:     }
  27:     /// <summary>
  28:     /// Converter
  29:     /// </summary>
  30:     public static T To<T>(object value, T defaultValue)
  31:     {
  32:         if (value == DBNull.Value) return defaultValue;
  33:         Type t = typeof(T);
  34:         if (IsNullable(t))
  35:         {
  36:             if (value == null) return default(T);
  37:             t = UnderlyingTypeOf(t);
  38:         }
  39:         else
  40:         {
  41:             if ((value == null) && (t.IsValueType))
  42:             {
  43:                 return defaultValue;
  44:             }
  45:         }
  47:         return (T)Convert.ChangeType(value, t);
  48:     }
  49: }

I’ll be talking about this converter in more detail in the future posts, so stay tuned …

The full source code with tests/examples you can find as usual on my web site: http://igor.moochnick.googlepages.com.

DISCLAMER: The provided libraries are a work in progress and do not contain 100% wrapped CodeDom. There are a lot of things that can be added to make the whole user experience easier. You are welcome to use them AS-IS and to extend them to any degree you with. You’re welcome to send me your updates and I’ll be more than happy to post them for everybody else to use.


9 Responses to “CodeDom extensions and dynamic LINQ (string/script to LINQ emitting)”

  1. […] CodeDom extensions and dynamic LINQ (string/script to LINQ emitting) […]

  2. […] I think we need either another .NET compiler revision to support dynamic LINQ (where it’ll allow us to emit LINQ queries from strings) or a some kind of LINQ Dom in-memory compiler library. I’m, actually, leaning towards the lateral (see my post CodeDom extensions and dynamic LINQ (string/script to LINQ emitting)). […]

  3. Perfect article and great idea indeed! This already game me so much ideas, is it free to be used/modified?

  4. igormoochnick said

    Sure, there is nothing proprietary in all this. I always encourage people: Think outside of the box and don’t be afraid of exploring new things.

  5. I’ve reused your approach within a small “Immediate Window” tool. In case you’ll be intrested to see your ideas perfectly working you are welcome to my blog post:
    Thanks for sharing your ideas.

  6. Ryumkin said

    And what about the perfomance? Did you test it?

  7. Hi, Igor!
    I developed the idea. Take a look at Expressions to CodeDOM

  8. […] MSDN的namespace System.Linq.Dynamic是通过分析“where string”和“select string”,并借助表达式树来完成c的,虽然安全得多,但未免限制重重。我决定从其他方向入手。很快就发现了这篇CodeDom extensions and dynamic LINQ (string/script to LINQ emitting)。CodeDom,确实是一个很诱人的思路。 […]

  9. everything about java…

    […]CodeDom extensions and dynamic LINQ (string/script to LINQ emitting) « IgorShare Weblog[…]…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: