Execution-Deferred IQueryable from Dynamic Linq

Question

I am using Dynamic Linq to perform some queries (sorry but it's my only option). As a result, I am getting an IQueryable instead of an IQueryable<T>. In my case, I want an IQueryable<Thing> where Thing is a concrete type.

My query is as such:

public IQueryable<Thing> Foo(MyContext db)
{
    var rootQuery = db.People.Where(x => x.City != null && x.State != null);
    var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
    var finalLogicalQuery = groupedQuery.Select("new ( Count() as TotalNumber, Key.City as City, Key.State as State )");
    var executionDeferredResults = finalLogicalQuery.Take(10); // IQueryable

    IQueryable<Thing> executionDeferredTypedThings = ??; // <--- Help here!!!!

    return executionDeferredTypedThings;
}

My Thing.cs:

public class Thing
{
    public int TotalNumber { get; set; }
    public string City { get; set; }
    public string State { get; set; }
}

Yes, I know the exact above thing can be done without Dynamic Linq but I have some variableness going on that I've simplified out of here. I can get it to work with my variableness if my return type is simply IQueryable but I can't figure out how to convert to IQueryable<Thing> while keeping it execution-deferred and while also keeping Entity Framework happy. I do have the dynamic Select always returning something (with the correct data) that looks like a Thing. But I simply can't figure how to return the IQueryable<Thing> and could use some help there. Thanks!!

Failed Attempt 1

Based on Rex M's suggestion, I am now trying to use AutoMapper to solve this problem (although I am not committed to this approach and am willing to try other approaches). For the AutoMapper approach, I am doing it as such:

IQueryable<Thing> executionDeferredTypedThings = executionDeferredResults.ProjectTo<Thing>(); // <--- Help here!!!!

But this results in an InvalidOperationException:

Missing map from DynamicClass2 to Thing. Create using Mapper.CreateMap.

The thing is, while I have defined Thing, I have not defined DynamicClass2 and as such, I cannot map it.

Failed Attempt 2

IQueryable<Thing> executionDeferredTypedThings = db.People.Provider.CreateQuery<Thing>(executionDeferredResults.Expression);

This gives an InvalidCastException and seems to be the same underlying problem that the above AutoMapper fail hits:

Unable to cast object of type 'System.Data.Entity.Infrastructure.DbQuery'1[DynamicClass2]' to type 'System.Linq.IQueryable'1[MyDtos.Thing]'.

Answer 1

You can use AutoMapper's Queryable Extensions to produce an IQueryable which wraps the underlying IQueryable, thus preserving the original IQueryable's IQueryProvider and the deferred execution, but adds in a mapping/translating component to the pipeline to convert from one type to another.

There's also AutoMapper's UseAsDataSource which makes some common query extension scenarios easier.

Answer 2

If I understand correctly, the following extension method should do the job for you

public static class DynamicQueryableEx
{
    public static IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values)
    {
        if (source == null) throw new ArgumentNullException("source");
        if (selector == null) throw new ArgumentNullException("selector");
        var dynamicLambda = System.Linq.Dynamic.DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
        var memberInit = dynamicLambda.Body as MemberInitExpression;
        if (memberInit == null) throw new NotSupportedException();
        var resultType = typeof(TResult);
        var bindings = memberInit.Bindings.Cast<MemberAssignment>()
            .Select(mb => Expression.Bind(
                (MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
                mb.Expression));
        var body = Expression.MemberInit(Expression.New(resultType), bindings);
        var lambda = Expression.Lambda(body, dynamicLambda.Parameters);
        return source.Provider.CreateQuery<TResult>(
            Expression.Call(
                typeof(Queryable), "Select",
                new Type[] { source.ElementType, lambda.Body.Type },
                source.Expression, Expression.Quote(lambda)));
    }
}

(Side note: Frankly I have no idea what values argument is for, but added it to match the corresponding DynamicQueryable.Select method signature.)

So your example will become something like this

public IQueryable<Thing> Foo(MyContext db)
{
    var rootQuery = db.People.Where(x => x.City != null && x.State != null);
    var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
    var finalLogicalQuery = groupedQuery.Select<Thing>("new ( Count() as TotalNumber, Key.City as City, Key.State as State )");  // IQueryable<Thing>
    var executionDeferredTypedThings = finalLogicalQuery.Take(10);
    return executionDeferredTypedThings;
}

How it works

The idea is quite simple.

The Select method implementation inside the DynamicQueryable looks something like this

public static IQueryable Select(this IQueryable source, string selector, params object[] values)
{
    if (source == null) throw new ArgumentNullException("source");
    if (selector == null) throw new ArgumentNullException("selector");
    LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
    return source.Provider.CreateQuery(
        Expression.Call(
            typeof(Queryable), "Select",
            new Type[] { source.ElementType, lambda.Body.Type },
            source.Expression, Expression.Quote(lambda)));
}

What it does is to dynamically create a selector expression and bind it to the source Select method. We take exactly the same approach, but after modifying the selector expression created by the DynamicExpression.ParseLambda call.

The only requirement is that the projection is using "new (...)" syntax and the names and types of the projected properties match, which I think fits in your use case.

The returned expression is something like this

(source) => new TargetClass
{
    TargetProperty1 = Expression1(source),
    TargetProperty2 = Expression2(source),
    ...
}

where TargetClass is a dynamically generated class.

All we want is to keep the source part and just replace that target class/properties with the desired class/properties.

As for the implementation, first the property assignments are converted with

var bindings = memberInit.Bindings.Cast<MemberAssignment>()
    .Select(mb => Expression.Bind(
        (MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
        mb.Expression));

and then the new DynamicClassXXX { ... } is replaced with with

var body = Expression.MemberInit(Expression.New(resultType), bindings);

Answer 3

There is no need for Dynamic Linq on this one.

var groupedQuery = from p in db.People
    where p.City != null && p.State != null
    group p by new {p.City, p.State}
    into gp
    select new Thing {
        TotalNumber = gp.Count(),
        City = gp.Key.City,
        State = gp.Key.State
    };

IQueryable<Thing> retQuery = groupedQuery.AsQueryable(); 
retQuery= retQuery.Take(10);
return retQuery;

Answer 4

Would something like this be of benefit to you?

public static IQueryable<TEntity> GetQuery<TEntity>(this DbContext db, bool includeReferences = false) where TEntity : class
    {
        try
        {
            if (db == null)
            {
                return null;
            }

            var key = typeof(TEntity).Name;
            var metaWorkspace = db.ToObjectContext().MetadataWorkspace;
            var workspaceItems = metaWorkspace.GetItems<EntityType>(DataSpace.OSpace);
            var workspaceItem = workspaceItems.First(f => f.FullName.Contains(key));
            var navProperties = workspaceItem.NavigationProperties;

            return !includeReferences
                    ? db.Set<TEntity>()
                    : navProperties.Aggregate((IQueryable<TEntity>)db.Set<TEntity>(), (current, navProperty) => current.Include(navProperty.Name));
        }
        catch (Exception ex)
        {
            throw new ArgumentException("Invalid Entity Type supplied for Lookup", ex);
        }
    }

You may want to take a look into the Generic Search project on Github located here: https://github.com/danielpalme/GenericSearch