Share expressions between Linq to Entities and Lin

Question

I'm trying to "share" a set of conditions between a Linq to Entities call and a some other code, to reduce possible mismatches in conditions between the two calls.

I started off by declaring my conditions:

private Func<DateTime, Status, bool> _submissionDateExpiredCondition =
(submissionDate, status) => submissionDate < DateTime.Now && status == Status.OK;

private Func<DateTime, Status, bool> _submissionDateWithinOneWeekCondition =
(submissionDate, status) => DateTime.Now < DbFunctions.AddDays(submissionDate, -7) && status == Status.Pending;

private Func<DateTime?, Status, bool> _bidValidityEndPeriodWithinThirtyDaysCondition =
(bidValidityEndPeriod, status) =>  bidValidityEndPeriod.HasValue && DateTime.Now < DbFunctions.AddDays(bidValidityEndPeriod.Value, -30) && (status == Status.OK);

I then want to use these conditions inside my where clause in both a Linq to Entities where call and as functions in an if statement (or possibly the where call of a Linq to Objects query):

myRepository
    .FindAll()
    .Where(x => x.Property == "value" 
    && x.Data.AnotherProperty == true 
    && _submissionDateExpiredCondition(x.Data.Timestamp, x.Data.Status) 
    || _submissionDateWithinOneWeekCondition(x.Data.Timestamp, x.Data.Status) 
    || _bidValidityEndPeriodWithinThirtyDaysCondition(x.Data.ValidityStamp, x.Data.Status))

and (please note that MyCustomObject is not the same type as returned by myRepository.FindAll())

private void ApplyConditions(List<MyCustomObject> items) {
    foreach(var x in items){
        if(_submissionDateExpiredCondition(x.Data.Timestamp, x.Data.Status)){
            x.Property = "condition 1";
        }
        else if(_submissionDateWithinOneWeekCondition(x.Data.Timestamp, x.Data.Status))
        {
            x.Property = "condition 2";
        }
        else if(_bidValidityEndPeriodWithinThirtyDaysCondition(x.Data.ValidityStamp, x.Data.Status))
        {
            x.Property = "condition 3";
        }
    }
}

But I keep bumping into the regular issues like
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.
when executing the repository query...

I've tried building a predicate with a predicate builder (as per https://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/) but no luck.

Can anyone point me in the right direction?

Answer 1

Late to the party, but someone may find my way of attacking the problem useful. However it can't easily be done without some expression manipulation.

Main problem is: inside the .Where's predicate expression you have InvocationExpressions of delegates (i.e. compiled code). EF has no way to find out what logic is baked into that delegates and thus won't be able to translate it into SQL. That's where the exception originates.

The goal is to get a .Where predicate lambda expression that is logically equivalent to yours, yet understandable by EF. That means we have to get from

Expression<Func<EntityType, bool>> xPredicate = x =>
    x.Property == "value" &&
    x.Data.AnotherProperty == true && 
    _submissionDateExpiredCondition(x.Data.Timestamp, x.Data.Status)
    || ...;

to

Expression<Func<EntityType, bool>> xPredicate = x =>
    x.Property == "value" &&
    x.Data.AnotherProperty == true && 
    x.Data.Timestamp < DateTime.Now && x.Data.Status == Status.OK
    || ...;

to be used in

myRepository.FindAll().Where(xPredicate)

, where EntityTypeis the element type of the queryable returned by Find - the one that is different from MyCustomObject.

Note that the invocation of the delegate is being replaced by it's defining expression (lambda body), with the (lambda) parameters submissionDate and status replaced by the respective argument expressions of the invocation.

If you define the conditions as delegates, their internal logic is lost in compiled code, so we have to start off with lambda expressions rather than delegates:

private Expression<Func<DateTime, Status, bool>> _xSubmissionDateExpiredCondition = (submissionDate, status) => submissionDate < DateTime.Now && status == Status.OK;
// getting the delegate as before (to be used in ApplyConditions) is trivial:
private Func<DateTime, Status, bool> _submissionDateExpiredCondition = _xSubmissionDateExpiredCondition.Compile();

// ... other conditions here

Using the lambda expression rather than the delegate, the compiler lets you rewrite the original predicate like this:

Expression<Func<EntityType, bool>> xPredicate = x =>
    x.Property == "value" &&
    x.Data.AnotherProperty == true && 
    _xSubmissionDateExpiredCondition.Compile()(x.Data.Timestamp, x.Data.Status)
    || ...;

, which of course EF won't understand any better than before. What we however achieved is that the condition's internal logic is part of the expression tree. So all that's missing is some magic:

xPredicate = MAGIC(xPredicate);

What MAGIC does: Find an InvocationExpression of a delegate that is the result of a Compile() method call on a lambda expression and replace it with the lambda's body, but make sure to replace the lambda parameters in the body with the argument expressions of the invocation.

And here my implementation. Actually, MAGIC is called Express.Prepare here, which is slightly less unspecific.

/// <summary>
/// Helps in building expressions.
/// </summary>
public static class Express
{

    #region Prepare

    /// <summary>
    /// Prepares an expression to be used in queryables.
    /// </summary>
    /// <returns>The modified expression.</returns>
    /// <remarks>
    /// The method replaces occurrences of <see cref="LambdaExpression"/>.Compile().Invoke(...) with the body of the lambda, with it's parameters replaced by the arguments of the invocation.
    /// Values are resolved by evaluating properties and fields only.
    /// </remarks>
    public static Expression<TDelegate> Prepare<TDelegate>(this Expression<TDelegate> lambda) => (Expression<TDelegate>)new PrepareVisitor().Visit(lambda);

    /// <summary>
    /// Wrapper for <see cref="Prepare{TDelegate}(Expression{TDelegate})"/>.
    /// </summary>
    public static Expression<Func<T1, TResult>> Prepare<T1, TResult>(Expression<Func<T1, TResult>> lambda) => lambda.Prepare();

    /// <summary>
    /// Wrapper for <see cref="Prepare{TDelegate}(Expression{TDelegate})"/>.
    /// </summary>
    public static Expression<Func<T1, T2, TResult>> Prepare<T1, T2, TResult>(Expression<Func<T1, T2, TResult>> lambda) => lambda.Prepare();

    // NOTE: more overloads of Prepare here.

    #endregion

    /// <summary>
    /// Evaluate an expression to a simple value.
    /// </summary>
    private static object GetValue(Expression x)
    {
        switch (x.NodeType)
        {
            case ExpressionType.Constant:
                return ((ConstantExpression)x).Value;
            case ExpressionType.MemberAccess:
                var xMember = (MemberExpression)x;
                var instance = xMember.Expression == null ? null : GetValue(xMember.Expression);
                switch (xMember.Member.MemberType)
                {
                    case MemberTypes.Field:
                        return ((FieldInfo)xMember.Member).GetValue(instance);
                    case MemberTypes.Property:
                        return ((PropertyInfo)xMember.Member).GetValue(instance);
                    default:
                        throw new Exception(xMember.Member.MemberType + "???");
                }
            default:
                // NOTE: it would be easy to compile and invoke the expression, but it's intentionally not done. Callers can always pre-evaluate and pass a captured member.
                throw new NotSupportedException("Only constant, field or property supported.");
        }
    }

    /// <summary>
    /// <see cref="ExpressionVisitor"/> for <see cref="Prepare{TDelegate}(Expression{TDelegate})"/>.
    /// </summary>
    private sealed class PrepareVisitor : ExpressionVisitor
    {
        /// <summary>
        /// Replace lambda.Compile().Invoke(...) with lambda's body, where the parameters are replaced with the invocation's arguments.
        /// </summary>
        protected override Expression VisitInvocation(InvocationExpression node)
        {
            // is it what we are looking for?
            var call = node.Expression as MethodCallExpression;
            if (call == null || call.Method.Name != "Compile" || call.Arguments.Count != 0 || call.Object == null || !typeof(LambdaExpression).IsAssignableFrom(call.Object.Type))
                return base.VisitInvocation(node);

            // get the lambda
            var lambda = call.Object as LambdaExpression ?? (LambdaExpression)GetValue(call.Object);

            // get the expressions for the lambda's parameters
            var replacements = lambda.Parameters.Zip(node.Arguments, (p, x) => new KeyValuePair<ParameterExpression, Expression>(p, x));

            // return the body with the parameters replaced
            return Visit(new ParameterReplaceVisitor(replacements).Visit(lambda.Body));
        }
    }

    /// <summary>
    /// <see cref="ExpressionVisitor"/> to replace parameters with actual expressions.
    /// </summary>
    private sealed class ParameterReplaceVisitor : ExpressionVisitor
    {
        private readonly Dictionary<ParameterExpression, Expression> _replacements;

        /// <summary>
        /// Init.
        /// </summary>
        /// <param name="replacements">Parameters and their respective replacements.</param>
        public ParameterReplaceVisitor(IEnumerable<KeyValuePair<ParameterExpression, Expression>> replacements)
        {
            _replacements = replacements.ToDictionary(kv => kv.Key, kv => kv.Value);
        }

        protected override Expression VisitParameter(ParameterExpression node)
        {
            Expression replacement;
            return _replacements.TryGetValue(node, out replacement) ? replacement : node;
        }
    }
}