Multi-Mapper to create object hierarchy

Question

I've been playing around with this for a bit, because it seems like it feels a lot like the documented posts/users example, but its slightly different and isn't working for me.

Assuming the following simplified setup (a contact has multiple phone numbers):

public class Contact
{
    public int ContactID { get; set; }
    public string ContactName { get; set; }
    public IEnumerable<Phone> Phones { get; set; }
}

public class Phone
{
    public int PhoneId { get; set; }
    public int ContactID { get; set; } // foreign key
    public string Number { get; set; }
    public string Type { get; set; }
    public bool IsActive { get; set; }
}

I'd love to end up with something that returns a Contact with multiple Phone objects. That way, if I had 2 contacts, with 2 phones each, my SQL would return a join of those as a result set with 4 total rows. Then Dapper would pop out 2 contact objects with two phones each.

Here is the SQL in the stored procedure:

SELECT *
FROM Contacts
    LEFT OUTER JOIN Phones ON Phones.ReferenceId=Contacts.ReferenceId
WHERE clientid=1

I tried this, but ended up with 4 Tuples (which is OK, but not what I was hoping for... it just means I still have to re-normalize the result):

var x = cn.Query<Contact, Phone, Tuple<Contact, Phone>>("sproc_Contacts_SelectByClient",
                              (co, ph) => Tuple.Create(co, ph), 
                                          splitOn: "PhoneId", param: p, 
                                          commandType: CommandType.StoredProcedure);

and when I try another method (below), I get an exception of "Unable to cast object of type 'System.Int32' to type 'System.Collections.Generic.IEnumerable`1[Phone]'."

var x = cn.Query<Contact, IEnumerable<Phone>, Contact>("sproc_Contacts_SelectByClient",
                               (co, ph) => { co.Phones = ph; return co; }, 
                                             splitOn: "PhoneId", param: p,
                                             commandType: CommandType.StoredProcedure);

Am I just doing something wrong? It seems just like the posts/owner example, except that I'm going from the parent to the child instead of the child to the parent.

Thanks in advance

Answer 1

Multi result set support

In your case it would be much better (and easier as well) to have a multi resultset query. This simply means that you should write two select statements:

1. One that returns contacts
2. And one that returns their phone numbers

This way your objects would be unique and wouldn't duplicate.

Answer 2

Check out https://www.tritac.com/blog/dappernet-by-example/ You could do something like this:

public class Shop {
  public int? Id {get;set;}
  public string Name {get;set;}
  public string Url {get;set;}
  public IList<Account> Accounts {get;set;}
}

public class Account {
  public int? Id {get;set;}
  public string Name {get;set;}
  public string Address {get;set;}
  public string Country {get;set;}
  public int ShopId {get;set;}
}

var lookup = new Dictionary<int, Shop>()
conn.Query<Shop, Account, Shop>(@"
                  SELECT s.*, a.*
                  FROM Shop s
                  INNER JOIN Account a ON s.ShopId = a.ShopId                    
                  ", (s, a) => {
                       Shop shop;
                       if (!lookup.TryGetValue(s.Id, out shop)) {
                           lookup.Add(s.Id, shop = s);
                       }
                       shop.Accounts.Add(a);
                       return shop;
                   },
                   ).AsQueryable();
var resultList = lookup.Values;

I got this from the dapper.net tests: https://code.google.com/p/dapper-dot-net/source/browse/Tests/Tests.cs#1343

Answer 3

You are doing nothing wrong, it is just not the way the API was designed. All the Query APIs will always return an object per database row.

So, this works well on the many -> one direction, but less well for the one -> many multi-map.

There are 2 issues here:

1. If we introduce a built-in mapper that works with your query, we would be expected to "discard" duplicate data. (Contacts.* is duplicated in your query)

2. If we design it to work with a one -> many pair, we will need some sort of identity map. Which adds complexity.

---

Take for example this query which is efficient if you just need to pull a limited number of records, if you push this up to a million stuff get trickier, cause you need to stream and can not load everything into memory:

var sql = "set nocount on
DECLARE @t TABLE(ContactID int,  ContactName nvarchar(100))
INSERT @t
SELECT *
FROM Contacts
WHERE clientid=1
set nocount off 
SELECT * FROM @t 
SELECT * FROM Phone where ContactId in (select t.ContactId from @t t)"

What you could do is extend the GridReader to allow for the remapping:

var mapped = cnn.QueryMultiple(sql)
   .Map<Contact,Phone, int>
    (
       contact => contact.ContactID, 
       phone => phone.ContactID,
       (contact, phones) => { contact.Phones = phones };  
    );

Assuming you extend your GridReader and with a mapper:

public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
    (
    this GridReader reader,
    Func<TFirst, TKey> firstKey, 
    Func<TSecond, TKey> secondKey, 
    Action<TFirst, IEnumerable<TSecond>> addChildren
    )
{
    var first = reader.Read<TFirst>().ToList();
    var childMap = reader
        .Read<TSecond>()
        .GroupBy(s => secondKey(s))
        .ToDictionary(g => g.Key, g => g.AsEnumerable());

    foreach (var item in first)
    {
        IEnumerable<TSecond> children;
        if(childMap.TryGetValue(firstKey(item), out children))
        {
            addChildren(item,children);
        }
    }

    return first;
}

Since this is a bit tricky and complex, with caveats. I am not leaning towards including this in core.

Answer 4

FYI - I got Sam's answer working by doing the following:

First, I added a class file called "Extensions.cs". I had to change the "this" keyword to "reader" in two places:

using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;

namespace TestMySQL.Helpers
{
    public static class Extensions
    {
        public static IEnumerable<TFirst> Map<TFirst, TSecond, TKey>
            (
            this Dapper.SqlMapper.GridReader reader,
            Func<TFirst, TKey> firstKey,
            Func<TSecond, TKey> secondKey,
            Action<TFirst, IEnumerable<TSecond>> addChildren
            )
        {
            var first = reader.Read<TFirst>().ToList();
            var childMap = reader
                .Read<TSecond>()
                .GroupBy(s => secondKey(s))
                .ToDictionary(g => g.Key, g => g.AsEnumerable());

            foreach (var item in first)
            {
                IEnumerable<TSecond> children;
                if (childMap.TryGetValue(firstKey(item), out children))
                {
                    addChildren(item, children);
                }
            }

            return first;
        }
    }
}

Second, I added the following method, modifying the last parameter:

public IEnumerable<Contact> GetContactsAndPhoneNumbers()
{
    var sql = @"
SELECT * FROM Contacts WHERE clientid=1
SELECT * FROM Phone where ContactId in (select ContactId FROM Contacts WHERE clientid=1)";

    using (var connection = GetOpenConnection())
    {
        var mapped = connection.QueryMultiple(sql)    
            .Map<Contact,Phone, int>     (        
            contact => contact.ContactID,        
            phone => phone.ContactID,
            (contact, phones) => { contact.Phones = phones; }      
        ); 
        return mapped;
    }
}

Answer 5

I wanted to share my solution to this issue and see if anyone has any constructive feedback on the approach I've used?

I have a few requirements in the project I'm working on which I need to explain first up:

1. I have to keep my POCO's as clean as possible as these classes will be publicly shared in an API wrapper.
2. My POCO's are in a seperate Class Library because of the above requirement
3. There are going to be multiple object hierarchy levels that will vary depending on data (so I cannot use a Generic Type Mapper or I'd have to write tons of them to cater for all possible eventualities)

So, what I have done is to get SQL to handle the 2nd - nth Level heirarchy by returning a Single JSON string as a column on the original row as follows (stripped out the other columns / properties etc to illustrate):

Id  AttributeJson
4   [{Id:1,Name:"ATT-NAME",Value:"ATT-VALUE-1"}]

Then, my POCO's are built up like the below :

public abstract class BaseEntity
{
    [KeyAttribute]
    public int Id { get; set; }
}

public class Client : BaseEntity
{
    public List<ClientAttribute> Attributes{ get; set; }
}
public class ClientAttribute : BaseEntity
{
    public string Name { get; set; }
    public string Value { get; set; }
}

Where the POCO's inherit from BaseEntity. (To illustrate I've chosen a fairly simple, single level heirarchy as shown by the "Attributes" property of the client object. )

I then have in my Data Layer the following "Data Class" which inherits from the POCO Client.

internal class dataClient : Client
{
    public string AttributeJson
    {
        set
        {
            Attributes = value.FromJson<List<ClientAttribute>>();
        }
    }
}

As you can see above, whats happening is that SQL is returning a column called "AttributeJson" which is mapped to the property AttributeJson in the dataClient class. This has only a setter which deserialises the JSON to the Attributes property on the inherited Client class. The dataClient Class is internal to the Data Access Layer and the ClientProvider (my data factory) returns the original Client POCO to the calling App / Library like so :

var clients = _conn.Get<dataClient>();
return clients.OfType<Client>().ToList();

Note that I'm using Dapper.Contrib and have added a new Get<T> Method that returns an IEnumerable<T>

There are a couple things to note with this solution:

1. There's an obvious performance trade off with the JSON serialisation - I've benchmarked this against 1050 rows with 2 sub List<T> properties, each with 2 entities in the list and it clocks in at 279ms - which is acceptable for my projects needs - this is also with ZERO optimisation on the SQL side of things so I should be able to shave a few ms there.

2. It does mean additional SQL queries are required to build up the JSON for each required List<T> property, but again, this suits me as I know SQL pretty well and am not so fluent on dynamics / reflection etc.. so this way I feel like I have more control over things as I actually understand whats happening under the hood :-)

There may well be a better solution than this one and if there is I would really appreciate hearing your thoughts - this is just the solution I came up with that so far fits my needs for this project (although this is experimental at the stage of posting).

Answer 6

Based on Sam Saffron's (and Mike Gleason's) approach, here is a solution which will allow for multiple children and multiple levels.

using System;
using System.Collections.Generic;
using System.Linq;
using Dapper;

namespace TestMySQL.Helpers
{
    public static class Extensions
    {
        public static IEnumerable<TFirst> MapChild<TFirst, TSecond, TKey>
            (
            this SqlMapper.GridReader reader,
            List<TFirst> parent,
            List<TSecond> child,
            Func<TFirst, TKey> firstKey,
            Func<TSecond, TKey> secondKey,
            Action<TFirst, IEnumerable<TSecond>> addChildren
            )
        {
            var childMap = child
                .GroupBy(secondKey)
                .ToDictionary(g => g.Key, g => g.AsEnumerable());
            foreach (var item in parent)
            {
                IEnumerable<TSecond> children;
                if (childMap.TryGetValue(firstKey(item), out children))
                {
                    addChildren(item, children);
                }
            }
            return parent;
        }
    }
}

Then you can have it read outside of the function.

using (var multi = conn.QueryMultiple(sql))
{
    var contactList = multi.Read<Contact>().ToList();
    var phoneList = multi.Read<Phone>().ToList;
    contactList = multi.MapChild
        (
            contactList,
            phoneList,
            contact => contact.Id, 
            phone => phone.ContactId,
            (contact, phone) => {contact.Phone = phone;}
        ).ToList();
    return contactList;
}

The map function can then be called again for the next child object using the same parent object. You can also implement splits on the parent or child read statements independently of the map function.

Here is a 'single to N' additional extension method

    public static TFirst MapChildren<TFirst, TSecond, TKey>
        (
        this SqlMapper.GridReader reader,
        TFirst parent,
        IEnumerable<TSecond> children,
        Func<TFirst, TKey> firstKey,
        Func<TSecond, TKey> secondKey,
        Action<TFirst, IEnumerable<TSecond>> addChildren
        )
    {
        if (parent == null || children == null || !children.Any())
        {
            return parent;
        }

        Dictionary<TKey, IEnumerable<TSecond>> childMap = children
            .GroupBy(secondKey)
            .ToDictionary(g => g.Key, g => g.AsEnumerable());

        if (childMap.TryGetValue(firstKey(parent), out IEnumerable<TSecond> foundChildren))
        {
            addChildren(parent, foundChildren);
        }

        return parent;
    }