Entity Framework entity is not in DataSpace.OSpace

I have been mucking around with XMLs for entity Framework. I tried to create a type of entity that could have properties injected at runtime, First I created DynamicEntity object that is dynamic

public class DynamicEntity : DynamicObject
{
    Dictionary<string, object> dynamicMembers = new Dictionary<string, object>();

    public override bool TrySetMember(SetMemberBinder binder, object value)
    {
        dynamicMembers[binder.Name] = value;
        return true;
    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        if (dynamicMembers.TryGetValue(binder.Name, out result))
        {
            return dynamicMembers.TryGetValue(binder.Name, out result);
        }

        result = "";
        return true;
    }
}

then entity inherits from this

public partial class QUOTE_HOUSE : DynamicEntity

(and it does seem to work when I set properties manually after I get data from db).

so based on this mechanism of removing properties I tried to do another one that inserts properties into XMLs, and whole thing seems to hold up ok (at least it does not blow up on mapping which it usually does when XMLs are not right var mappingCollection = new StorageMappingItemCollection(conceptualCollection, storageCollection, new[] {mappingXml.CreateReader()});).

Problem is EF when executing query blows up with

The entity type QUOTE_HOUSE is not part of the model for the current context.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: The entity type QUOTE_HOUSE is not part of the model for the current context.

[InvalidOperationException: The entity type QUOTE_HOUSE is not part of the model for the current context.]
System.Data.Entity.Internal.InternalContext.UpdateEntitySetMappingsForType(Type entityType) +208
System.Data.Entity.Internal.InternalContext.GetEntitySetAndBaseTypeForType(Type entityType) +50

Which I traced to TryUpdateEntitySetMappingsForType in System.Data.Entity.Internal.InternalContext after loading pdb for EF

Basically what happens my QUOTE_HOUSE is not in this._workspace.GetItemCollection(DataSpace.OSpace) where UpdateEntitySetMappings tries to map it from.

It checks if it's in this._entitySetMappingsCache.ContainsKey(entityType)) and since it's not it then tries update mappings iterating over this._workspace.GetItemCollection(DataSpace.OSpace) where my item doesn't exist

However I can see that my entity does exist in this._workspace.GetItems<EntityContainer>(DataSpace.CSpace).

Full UpdateEntitySetMappings looks following:

private void UpdateEntitySetMappings()
{
  ObjectItemCollection objectItemCollection = (ObjectItemCollection) this._workspace.GetItemCollection(DataSpace.OSpace);
  ReadOnlyCollection<EntityType> items = this._workspace.GetItems<EntityType>(DataSpace.OSpace);
  Stack<EntityType> entityTypeStack = new Stack<EntityType>();
  foreach (EntityType entityType1 in items)
  {
    entityTypeStack.Clear();
    EntityType cspaceType = (EntityType) this._workspace.GetEdmSpaceType((StructuralType) entityType1);
    do
    {
      entityTypeStack.Push(cspaceType);
      cspaceType = (EntityType) cspaceType.BaseType;
    }
    while (cspaceType != null);
    EntitySet entitySet = (EntitySet) null;
    while (entitySet == null && entityTypeStack.Count > 0)
    {
      cspaceType = entityTypeStack.Pop();
      foreach (EntityContainer entityContainer in this._workspace.GetItems<EntityContainer>(DataSpace.CSpace))
      {
        List<EntitySetBase> list = entityContainer.BaseEntitySets.Where<EntitySetBase>((Func<EntitySetBase, bool>) (s => s.ElementType == cspaceType)).ToList<EntitySetBase>();
        int count = list.Count;
        if (count > 1 || count == 1 && entitySet != null)
          throw Error.DbContext_MESTNotSupported();
        if (count == 1)
          entitySet = (EntitySet) list[0];
      }
    }
    if (entitySet != null)
    {
      EntityType entityType2 = (EntityType) this._workspace.GetObjectSpaceType((StructuralType) cspaceType);
      Type clrType1 = objectItemCollection.GetClrType((StructuralType) entityType1);
      Type clrType2 = objectItemCollection.GetClrType((StructuralType) entityType2);
      this._entitySetMappingsCache[clrType1] = new EntitySetTypePair(entitySet, clrType2);
    }
  }
}

How do entities get into this._workspace.GetItemCollection(DataSpace.OSpace)? Why would entity be in CSpace but not in OSpace ?

EDIT: For those who might wanna have a crack at bounty, below are components you might need to set-up environment to reproduce the issue.

public class SystemToDatabaseMapping
{
    public SystemToDatabaseMapping(string system, string databaseType, string database, string connectionString, Type enitityType)
    {
        System = system;
        Database = database;
        DatabaseType = databaseType;
        ConnectionString = connectionString;
        EntityType = enitityType;
    }

    public Type EntityType { get; set; }
    public string System { get; set; }
    public string Database { get; set; }
    public string DatabaseType { get; set; }
    public string ConnectionString { get; set; }
    public List<ColumnToModify> ColumnsToModify  { get; set; }
}

public abstract class ColumnToModify
{
    protected ColumnToModify(string table, string column)
    {
        Table = table;
        Column = column;
    }

    public string Table { get; set; }
    public string Column { get; set; }

    public abstract bool IsRemove{ get; }
}

public class ColumnToRemove : ColumnToModify
{
    public ColumnToRemove(string table, string column) : base(table, column)
    {
    }

    public override bool IsRemove
    {
        get { return true; }
    }
}

public class ColumnToAdd : ColumnToModify
{
    public ColumnToAdd(string table, string column, Type type) : base(table, column)
    {
        this.Type = type;
    }

    public override bool IsRemove
    {
        get { return false; }
    }

    public Type Type { get; set; }
}

Entity generated from db first, (DynamicEntity code is above)

public partial class QUOTE_HOUSE : DynamicEntity
{
    public long UNIQUE_ID { get; set; }
}

DbContext for database requires constructor overloads

 public partial class EcomEntities : DbContext
 {

    public EcomEntities(DbConnection connectionString)
        : base(connectionString, false)
    {
    }

    public virtual DbSet<QUOTE_HOUSE > QUOTE_HOUSE { get; set; }
....
}

Mechanism that does column injection (it's a rough prototype so be forgiving to how bad it looks atm), when injecting try string column I know that it maps ok.

public static class EntityConnectionExtensions
{
    public static IEnumerable<XElement> ElementsAnyNS<T>(this IEnumerable<T> source, string localName)
        where T : XContainer
    {
        return source.Elements().Where(e => e.Name.LocalName == localName);
    }

    public static IEnumerable<XElement> ElementsAnyNS(this XContainer source, string localName)
    {
        return source.Elements().Where(e => e.Name.LocalName == localName);
    }

    private static void ModifyNodes(XElement element, List<ColumnToModify> tableAndColumn)
    {
        if (element.Attribute("Name") != null && tableAndColumn.Any(oo => oo.Table == element.Attribute("Name").Value) ||
            element.Attribute("StoreEntitySet") != null && tableAndColumn.Any(oo => oo.Table == element.Attribute("StoreEntitySet").Value))
        {
            var matchingRemoveSelectParts = tableAndColumn.Where(oo => oo.IsRemove && element.Value.Contains(string.Format("\"{0}\".\"{1}\" AS \"{1}\"", oo.Table, oo.Column))).ToList();

            if (matchingRemoveSelectParts.Any())
            {
                foreach (var matchingRemoveSelectPart in matchingRemoveSelectParts)
                {
                    var definingQuery = element.ElementsAnyNS("DefiningQuery").Single();
                    definingQuery.Value = definingQuery.Value.Replace(string.Format(", \n\"{0}\".\"{1}\" AS \"{1}\"", matchingRemoveSelectPart.Table, matchingRemoveSelectPart.Column), "");
                }
            }
            else
            {
                var nodesToRemove = element.Nodes()
                    .Where(o =>
                        o is XElement
                        && ((XElement) o).Attribute("Name") != null
                        && tableAndColumn.Any(oo => oo.IsRemove && ((XElement) o).Attribute("Name").Value == oo.Column));

                foreach (var node in nodesToRemove.ToList())
                {
                    node.Remove();
                }

                if (element.Attribute("Name") != null && tableAndColumn.Any(oo => oo.Table == element.Attribute("Name").Value))
                {
                    var elementsToAdd = tableAndColumn.Where(o => !o.IsRemove && o.Table == element.Attribute("Name").Value);
                    if (new[] {"Type=\"number\"", "Type=\"varchar2\"", "Type=\"date\""}.Any(o => element.ToString().Contains(o)))
                    {
                        foreach (var columnToModify in elementsToAdd)
                        {
                            var columnToAdd = (ColumnToAdd) columnToModify;

                            var type = new[] {typeof (decimal), typeof (float), typeof (int), typeof (bool)}.Contains(columnToAdd.Type)
                                ? "number"
                                : columnToAdd.Type == typeof (DateTime) ? "date" : "varchar2";

                            var precision = "";
                            var scale = "";
                            var maxLength = "";
                            if (type == "number")
                            {
                                precision = "38";
                                scale = new[] {typeof (decimal), typeof (float)}.Contains(columnToAdd.Type) ? "2" : "0";
                            }

                            if (type == "varchar2")
                            {
                                maxLength = "500";
                            }

                            var newProperty = new XElement(element.GetDefaultNamespace() + "Property", new XAttribute("Name", columnToAdd.Column), new XAttribute("Type", type));
                            if (!string.IsNullOrWhiteSpace(precision))
                            {
                                newProperty.Add(new XAttribute("Precision", precision));
                            }

                            if (!string.IsNullOrWhiteSpace(scale))
                            {
                                newProperty.Add(new XAttribute("Scale", scale));
                            }

                            if (!string.IsNullOrWhiteSpace(maxLength))
                            {
                                newProperty.Add(new XAttribute("MaxLength", maxLength));
                            }

                            element.Add(newProperty);
                        }
                    }
                    else if (
                        new[] {"Type=\"Decimal\"", "Type=\"String\"", "Type=\"DateTime\"", "Type=\"Boolean\"", "Type=\"Byte\"", "Type=\"Int16\"", "Type=\"Int32\"", "Type=\"Int64\""}.Any(
                            o => element.ToString().Contains(o)))
                    {
                        foreach (var columnToModify in elementsToAdd)
                        {
                            var columnToAdd = (ColumnToAdd) columnToModify;

                            var type = new[] {typeof (decimal), typeof (float), typeof (int), typeof (bool)}.Contains(columnToAdd.Type)
                                ? "Decimal"
                                : columnToAdd.Type == typeof (DateTime) ? "DateTime" : "String";

                            var precision = "";
                            var scale = "";
                            var maxLength = "";
                            if (type == "Decimal")
                            {
                                precision = "38";
                                scale = new[] {typeof (decimal), typeof (float)}.Contains(columnToAdd.Type) ? "2" : "0";
                            }

                            if (type == "String")
                            {
                                maxLength = "500";
                            }

                            var newProperty = new XElement(element.GetDefaultNamespace() + "Property", new XAttribute("Name", columnToAdd.Column), new XAttribute("Type", type));
                            if (!string.IsNullOrWhiteSpace(precision))
                            {
                                newProperty.Add(new XAttribute("Precision", precision));
                            }

                            if (!string.IsNullOrWhiteSpace(scale))
                            {
                                newProperty.Add(new XAttribute("Scale", scale));
                            }

                            if (!string.IsNullOrWhiteSpace(maxLength))
                            {
                                newProperty.Add(new XAttribute("MaxLength", maxLength));
                                newProperty.Add(new XAttribute("FixedLength", "false"));
                                newProperty.Add(new XAttribute("Unicode", "false"));
                            }

                            element.Add(newProperty);
                        }
                    }
                }
            }

            if (element.Attribute("Name") != null && tableAndColumn.Any(oo => oo.Table == element.Attribute("Name").Value) && element.GetNamespaceOfPrefix("store") != null &&
                element.Attribute(element.GetNamespaceOfPrefix("store") + "Type") != null &&
                element.Attribute(element.GetNamespaceOfPrefix("store") + "Type").Value == "Tables")
            {
                var matchingAddSelectParts = tableAndColumn.Where(o => !o.IsRemove && o.Table == element.Attribute("Name").Value);
                foreach (var matchingAddSelectPart in matchingAddSelectParts)
                {
                    var definingQuery = element.ElementsAnyNS("DefiningQuery").Single();
                    var schemaRegex = new Regex(string.Format("\\nFROM \\\"([a-zA-Z0-9]*)\\\".\\\"{0}\\\"", matchingAddSelectPart.Table));
                    var schema = schemaRegex.Matches(definingQuery.Value)[0].Groups[1].Value;
                    definingQuery.Value = definingQuery.Value.Replace(
                        string.Format("\nFROM \"{0}\".\"{1}\" \"{1}\"", schema, matchingAddSelectPart.Table),
                        string.Format(", \n\"{0}\".\"{1}\" AS \"{1}\"\nFROM \"{2}\".\"{0}\" \"{0}\"", matchingAddSelectPart.Table, matchingAddSelectPart.Column, schema));
                }
            }

            if (element.Attribute("StoreEntitySet") != null && tableAndColumn.Any(oo => !oo.IsRemove && oo.Table == element.Attribute("StoreEntitySet").Value))
            {
                var matchingAddSelectParts = tableAndColumn.Where(o => !o.IsRemove && o.Table == element.Attribute("StoreEntitySet").Value);
                foreach (var matchingAddSelectPart in matchingAddSelectParts)
                {
                    element.Add(new XElement(element.GetDefaultNamespace() + "ScalarProperty", new XAttribute("Name", matchingAddSelectPart.Column),
                        new XAttribute("ColumnName", matchingAddSelectPart.Column)));
                }
            }
        }
    }

    public static EntityConnection Create(List<ColumnToModify> tablesAndColumns, string connString)
    {
        var modelNameRegex = new Regex(@".*metadata=res:\/\/\*\/([a-zA-Z.]*).csdl|.*");
        var model = modelNameRegex.Matches(connString).Cast<Match>().SelectMany(o => o.Groups.Cast<Group>().Skip(1).Where(oo => oo.Value != "")).Select(o => o.Value).First();

        var conceptualReader = XmlReader.Create(Assembly.GetExecutingAssembly().GetManifestResourceStream(model + ".csdl"));
        var mappingReader = XmlReader.Create(Assembly.GetExecutingAssembly().GetManifestResourceStream(model + ".msl"));
        var storageReader = XmlReader.Create(Assembly.GetExecutingAssembly().GetManifestResourceStream(model + ".ssdl"));

        var conceptualXml = XElement.Load(conceptualReader);
        var mappingXml = XElement.Load(mappingReader);
        var storageXml = XElement.Load(storageReader);

        foreach (var entitySet in new[] {storageXml, conceptualXml}.SelectMany(xml => xml.Elements()))
        {
            if (entitySet.Attribute("Name").Value == "ModelStoreContainer")
            {
                foreach (var entityContainerEntitySet in entitySet.Elements())
                {
                    ModifyNodes(entityContainerEntitySet, tablesAndColumns);
                }
            }

            ModifyNodes(entitySet, tablesAndColumns);
        }

        foreach (var entitySet in mappingXml.Elements().ElementAt(0).Elements())
        {
            if (entitySet.Name.LocalName == "EntitySetMapping")
            {
                foreach (var entityContainerEntitySet in entitySet.Elements().First().Elements())
                {
                    ModifyNodes(entityContainerEntitySet, tablesAndColumns);
                }
            }

            ModifyNodes(entitySet, tablesAndColumns);
        }

        var storageCollection = new StoreItemCollection(new [] {storageXml.CreateReader()});
        var conceptualCollection = new EdmItemCollection(new[] { conceptualXml.CreateReader() });
        var mappingCollection = new StorageMappingItemCollection(conceptualCollection, storageCollection, new[] {mappingXml.CreateReader()});

        var workspace = new MetadataWorkspace();

        workspace.RegisterItemCollection(conceptualCollection);
        workspace.RegisterItemCollection(storageCollection);
        workspace.RegisterItemCollection(mappingCollection);
        var connectionData = new EntityConnectionStringBuilder(connString);
        var connection = DbProviderFactories
            .GetFactory(connectionData.Provider)
            .CreateConnection();
        connection.ConnectionString = connectionData.ProviderConnectionString;

        return new EntityConnection(workspace, connection);
    }
}

Initialization:

public ActionResult QUOTE_HOUSE()
    {
        var onlineDocs = Enumerable.Empty<QUOTE_HOUSE>();
        var mappings = new List<SagaSystemToDatabaseMapping>{new SagaSystemToDatabaseMapping("x", "Oracle", "Db1",
                   "metadata=res://*/Ecom.Ecom.csdl|res://*/Ecom.Ecom.ssdl|res://*/Ecom.Ecom.msl;provider=Oracle.ManagedDataAccess.Client;provider connection string='...'", typeof(EcomEntities))
                {
                    ColumnsToModify = new List<ColumnToModify> { new ColumnToAdd("QUOTE_HOUSE","TESTCOL", typeof(string))    }
                }};
        var entityConnection = EntityConnectionExtensions.Create(mappings[0].ColumnsToModify,mappings[0].ConnectionString);
        using (var db = new EcomEntities(entityConnection))
        {
            onlineDocs = db.QUOTE_HOUSE.Take(10);
        }

        return View("QUOTE_HOUSE", onlineDocs.ToList());
    }

You should be able to generate oracle database from entity QUOTE_HOUSE and enter some dummy values, don't think you need a view as it blows up on .ToList(). After you generated the database add additional column to database but not model (alter table QUOTE_HOUSE add TESTCOL Varchar2(20)) - to have column in database that is being injected at runtime in model. You might also need to debug EF assemblies here's how to do it. Please let me know if you need more info or I have missed something.

I know this is probably not what you are expecting, but I guess at least it would help you not wasting more time in that direction.

The good news is that the problem is not caused by your "hackish" code. In fact I was able to reproduce the issue without using that code. Just created a QUOTE_HOUSE table containing a TestCol, imported it into a new edmx context and just deleted the generated TestCol property from the entity class. Then I've made the class inherit the DynamicEntity, created a context using the default constructor, called context.QUOTE_HOUSE.ToList() and it blew up with the exact same exception.

The bad news is that what you are trying to achieve is just not possible. EF uses nothing more and nothing less than reflection for mapping "object space" members. It does not offer any type extension mechanism like TypeDescriptor for instance, or dynamic runtime (it does not even allow you to project to dynamic objects). The later is understandable, since every dynamic object may have a different properties, and there is no such thing as dynamic type. Note that the trick with "removing" columns at run time works because what it does is basically the same as using NotMapped in code first, i.e. the property really exists, but is ignored by EF.

If you are interested in why the entity is in CSpace but not in OSpace, the answer is contained in an internal class called OSpaceTypeFactory (inside System.Data.Entity.Core.Metadata.Edm namespace) - source. There is a method called TryCreateStructuralType, which calls TryCreateMembers and if it returns false, the type is not added. TryCreateMembers in turns calls TryFindAndCreatePrimitiveProperties, passing the PropertyInfo list extracted using reflection, and the later returns false if it cannot map any CSpace member to the OSpace object property, thus effectively preventing the type to be added to the OSpace type collection.