JsonConvert.Deserializer indexing issues

Question

While playing around Stack collection in C# I encountered the following issue. Exactly I am not sure why it is happening. Please put some light on the reason and alternative to the solution.

Problem -

A class having Stack as property. For example name that class as Progress. T is of class type Item.

Now whenever the user makes any progress we will be storing in stack. And if user leaves in between, then next time we will peek the item from the stack so from that stage. Below code snippet will give an idea of what is being tried...

using static System.Console;
using System.Collections.Generic;
using Newtonsoft.Json;

namespace StackCollection
{
    class Program
    {
        static void Main(string[] args)
        {
            Progress progress = new Progress();

            progress.Items.Push(new Item { PlanID = null, PlanName = "Plan A" });

            var jsonString = JsonConvert.SerializeObject(progress);
            var temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            temp.Items.Push(new Item { PlanID = null, PlanName = "Plan B" });

            jsonString = JsonConvert.SerializeObject(temp);
            temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            temp.Items.Push(new Item { PlanID = null, PlanName = "Plan C" });

            jsonString = JsonConvert.SerializeObject(temp);
            temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            WriteLine(temp.Items.Peek().PlanName);

            ReadLine();
        }
    }

    class Progress
    {
        public Stack<Item> Items { get; set; }

        public Progress()
        {
            Items = new Stack<Item>();
        }
    }

    class Item
    {
        public string PlanID { get; set; }

        public string PlanName { get; set; }
    }
}

now the line -

WriteLine(temp.Items.Peek().PlanName);

should return

Plan C

but it is returning

Plan B

So, why the index is being changed, any clue or pointer will be helpful.

Answer 1

It seems like the stack is being serialized as a List. The problem is that this does not preserve the proper order when deconstructing the stack (the items are actually pushed in the reverse order). Here's a quick workaround to this issue:

using System;
using static System.Console;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using Newtonsoft.Json;

namespace StackCollection
{
    class Program
    {
        static void Main(string[] args)
        {
            Progress progress = new Progress();

            progress.Items.Push(new Item { PlanID = null, PlanName = "Plan A" });

            var jsonString = JsonConvert.SerializeObject(progress);
            var temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            temp.Items.Push(new Item { PlanID = null, PlanName = "Plan B" });

            jsonString = JsonConvert.SerializeObject(temp);
            temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            temp.Items.Push(new Item { PlanID = null, PlanName = "Plan C" });

            jsonString = JsonConvert.SerializeObject(temp);
            temp = JsonConvert.DeserializeObject<Progress>(jsonString);

            WriteLine(temp.Items.Peek().PlanName);

            ReadLine();
        }
    }

    class Progress
    {
        [JsonIgnore]
        public Stack<Item> Items { get; set; }

        public List<Item> ItemList { get; set; }

        [OnSerializing]
        internal void OnSerializing(StreamingContext context)
        {
            ItemList = Items?.ToList();
        }

        [OnDeserialized]
        internal void OnDeserialized(StreamingContext context)
        {
            ItemList?.Reverse();
            Items = new Stack<Item>(ItemList ?? Enumerable.Empty<Item>());
        }

        public Progress()
        {
            Items = new Stack<Item>();
        }
    }

    class Item
    {
        public string PlanID { get; set; }

        public string PlanName { get; set; }
    }
}

Answer 2

Since this is a known behavior of Json.NET, as noted by this answer, a custom JsonConverter can be used when deserializing a stack that pushes items on in the correct order.

The following universal converter can be used with Stack<T> for any T:

/// <summary>
/// Converter for any Stack<T> that prevents Json.NET from reversing its order when deserializing.
/// </summary>
public class StackConverter : JsonConverter
{
    // Prevent Json.NET from reversing the order of a Stack<T> when deserializing.
    // https://github.com/JamesNK/Newtonsoft.Json/issues/971
    static Type StackParameterType(Type objectType)
    {
        while (objectType != null)
        {
            if (objectType.IsGenericType)
            {
                var genericType = objectType.GetGenericTypeDefinition();
                if (genericType == typeof(Stack<>))
                    return objectType.GetGenericArguments()[0];
            }
            objectType = objectType.BaseType;
        }
        return null;
    }

    public override bool CanConvert(Type objectType)
    {
        return StackParameterType(objectType) != null;
    }

    object ReadJsonGeneric<T>(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        var list = serializer.Deserialize<List<T>>(reader);
        var stack = existingValue as Stack<T> ?? (Stack<T>)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
        for (int i = list.Count - 1; i >= 0; i--)
            stack.Push(list[i]);
        return stack;
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        try
        {
            var parameterType = StackParameterType(objectType);
            var method = GetType().GetMethod("ReadJsonGeneric", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public);
            var genericMethod = method.MakeGenericMethod(new[] { parameterType });
            return genericMethod.Invoke(this, new object[] { reader, objectType, existingValue, serializer });
        }
        catch (TargetInvocationException ex)
        {
            // Wrap the TargetInvocationException in a JsonSerializerException
            throw new JsonSerializationException("Failed to deserialize " + objectType, ex);
        }
    }

    public override bool CanWrite { get { return false; } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

Then add it to JsonSerializerSettings to correct the ordering of stacks when deserializing:

var settings = new JsonSerializerSettings { Converters = new[] { new StackConverter() } };
var jsonString = JsonConvert.SerializeObject(progress, settings);
var temp = JsonConvert.DeserializeObject<Progress>(jsonString, settings);

Or mark the Stack<T> property directly with [JsonConverter(typeof(StackConverter))]:

class Progress
{
    [JsonConverter(typeof(StackConverter))]
    public Stack<Item> Items { get; set; }

    public Progress()
    {
        Items = new Stack<Item>();
    }
}

Answer 3

If you try to debug it then you will notice that items order is broken after Stack deserialization.

The same question has been asked on JSON.NET GitHub issue tracker a month ago.

The answer from JamesNK:

I'm afraid this is a limition of a Stack. The results returned when it is serialized and the opposite order for when it is deserialized.