Orleans: Class with autogenerated serializer fails to deserialize

[mehmetakbulut]

Instances of MyClass in the following snippet serializes fine via the autogenerated Orleans serializer however they fail to deserialize upon arrival at the client.

namespace OtherLibrary
{
    [Serializable]
    public abstract class BaseClass
    {
    }

    [Serializable]
    public abstract class GenericClass<T> : BaseClass where T : BaseClass
    {
    }

    [Serializable]
    public class MyClass : GenericClass<MyClass>
    {
    }
}

In addition to the OtherLibrary, the solution contains 3 more projects which are the default Orleans templates for silo host, grain interfaces and grain collection with the following changes.

Silo Host:

namespace SiloHost1
{
    /// <summary>
    /// Orleans test silo host
    /// </summary>
    public class Program
    {
        static void Main(string[] args)
        {
            // The Orleans silo environment is initialized in its own app domain in order to more
            // closely emulate the distributed situation, when the client and the server cannot
            // pass data via shared memory.
            AppDomain hostDomain = AppDomain.CreateDomain("OrleansHost", null, new AppDomainSetup
            {
                AppDomainInitializer = InitSilo,
                AppDomainInitializerArguments = args,
            });

            var config = ClientConfiguration.LocalhostSilo();
            GrainClient.Initialize(config);

            var my_grain = GrainClient.GrainFactory.GetGrain<IGenericGrain<MyClass>>(Guid.Empty);
            var x = my_grain.GetValue();
            x.Wait();
            Console.WriteLine(x);

            Console.WriteLine("Orleans Silo is running.\nPress Enter to terminate...");
            Console.ReadLine();

            hostDomain.DoCallBack(ShutdownSilo);
        }
...
}

Grain Interfaces:

namespace GrainInterfaces1
{
    public interface IBaseGrain : Orleans.IGrainWithGuidKey
    {
    }

    public interface IGenericGrain<T> : IBaseGrain
        where T : GenericClass<T>
    {
        Task<T> GetValue();
    }
}

Grain Implementations/Collection:

namespace Grains1
{
    public abstract class GenericGrain<T> : Orleans.Grain, IGenericGrain<T>
        where T : GenericClass<T>, new()
    {
        private T Value;

        public override Task OnActivateAsync()
        {
            Value = new T();
            return base.OnActivateAsync();
        }

        public Task<T> GetValue()
        {
            return Task.FromResult(Value);
        }
    }

    public class MyClassGrain : GenericGrain<MyClass>
    {
    }
}

Exception gets raised at x.Wait() in silohost/client with the following stack trace:

[2017-06-14 01:28:47.532 GMT    32  ERROR   101030  Message 10.0.0.4:0] !!!!!!!!!! Exception deserializing message body 
Exc level 0: System.Runtime.Serialization.SerializationException: Unsupported type 'OtherLibrary.MyClass' encountered. Perhaps you need to mark it [Serializable] or define a custom serializer for it?
   at Orleans.Serialization.SerializationManager.DeserializeInner(Type expected, IDeserializationContext context)
   at Orleans.Serialization.BuiltInTypes.DeserializeOrleansResponse(Type expected, IDeserializationContext context)
   at Orleans.Serialization.SerializationManager.DeserializeInner(Type expected, IDeserializationContext context)
   at Orleans.Serialization.SerializationManager.Deserialize(Type t, BinaryTokenStreamReader stream)
   at Orleans.Runtime.Message.DeserializeBody(List`1 bytes)

Grain Implementation project produces the following autogenerated code

#if !EXCLUDE_CODEGEN
#pragma warning disable 162
#pragma warning disable 219
#pragma warning disable 414
#pragma warning disable 649
#pragma warning disable 693
#pragma warning disable 1591
#pragma warning disable 1998
[assembly: global::System.CodeDom.Compiler.GeneratedCodeAttribute("Orleans-CodeGenerator", "1.4.0.0")]
[assembly: global::Orleans.CodeGeneration.OrleansCodeGenerationTargetAttribute("Grains1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
namespace OtherLibrary
{
    using global::Orleans.Async;
    using global::Orleans;
    using global::System.Reflection;

    [global::System.CodeDom.Compiler.GeneratedCodeAttribute("Orleans-CodeGenerator", "1.4.0.0"), global::System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute, global::Orleans.CodeGeneration.SerializerAttribute(typeof (global::OtherLibrary.MyClass))]
    internal class OrleansCodeGenOtherLibrary_MyClassSerializer
    {
        [global::Orleans.CodeGeneration.CopierMethodAttribute]
        public static global::System.Object DeepCopier(global::System.Object original, global::Orleans.Serialization.ICopyContext context)
        {
            global::OtherLibrary.MyClass input = ((global::OtherLibrary.MyClass)original);
            global::OtherLibrary.MyClass result = new global::OtherLibrary.MyClass();
            context.@RecordCopy(original, result);
            return result;
        }

        [global::Orleans.CodeGeneration.SerializerMethodAttribute]
        public static void Serializer(global::System.Object untypedInput, global::Orleans.Serialization.ISerializationContext context, global::System.Type expected)
        {
            global::OtherLibrary.MyClass input = (global::OtherLibrary.MyClass)untypedInput;
        }

        [global::Orleans.CodeGeneration.DeserializerMethodAttribute]
        public static global::System.Object Deserializer(global::System.Type expected, global::Orleans.Serialization.IDeserializationContext context)
        {
            global::OtherLibrary.MyClass result = new global::OtherLibrary.MyClass();
            context.@RecordObject(result);
            return (global::OtherLibrary.MyClass)result;
        }
    }
}
#pragma warning restore 162
#pragma warning restore 219
#pragma warning restore 414
#pragma warning restore 649
#pragma warning restore 693
#pragma warning restore 1591
#pragma warning restore 1998
#endif

As it can be seen here, MyClass does indeed have a serializer. Nevertheless, client fails to deserialize it.

I'm assuming this is due to the rather intricate inheritance hierarchies involved however I'm unclear as to why this would serialize but not deserialize.

Tried with both 1.4.2 and 1.4.0. Exact same error on both versions.

[jdom]

I assume this is because the client does not have a reference to where MyClass is defined, or if it has a reference, the assembly might not be loaded, and hence it fails to find the type that in needs for deserialization.

Since MyClass seems to be part of your contract (even though implicitly, since it's not in a grain method signature itself), try moving it to the grain interfaces project.

[mehmetakbulut]

Even though the client calls the grain using var my_grain = GrainClient.GrainFactory.GetGrain<IGenericGrain<MyClass>>(Guid.Empty);, would you still say MyClass may not be loaded?

[jdom]

hmmm, no, I missed that, it's probably not that the assembly isn't loaded. But before we move to other theories, try doing var ignore = typeof(MyClass); just before you initialize the client. This way we can be sure that the type is available by the time the serialization manager is being configured.
I'm insisting with that before moving on, as that exception is typically seen with assembly load issues.

[mehmetakbulut]

Just tried it and no change. Stack trace is same.

I also tried moving the OtherLibrary namespace into the grain implementation project. This seems to resolve the deserialization problem but this is not a solution since: the underlying issue is still there and it is unviable for production.

It appears to me that the problem might be resulting from a parity between handling of serialization between client and silo. The grain implementation assembly contains the autogenerated serializer for MyClass whereas the grain interface assembly does not. Silo and the client live on the same assembly so in theory they should have access to the same generated serializers from both implementation and interface assemblies.

I just ran a test where the client called x.SetValue(new MyClass());. The object was successfully serialized by the client and deserialized by the silo. So the problem seems limited to deserialization at the client.

Grain interface and implementation are changed as follows for the SetValue(T value) method.

...
    public abstract class GenericGrain<T> : Orleans.Grain, IGenericGrain<T>
        where T : GenericClass<T>, new()
    {
...
        public Task SetValue(T value)
        {
            this.Value = value;
            return TaskDone.Done;
        }
    }
...

...
    public interface IGenericGrain<T> : IBaseGrain
        where T : GenericClass<T>
    {
...
        Task SetValue(T value);
    }
...

[jdom]

Not sure why it's not finding the serializer. Nevertheless, the OtherAssembly is technically part of the contract (albeit in an external assembly), so add an assembly level attribute to hint codegen about that fact. You need to add something like: [assembly: KnownAssembly(typeof(MyClass))] into the interfaces project's AssemblyInfo.cs

[jdom]

BTW, @mehmetakbulut did it work after adding the KnownAssembly attribute?

[mehmetakbulut]

It did work! Thank you so much!

I spent some time trying to figure out why codegen wasn't picking it up in the first place (hence why I asked about the VS solution on gitter so I can use the debugger) but I haven't been able to isolate it yet.

This does solve my problem, I'm happy to have the ticket closed. However if you have any ideas on the cause of this, I'd like to hear.

[jdom]

Good to hear that it worked.
Having part of the contract in an external assembly and having to use the KnownAssembly attribute is indeed by design, otherwise we would have to generate serializers for the entire world that is referenced by your grain interfaces.
Normally if you split the grains and the implementation in different projects, it's because you want your clients to not even reference the implementations project, otherwise it's just easier to collapse them to one.
What was surprising to me is that since there was already a serializer being generated in the grain implementations project (even though it should be in the interfaces project, where the contracts are), that the serialization manager scanning didn't find those automatically on the client, given that you were indeed referencing the implementations from it. Again, technically this is correct, but still was surprised that Orleans didn't find that type anyway.

/cc @ReubenBond