Akka.net: An Akka.NET App fails to start under .NET Core 3.0

[mmisztal1980]

I resumed work on an old pet-project (stated at 1.3.9), updated the packages to 1.3.15 and started to get a NullReferenceException on ActorSystem.Create (project was upgraded to netcoreapp3.0):
StackTrace:

   at Akka.Configuration.Config.WithFallback(Config fallback)

Failing code:

        public ActorSystem Build()
        {
            if (string.IsNullOrEmpty(actorSystemName))
            {
                throw new InvalidOperationException("ActorSystem does not have a name");
            }

            if (this.akkaConfigs.Count == 0)
            {
                throw new InvalidOperationException("Unconfigured ActorSystem");
            }

            var config = ConfigurationFactory.Empty;

            foreach (var cfg in akkaConfigs)
            {
                config = config.WithFallback(cfg);
            }

            return ActorSystem.Create(actorSystemName, config);
        }

The configs in akkaConfigs were:

  akka : {
    stdout-loglevel : INFO
    loglevel : INFO
    log-config-on-start : on
    loggers : ["Akka.Logger.Serilog.SerilogLogger, Akka.Logger.Serilog"]
    actor : {
      debug : {
        receive : on
        autoreceive : on
        lifecycle : on
        event-stream : on
        unhandled : on
      }
    }
  }

and

  akka : {
    remote : {
      dot-netty : {
        tcp : {
          log-transport : true
          transport-class : "Akka.Remote.Transport.DotNetty.TcpTransport, Akka.Remote"
          transport-protocol : tcp
          hostname : 0.0.0.0
          public-hostname : localhost
          port : 9000
        }
      }
    }
  }

and finally:

  akka : {
    actor : {
      provider : "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
    }
    cluster : {
      log-info : on
      seed-nodes : ["akka.tcp://System@localhost:9000"]
      roles : []
      role : []
    }
  }

Any hints what may have changed? .WithFallback used to be perfectly legit as far as I recall - unless .NET Core 3.0 is unsupported...? I've created a simplified repro project here: https://github.com/mmisztal1980/akka-netcore30-repro

[Aaronontheweb]

Thanks @mmisztal1980 - I will look at this ASAP and see if we can reproduce. Having some other .NET Core 3.0-related issues with Hyperion and Petabridge.Cmd coming up as well, so I guess it's time to pay the piper...

[bromanko]

The underlying issue has been fixed in Hyperion.

[Aaronontheweb]

Is this app using Hyperion?

[bromanko]

I don't think so. I've configured my serializers to use binary serialization. However, I was just starting to add clustering and received this error. It's stemming from the fact that the ReplicatedDataSerializer creates a HyperionSerializer in its constructor.

[Aaronontheweb]

@mmisztal1980 so I've been able to reproduce this error, but also on .NET Core 2.2. I think this problem is related to the config or an issue with our parser, not a runtime or platform issue.

[Aaronontheweb]

Also, some examples of Akka.NET apps on .NET Core 3.0 that run without any issues:

• Petabridge.App - uses Akka.Cluster
• Petabridge.App.Web - uses Akka.Cluster inside ASP.NET Core 3.0

I'll look at the Hyperion PRs with @Danthar and see about doing a release there to alleviate this.

[mmisztal1980]

@Aaronontheweb just so that I understand better, is the HOCON malformed?

[Aaronontheweb]

https://github.com/akkadotnet/akka.net/pull/3953 - @mmisztal1980 working on a repro spec now.

For reasons unknown, source link was letting me see where the error was happening. It may be set up incorrectly or our symbol package publishing may not be working correctly.

[Aaronontheweb]

Looks like the issue is here:

https://github.com/akkadotnet/akka.net/blob/ca9916097b5efc06b7e9fbd408306f2df456a3da/src/core/Akka/Configuration/Config.cs#L435-L447

The fallback passed into this method was null. Happened when looking at the role property in the example HOCON you provided in your reproduction spec.

[Aaronontheweb]

Actually, the fallback param isn't null - something inside this line is:

var mergedRoot = Root.GetObject().MergeImmutable(fallback.Root.GetObject()); 

[Aaronontheweb]

Ok, so this is pretty odd - looks like this extra / unknown HOCON setting, akka.cluster.role is what broke this code.

Removing it fixed it - just pushed a new commit that will reflect that. I don't know if we're going to retroactively patch this, as we're trying to get rid of the HOCON built-in to this library with the new stand-alone HOCON library, which has radically different (better) code for handling this type of issue.

[bromanko]

I confirmed that the issue I'm reporting isn't exactly the same. I grabbed the Petabridge.App.Web project @Aaronontheweb mentioned and added two lines two create a shard configuration. That's what causes the error with Hyperion:

Adding:

var clusterSharding = ClusterSharding.Get(Sys);
var shardRegion = clusterSharding.Start("test", Props.Empty,
                ClusterShardingSettings.Create(Sys), new MessageExtractor());

Leads to this exception:

Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.TypeInitializationException: The type initializer for 'Hyperion.SerializerOptions' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'Hyperion.SerializerFactories.ExceptionSerializerFactory' threw an exception.
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'type')
   at System.Reflection.TypeExtensions.GetMethod(Type type, String name, BindingFlags bindingAttr)
   at Hyperion.SerializerFactories.ExceptionSerializerFactory..cctor()
   --- End of inner exception stack trace ---
   at Hyperion.SerializerFactories.ExceptionSerializerFactory..ctor()
   at Hyperion.SerializerOptions..cctor()
   --- End of inner exception stack trace ---
   at Hyperion.SerializerOptions..ctor(Boolean versionTolerance, Boolean preserveObjectReferences, IEnumerable`1 surrogates, IEnumerable`1 serializerFactories, IEnumerable`1 knownTypes, Boolean ignoreISerializable)
   at Akka.DistributedData.Serialization.ReplicatorMessageSerializer..ctor(ExtendedActorSystem system)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
   at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
   at Akka.Serialization.Serialization..ctor(ExtendedActorSystem system)
   at Akka.Actor.Internal.ActorSystemImpl.ConfigureSerialization()
   at Akka.Actor.Internal.ActorSystemImpl.Akka.Actor.Internal.ISupportSerializationConfigReload.ReloadSerialization()
   at Akka.Actor.Settings.RebuildConfig()
   at Akka.Actor.Settings.InjectTopLevelFallback(Config config)
   at Akka.Cluster.Sharding.ClusterSharding..ctor(ExtendedActorSystem system)
   at Akka.Cluster.Sharding.ClusterShardingExtensionProvider.CreateExtension(ExtendedActorSystem system)
   at Akka.Actor.ExtensionIdProvider`1.Akka.Actor.IExtensionId.CreateExtension(ExtendedActorSystem system)
   at Akka.Actor.Internal.ActorSystemImpl.<>c__DisplayClass56_0.<RegisterExtension>b__1()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Akka.Actor.Internal.ActorSystemImpl.TryGetExtension(Type extensionType, Object& extension)
   at Akka.Actor.Internal.ActorSystemImpl.GetExtension(IExtensionId extensionId)
   at Akka.Actor.ExtensionIdProvider`1.Get(ActorSystem system)
   at Akka.Actor.ExtensionIdProvider`1.Akka.Actor.IExtensionId.Get(ActorSystem system)
   at Akka.Actor.Internal.ActorSystemImpl.RegisterExtension(IExtensionId extension)
   at Akka.Actor.ActorSystemWithExtensions.WithExtension[T,TI](ActorSystem system)
   at Akka.Cluster.Sharding.ClusterSharding.Get(ActorSystem system)
   at Petabridge.App.Web.AkkaService.StartAsync(CancellationToken cancellationToken) in /Users/bromanko/Code/oss/Petabridge.App.Web/src/Petabridge.App.Web/AkkaService.cs:line 45
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at Petabridge.App.Web.Program.Main(String[] args) in /Users/bromanko/Code/oss/Petabridge.App.Web/src/Petabridge.App.Web/Program.cs:line 18

Do you want me to move this to a new issue?

[Aaronontheweb]

@bromanko yes please! That's a real issue and we need to fix it.

[mmisztal1980]

Ok I think I see it now:

https://getakka.net/articles/configuration/akka.cluster.html indicates that a akka.cluster.role is a valid HOCON setting, however the expected type is an object rather than an array

[Aaronontheweb]

@mmisztal1980 so, can we close this?

[mmisztal1980]

by all means