Akka.net: Cluster.Metrics Adaptive Load Balancer not collecting metrics

[simonlaroche]

When using the Adaptive Load Balancer with a Cluster Group Router the metrics are not collected by the AdaptiveLoadBalancingMetricsListener unless a new router management message that is not handled by the ClusterRouterActor and the RouterActor is sent to the router,

`
// RouterActor

    protected override void OnReceive(object message)
    {
        switch (message)
        {
            case GetRoutees getRoutees:
                Sender.Tell(new Routees(Cell.Router.Routees));
                break;
            case AddRoutee addRoutee:
                Cell.AddRoutee(addRoutee.Routee);
                break;
            case RemoveRoutee removeRoutee:
                Cell.RemoveRoutee(removeRoutee.Routee, stopChild: true);
                StopIfAllRouteesRemoved();
                break;
            case Terminated terminated:
                Cell.RemoveRoutee(new ActorRefRoutee(terminated.ActorRef), stopChild: false);
                StopIfAllRouteesRemoved();
                break;
            default:
                // Any other unhandled management message wil be forwarded to the controller and start collecting metrics
                RoutingLogicController?.Forward(message);
                break;
        }
    }

`

• Akka 1.14.12
• dotnet core 3.1

[Aaronontheweb]

We'll look into this @simonlaroche

[Arkatufus]

@simonlaroche which message/metrics are lost/missing?

[Aaronontheweb]

@Arkatufus looks like none of them? Like a misconfiguration issue with the wrong base class being used

[simonlaroche]

This is our router configuration:

var routerGroup = actorSystem.ActorOf(
                new ClusterRouterGroup(
                new AdaptiveLoadBalancingGroup(MixMetricsSelector.Instance),
                new ClusterRouterGroupSettings(
                        totalInstances: 100, 
                        new[] { $"/user/worker" },
                        allowLocalRoutees: false,
                        "workers")).Props(),
                "job-worker-group");

                routerGroup.Tell(PingAdaptiveLoadBalancingMetricsListener.Instance); 

public sealed class PingAdaptiveLoadBalancingMetricsListener : RouterManagementMessage
{
    private PingAdaptiveLoadBalancingMetricsListener()
    {
    }

    public static PingAdaptiveLoadBalancingMetricsListener Instance { get; } = new PingAdaptiveLoadBalancingMetricsListener();
}

Unless there is an error in our configuration, the only way we have found to activate the RoutingLogicController was by sending a custom router management message.

[Arkatufus]

Ok, thanks for the repro case, I'll start working on that.

[Arkatufus]

@simonlaroche I could not find any problem with the current code implementation, can you compare your code with the sample project I've made here?

[simonlaroche]

Sure I'll give it a look

[simonlaroche]

@Arkatufus I have just tried the sample and the same thing happens.

The AdaptiveLoadBalancingMetricsListener is never created and no metric is sent to the routing logic

When I send a custom router management message to the router it activates the AdaptiveLoadBalancingMetricsListener and it send the cluster metric changed message to the routing logic.

[Arkatufus]

@simonlaroche

Did you set up the listener actor like the one I did here and here?

When you send a message to the /user/worker actor from another node in the cluster, did it receive the message?

Can you check the log for any errors?

Can you make a simple sample project to illustrate the problem?

[simonlaroche]

I used the sample project you built in PR #4679. Started both the front end app and the back end app.

The cluster metrics are collected but they are not used by the router.

As I pointed out initially I think it is because the AdaptiveLoadBalancingMetricsListener actor is never created because it never receives any message.

[Arkatufus]

Ah, yes, I see what you mean now. AdaptiveLoadBalancingMetricsListener will never get created until RouterActor receives at least one routing message because it is instantiated inside OnReceive instead of the class constructor. Let me fix that right now.

[Arkatufus]

@simonlaroche should be fixed in #4700

[Aaronontheweb]

Still need to merge in #4700 to close this

[Aaronontheweb]

This should be resolved in tonight's nightly build and in Akka.NET v1.4.14