Moleculer: NEXT: Jaeger don't trace remote actions

[me23]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [x] I am running the latest version
• [x] I checked the documentation and found no answer
• [x] I checked to make sure that this issue has not already been filed
• [x] I'm reporting the issue to the correct repository

Current Behavior

Hi all,

i am evaluation tracing with jaeger using the current "next" branch:

I am running a jaeger instance in docker.
I created one service with one action (service1.ping)
If i call the action locally in the broker.start hook of this service, the span will successfully
exported to jaeger.
If i call the service from an other container using rabbitmq
the remote span is not exported to jaeger.

maybe this feature is not ready, because the remoteAction is commented out in middlewares/tracing.js:

return {
...
        name: "Tracing",

        localAction: broker.isTracingEnabled() && tracer.opts.actions ? tracingLocalActionMiddleware : null,
        localEvent: broker.isTracingEnabled() && tracer.opts.events ? tracingLocalEventMiddleware : null,
        //remoteAction: wrapRemoteTracingMiddleware
    };

Expected Behavior

The remote call of the service should also be exported to jaeger.

Failure Information

Steps to Reproduce

...
const broker = new ServiceBroker({
   ...
   tracing: {
        enabled: true,
        stackTrace: true,
        exporter: [
          {
            type: "Jaeger",
            options: {
              endpoint:  "http://192.168.101.1:14268/api/traces", // jaeger in docker            
              sampler: {
                type: "const",
                options: {
                },
              },
              tracerOptions: {
              },
            },
          },
        ],
});
broker.loadService("services/service1"); //action: service1.ping 

broker.start()
        .then(() => {
            console.log("Service1 started");

            broker.call("service1.ping").then((res: any) => {
                console.log(res); // => span goes to jaeger
            });
        });
...

then in other instance, i called the service1 over rabbitmq like this:

...
broker2.call("service1.ping").then(...); // => not reported to jaeger
...

The local call in the start-hook is reported to jaeger successfull.

If i call server1.ping over rabbitmq the span is not reported to jaeger.

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

• Moleculer version: 0.14.0-beta3
• NodeJS version: 12.8.1
• Operating System: alpine in docker

[icebob]

Hi,

the //remoteAction: wrapRemoteTracingMiddleware line is not necessary.
Did you configure tracing in moleculer.config in all (including remote) nodes?

[me23]

Hi,
yes, all nodes have tracing configured. I created a test-repo here me23/moleculer-jaeger-test.
My test scenario looks like this:
The invoker calls service1 every 15 seconds.
service1 first calls service2 and returns this result back.

So i expected to see a trace with 3 spans: invoker -> service1 -> service2.
But none of these are exported to jaeger.

If i call service1 or service2 in the corresponding start-hooks of each service locally, then i
see 2 traces. One service1 -> service2 and one only for service2.

May be i miss something?

Thank you!
me23

[icebob]

Thank you for your test repo. I've found the issue and fixed.

[icebob]

Please try it. Install with npm i moleculerjs/moleculer#next

[me23]

Thank you, now it looks much better: each call show service1 and service2 in jaeger.
But i am missing the span for the invoker call...

In my example i would expect a trace with 3 spans: Invoker -> servcie1 -> service2

[icebob]

Tracing logs the action calls, the invoker just executes a broker.call, there is no action invoking locally.

[me23]

Ok, do you see a way i could start a span (for jaeger) to trace the invoker also?

[icebob]

Yes of course. You can anywhere create custom tracing spans.

broker1.start()
    .then(() => {
        console.log("invoker started");
        setTimeout(async () => {
            const span = broker1.tracer.startSpan("Invoke service1.ping");
            const res = await broker1.call("service1.ping", null, { parentSpan: span });
            console.log(res);
            span.finish();
        }, 5000);
    });