Opentelemetry-dotnet: Jaeger Exporter isn't working with OpenTelemetry.Collector.AspNetCore on OpenTelemetry.Hosting

[thangchung]

Currently, I have used OpenTelemetry at version 0.2.0-alpha.110. I used AspNetCore Collector on OpenTelemetry.Hosting with Zipkin Exporter, everything is working. Then I tried to use Jaeger Exporter when the application is running, but nothing exporting to Jaeger. The code as below

services.AddOpenTelemetry(builder =>
{
    builder.SetSampler(Samplers.AlwaysSample);

    builder.AddProcessorPipeline(c => c
        .SetExporter(new JaegerTraceExporter(Configuration.GetOptions<JaegerExporterOptions>("Jaeger")))
        .SetExportingProcessor(e => new BatchingSpanProcessor(e)));

    builder
        .UseZipkin(o => Configuration.Bind("Zipkin", o));

    builder.AddRequestCollector()
        .AddDependencyCollector();
});

I 'm wondering might Jaeger Exporter has a bug, but I don't know how to fix it.
The sample source code can be found at https://github.com/thangchung/opentelemetry-experiment/blob/master/src/SampleWeb/Startup.cs
Could anyone on the OT team tell me a solution? Thank you.

/cc @tungphuong

[MovGP0]

It seems like the UseJaeger() extension is missing for TracerBuilder:

using System;
using JetBrains.Annotations;
using OpenTelemetry.Exporter.Jaeger;
using OpenTelemetry.Trace.Configuration;
using OpenTelemetry.Trace.Export;

namespace OpenTelemetry.Exporter.Jaeger
{
    public static class TracerBuilderExtensions
    {
        /// <summary>Registers Jaeger exporter.</summary>
        /// <param name="builder">Trace builder to use.</param>
        /// <param name="configure">Configuration options.</param>
        /// <returns>The instance of <see cref="T:OpenTelemetry.Trace.Configuration.TracerBuilder" /> to chain the calls.</returns>
        public static TracerBuilder UseJaeger(
            [NotNull] this TracerBuilder builder,
            [NotNull] Action<JaegerExporterOptions> configure)
        {
            if (builder == null)
                throw new ArgumentNullException(nameof(builder));

            if (configure == null)
                throw new ArgumentNullException(nameof(configure));

            var options = new JaegerExporterOptions();
            configure(options);

            return builder.AddProcessorPipeline(b => b
                .SetExporter(new JaegerTraceExporter(options, null))
                .SetExportingProcessor(e => (SpanProcessor)new BatchingSpanProcessor(e)));
        }
    }
}

Startup.ConfigureServices

            services.AddOpenTelemetry(builder =>
            {
                builder
                    .SetSampler(Samplers.AlwaysSample)
                    .UseJaeger(opts =>
                    {
                        Configuration.GetSection("Telemetry").GetSection("Jaeger").Bind(opts);
                        opts.ServiceName = $"{HostingEnvironment.ApplicationName}_{HostingEnvironment.EnvironmentName}";
                    })
                    .AddRequestCollector()
                    .AddDependencyCollector();
            });

[thangchung]

@MovGP0 Thanks for your suggestion. I tried it, but still not working. Still waiting for the OT team to react

[hannahchan]

I'm having the same problem too. That or I don't understand the documentation.

[MikeGoldsmith]

I added the missing UseJaeger extension method in #347.

I'll try to test exporting to Jaeger using @thangchung's sample project soon.

[DOMZE]

Same here,

I'm using the code

services.AddOpenTelemetry(builder =>
{
    builder.SetSampler(Samplers.AlwaysSample)
        .UseJaeger(o =>
        {
            o.ServiceName = "my-service";
            o.AgentHost = JaegerTraceExporter.DefaultAgentUdpHost;
            o.AgentPort = JaegerTraceExporter.DefaultAgentUdpCompactPort;
        })
        .AddRequestCollector()
        .AddDependencyCollector()
        .SetResource(new Resource(new Dictionary<string, string> { { "service.name", "my-service" } }));
});

and nothing gets exported to jaegger. Using 0.2.0-alpha.121
The same code works if I use the zipkin exporter

[MikeGoldsmith]

I've just got round to testing @thangchung's example code and I can see spans in the Jaeger UI.

By default Jaeger doesn't select a service type - eg I had to select _meteorite-service_ from the Services dropdown in the top left.

@thangchung @DOMZE can you confirm if you've selected an active service when searching for spans?

[SergeyKanzhelev]

need a repro. Closing for now

[macon]

Can confirm this is still not working. The only service-type I see in Jaeger UI is it's own "jaeger-query" service, nothing from my app. As mentioned above, works with the Zipkin exporter.

[lingabalaji17]

Am also facing the above mentioned issue - traces are not logged from asp.net core web API to Jaeger UI. The service dropdown doesn't have the asp.net core service created.

Here is the configuration that I have followed in Startup -> ConfigureServices method. Could you let me know if I miss something here

```
services.AddSingleton(serviceProvider =>
{
return TracerFactory.Create(builder => builder.UseJaeger(o =>
{
o.ServiceName = "OpenTelemetry-NetCore";
o.AgentHost = "localhost";
o.AgentPort = 6831;

            }).AddDependencyCollector()
            .SetSampler(new AlwaysSampleSampler()));
        });

 services.AddOpenTelemetry();

[heldergregorio]

I was having the same problem. Had to download the code to debug what the problem was. In my case it was related to trying to load wrong dll versions.
When calling AppendAsync in JaegerTracerExpoerter.cs it was throwing an exception with the error, but since there isn't a catch block, I couldn't see the error anywhere.

[longility]

@heldergregorio Can you elaborate on how you debug by downloading the code? I have a different issue, and I want to go down this path. Did you have to pull in the project references into your own solution and add project dependencies, or is there a cleaner way?

[cijothomas]

One really fast way is to clone this repo, and run the samples. (samplesExporters).

[heldergregorio]

I've cloned the repo and instead of adding the reference to the nuget package, I've added the reference to the project. After that, build and everything works fine

[DOMZE]

@heldergregorio Could you elaborate on the versions (nuget packages) you were using when the problem occurred of all libraries involved?

[CodeBlanch]

I just tested JaegerExporter via the MyGet 0.2.0-alpha.246 package on a .NET Core 3.1 app, everything worked fine.

@lingabalaji17 I don't think your bootstrap is valid. You are manually registering TracerFactory and then calling AddOpenTelemetry which will spin up its own TracerFactory via:

services.AddOpenTelemetry(() => TracerFactory.Create(configure));

Meaning, your code is never used. If you switch to...

            services.AddOpenTelemetry(builder => {
                builder.UseJaeger(o =>
                {
                    o.ServiceName = "OpenTelemetry-NetCore";
                    o.AgentHost = "localhost";
                    o.AgentPort = 6831;

                }).AddDependencyCollector()
               .SetSampler(new AlwaysOnSampler());
            });

You should get up and running.

@heldergregorio For your case where you are getting assembly load errors, what version of .NET Core are you targeting? 2.0 or 2.1? You might be running into #605.

[heldergregorio]

@heldergregorio Could you elaborate on the versions (nuget packages) you were using when the problem occurred of all libraries involved?

I was using MyGet 0.2.0-alpha.246

@heldergregorio For your case where you are getting assembly load errors, what version of .NET Core are you targeting? 2.0 or 2.1? You might be running into #605 .

I'm targeting .NET Framework 4.7.2. I don't think it is related, I was trying to load a wrong version of System.Buffers (if I recall correctly). I was just trying to point out to a pattern that could be problematic and how I solved it

[longility]

So @heldergregorio does not have any issue. (thanks for providing details and making that point)

@macon @lingabalaji17 Can you provide details? What version are you running? I know when mine didn't work, I switch to the latest on myget and that works. I am assuming you were trying to use the packages from nuget.org.

@lingabalaji17 Can you also add .AddRequestCollector() to create the initial span when a request comes in? https://github.com/open-telemetry/opentelemetry-dotnet#configuration-with-microsoftextensionsdependencyinjection . If that still doesn't work, have you verified that your jaeger is working properly? I tested it with this: https://github.com/open-telemetry/opentelemetry-dotnet/blob/838d25f78b158c9aa40778fb4e28791fb89800ba/samples/Exporters/Console/TestJaeger.cs

[CodeBlanch]

After my comment yesterday @heldergregorio sent me a code snippet on gitter. It's just a snippet, so I can't say for certain, but based on the description I think the issue is he's trying to create the TracerFactory over and over again in Application_BeginRequest. I took the sample in the OT solution and switched it to MyGet + code form his snippet, seems to work fine: https://github.com/CodeBlanch/ot_jaeger/blob/master/AspNet/Global.asax.cs

The System.Buffers thing, who knows. Probably missing an assembly binding redirect in web.config? The VS tooling can be kind of touch-and-go with those. Maybe there's a message sitting in build output window saying that it's missing. The ASP.NET example in the OT solution, which uses Jaeger, does have this redirect (among others):

      <dependentAssembly>
        <assemblyIdentity name="System.Buffers" publicKeyToken="CC7B13FFCD2DDD51" culture="neutral"/>
        <bindingRedirect oldVersion="0.0.0.0-4.0.3.0" newVersion="4.0.3.0"/>
      </dependentAssembly>

[lingabalaji17]

So @heldergregorio does not have any issue. (thanks for providing details and making that point)

@macon @lingabalaji17 Can you provide details? What version are you running? I know when mine didn't work, I switch to the latest on myget and that works. I am assuming you were trying to use the packages from nuget.org.

@lingabalaji17 Can you also add .AddRequestCollector() to create the initial span when a request comes in? https://github.com/open-telemetry/opentelemetry-dotnet#configuration-with-microsoftextensionsdependencyinjection . If that still doesn't work, have you verified that your jaeger is working properly? I tested it with this: https://github.com/open-telemetry/opentelemetry-dotnet/blob/838d25f78b158c9aa40778fb4e28791fb89800ba/samples/Exporters/Console/TestJaeger.cs

you provide details? What version are you running? I know when mine didn't work, I switch to the latest on myget and that works. I am assuming you were trying to use the packages from nuget.org.

I am using ASP.net core 2.1 and I tried both against nuget and myget. with myget package and asp.net core 2.1, I got the issue as mentioned here :
https://github.com/open-telemetry/opentelemetry-dotnet/issues/605

[francisminu]

Hi
I am trying to find AddRequestCollector and AddDependenyCollector and is unable to locate the nuget. Could anyone please let me know which package this comes from?

[francisminu]

@MikeGoldsmith I have the below code:
services.AddOpenTelemetry((builder) =>
{
builder.UseJaeger((o) =>
{
o.AgentHost = "localhost";
o.AgentPort = 6831;
o.ServiceName = "SampleJaegerApplication";
})
.SetSampler(new AlwaysOnSampler());
});
The only difference I see from what you have mentioned is that it is missing .AddDependencyCollector()
I couldn't figure out the package that has this. Could someone help please?

[CodeBlanch]

Apologies @francisminu there was some renaming. "Collectors" were renamed "Adapters" so you'll want...

• OpenTelemetry.Adapter.AspNetCore or OpenTelemetry.Adapter.AspNet to get AddRequestAdapter.
• OpenTelemetry.Adapter.Dependencies to get AddDependencyAdapter.

I'm doing that from memory please reply back if you get it working or I was off about something.

The bad news is they have been renamed _again_ but the latest change is not available on NuGet yet. What we have at the moment is "Adapter" became "Instrumentation" everywhere.

I'm really sorry about this but we don't have much choice. These naming changes happen in the parent OT spec and we're just responding to stay in line with it.

[francisminu]

Thank you @CodeBlanch I will try and gte back to you. So all I need to do is add - AddDependencyAdapter and AddRequestAdapter just like it was when they were named collector?

[CodeBlanch]

@francisminu Correct new names, same behavior.

[CodeBlanch]

By the way, use OpenTelemetry.Extensions.Hosting package not the older OpenTelemetry.Hosting. @MikeGoldsmith @SergeyKanzhelev We should de-list/invalidate OpenTelemetry.Hosting on NuGet?

[francisminu]

Yes, I am using the Extensions.Hosting package. I haven't looked into the details. But straight out adding those 2 fields is giving me this error:

MissingMethodException: Method not found: 'OpenTelemetry.Trace.Configuration.TracerBuilder OpenTelemetry.Trace.Configuration.TracerBuilder.AddAdapter(System.Func`2)'.

This is what my code looks like:
services.AddOpenTelemetry((builder) =>
{
builder.UseJaeger((o) =>
{
o.AgentHost = "localhost";
o.AgentPort = 6831;
o.ServiceName = "SampleJaegerApplication";
})
//.AddRequestAdapter()
.AddDependencyAdapter()
.SetSampler(new AlwaysOnSampler());
});

[CodeBlanch]

@francisminu Try this...

csproj:

    <PackageReference Include="OpenTelemetry.Adapter.AspNetCore" Version="0.2.0-alpha.275" />
    <PackageReference Include="OpenTelemetry.Adapter.Dependencies" Version="0.2.0-alpha.275" />
    <PackageReference Include="OpenTelemetry.Exporter.Jaeger" Version="0.2.0-alpha.275" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="0.2.0-alpha.275" />

Startup.cs:

using OpenTelemetry.Trace.Configuration;
using OpenTelemetry.Trace.Samplers;

            services.AddOpenTelemetry((builder) =>
            {
                builder.UseJaeger((o) =>
                {
                    o.AgentHost = "localhost";
                    o.AgentPort = 6831;
                    o.ServiceName = "SampleJaegerApplication";
                })
                .AddRequestAdapter()
                .AddDependencyAdapter()
                .SetSampler(new AlwaysOnSampler());
            });

[francisminu]

Thank you so much @CodeBlanch - updating to version 275 of all packages seems to have fixed it. I can now see my application on the Jaeger UI. Thank you once again!

[CumpsD]

Where did OpenTelemetry.Adapter.AspNetCore go? Is it not needed anymore with 1.0.0-rc1.1?

[kirillta]

@CumpsD now it calling OpenTelemetry.Instrumentation.AspNetCore. The package name has been changed

[CumpsD]

@kirillta thanks!

[CumpsD]

Any chance you know where OpenTelemetry.Adapter.Dependencies has gone too?

[kirillta]

It was split to OpenTelemetry.Instrumentation.SqlClient, OpenTelemetry.Instrumentation.GrpcNetClient, and OpenTelemetry.Instrumentation.Http I believe