Spring-cloud-sleuth: OpenTracing AutoConfiguration crashes with a non-brave OT implementation

[realark]

OpenTracingAutoConfiguration creates a Brave Tracer if OpenTracing classes are on the classpath.

https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/instrument/opentracing/OpentracingAutoConfiguration.java#L40-L41

This crashes the app if a non-brave OT implementation is used. To Repro:

1. Launch SpringBoot2 with a Datadog JavaAgent with no Brave classes present: https://docs.datadoghq.com/tracing/setup/java/#installation-and-getting-started
2. App will crash on startup: java.lang.ClassNotFoundException: brave.opentracing.BraveTracer

As a workaround, the OT configuration can be disabled.

https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/instrument/opentracing/OpentracingAutoConfiguration.java#L39

Seems like a simple fix would be to add a check for a brave implementation to the OT autoconfiguration.

@ConditionalOnClass(Tracer.class, BraveTracer.clss)

Happy to open a PR if that helps.

[devinsba]

This won't actually work the way you are expecting it to. If you disable the BraveTracer and create an instance of some OtherTracer then you will end up with 2 different tracing systems; sleuth (built on brave) and your OT tracer. The OT support in Sleuth is explicitly to allow the use of OT instrumentation not to swap the tracer since sleuth is a tracer itself historically.

[erie149]

This bug that @realark submited was when Spring Cloud (specifically spring-cloud-zuul) Finchley SR1 and Spring Boot 2.0.4 were used. We would love to exclude all of Spring Sleuth and not even bring in the libraries but compilation will not work then. The only way this works with datadog APM is to disable sleuth (spring.sleuth.enabled=false)

[codefromthecrypt]

I'm guessing this is because datadog APM is affecting the classpath of the VM by sharing a type that might be present in the application classpath. This will also be a problem with opentracing 0.32 is release, as well 0.33 etc.

I would recommend asking DD APM to hide their types used internally so that they won't conflict in situations such as this or other unrelated opentracing upgrades. I suspect most APM agents will not have this problem because they don't leak usage of opentracing apis into user space. In fact I know of no other apm which uses OT internally. For example, I'm quite sure instana agent supports multiple opentracing api versions.

TL;DR; sooner or later datadog will sort this out.. mind asking them to start now? The isolation will solve other problems that a "spot fix" here won't

[codefromthecrypt]

other thing.. can you confirm that setting "spring.sleuth.opentracing.enabled" to false isn't sufficient?

[codefromthecrypt]

actually disregard my question as it looks like from the original description, you only need to set "spring.sleuth.opentracing.enabled=false" not disable all of sleuth. I was distracted by the assumption you have to disable sleuth entirely which seems a bit off.

For those wondering why we shouldn't do @ConditionalOnClass(Tracer.class, BraveTracer.class)... Sleuth itself, its support of opentracing is indeed dependent on BraveTracer. The classnotfoundexception explains what is wrong, which is that the implementation is not available.

If we step away from whether or not DD agent should be leaking opentracing classes, we can get closer to what we could or should do here. Let's assume this wasn't about a redundant tracing setup.. that's the distraction.

Let's assume someone put in some code of their own, and wondered why it wasn't working? With the current setup, it would crash, which is probably a good thing as it is relatively easy to find the missing dependency as per our guide.

Let's say instead, we just started up anyway. For example, we did the @ConditionalOnClass(Tracer.class, BraveTracer.class) bit.. that silently failed and then didn't configure the tracer etc. There's no eager crash, people will wonder why their data is missing and probably open issues or start a wandering discussion on support. This falls onto volunteers to clean up.

That's why I'm personally ok with how it is now.. in the rare case you have an agent which activates the classpath and is redundantly setup with sleuth, and that results in the crash.. disable via "spring.sleuth.opentracing.enabled=false" (or rewrite an exception message to say something more clearly.. like "did you mean to include XXX dependency?").

[realark]

@adriancole Thanks for the explanation. That makes sense. Usually better to crash early than behave strangely later.

In terms of putting OT on the classpath, that's an early design decision of the dd tracer. We're working to decouple the dd tracer from OT so we don't have to inject OT classes for javaagent users.

[nnanda2016]

I am facing this same issue and Adrian's suggested flag spring.sleuth.opentracing.enabled=false helped. In my case it appeared by changing Spring Boot and Sleuth release train versions.

1. I was running my app on Spring Boot 2.0.3 and Sleuth Finchley.RC1. The app was working fine.
2. I updated the versions to Spring Boot 2.0.4 and Sleuth Finchley.SR1. My app broke for a different issue - this version of Sleuth and Couchbase Java SDK (2.5.9+) do not like each other.
3. Then I reverted my app to use Sleuth Finchley.RC1, but retained Spring Boot version as 2.0.4. This failed with the issue that is discussed in this ticket.

In all above scenarios, I was using Reactor Core and Reactor Extra 3.2.0.RELEASE.

[codefromthecrypt]

yes it is interesting that eventhough the version is 0.31 (not a final
version and different than what final would be) is a strict dependency
of couchbase!

https://github.com/couchbase/couchbase-jvm-core/blob/master/pom.xml#L331
On Thu, Sep 27, 2018 at 7:03 PM Niranjan [e-mail] wrote:
>

I am facing this same issue and Adrian's suggested flag spring.sleuth.opentracing.enabled=false helped. In my case it appeared by changing Spring Boot and Sleuth release train versions.

I was running my app on Spring Boot 2.0.3 and Sleuth Finchley.RC1. The app was working fine.
I updated the versions to Spring Boot 2.0.4 and Sleuth Finchley.SR1. My app broke for a different issue - this version of Sleuth and Couchbase Java SDK (2.5.9+`) do not like each other.
Then I reverted my app to Sleuth Finchley.RC1, but retained Spring Boot version as 2.0.4. This failed with the issue that discussed in this ticket.

In all above scenarios, I was using Reactor Core and Reactor Extra 3.2.0.RELEASE.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.