Zipkin: upgrade to Spring Boot 2.0 NoClassDefFoundError UndertowEmbeddedServletContainerFactory

[JaxXu]

UndertowEmbeddedServletContainerFactory has been renamed in Spring Boot 2.0. It's now UndertowServletWebServerFactory.

[adriancole]

When we update to spring boot 2, it will be the only version we support. So
basically we just change everything to work with it.

Probably will do it soon. cc @shakuzen

[adriancole]

@llinder @devinsba do you have any concerns with the next zipkin minor
moving to spring boot 2? (implies this also happens to zipkin-gcp -aws etc)

I know we are missing the layout factory change..

[rahul-pharande]

@adriancole is there tentative date when we have spring boot 2.0 support available here.

[adriancole]

only recently we are getting interest. there is a patch so I am personally
more comfortable. I would like an answer from the others before proceeding.

We still need https://github.com/openzipkin/zipkin-layout-factory

I'm curious.. since zipkin-server is packaged as an all-in-one exec jar,
and we don't support custom servers. Why do you care what version of spring
boot it uses underneath?

Is there some feature you are looking for?

[JaxXu]

Because we recently used SpringCloud 2.0.0 and found this problem, there is a problem with the startup dependencies by Spring, or can there be other solutions?

[JaxXu]

Thanks for your reply, let's try it without Spring Boot

[bodiam]

Hi, just wanted to let you know I'm running into the same problem. All our applications are running on Spring Boot 2.0, which is why are running into this. I'll downgrade to 1.x for now.

[JaxXu]

There are more bugs, waiting for the official version to be used again @bodiam

[shakuzen]

All our applications are running on Spring Boot 2.0, which is why are running into this. I'll downgrade to 1.x for now.

Everyone please note that your applications can run on Spring Boot 2 and still interact with the current version of the Zipkin server just fine.
We don't support custom builds of the Zipkin server, which is what I assume is being done here. It would be easiest and more supportable to use the Quick Start instructions for running the Zipkin Server: https://github.com/openzipkin/zipkin#quick-start
If there are reasons you need a custom build, please share them with us on Gitter https://gitter.im/openzipkin/zipkin.

[bodiam]

Hi @shakuzen , sorry for the confusion. What I was trying to do is the following:

@SpringBootApplication
@EnableZipkinStreamServer
class InfrazipkinApplication

fun main(args: Array<String>) {
    runApplication<InfrazipkinApplication>(*args)
}

This is a Spring Boot 2.0 application.

The @EnableZipkinStreamServer doesn't seem to work, because of the missing class, which is also reported here: https://stackoverflow.com/questions/45581389/where-is-undertowembeddedservletcontainerfactory-in-spring-boot-2-0-0

[adriancole]

We don't support custom servers. Shortly we'll put a pull request to
deprecate that annotation.

[bodiam]

Hi @adriancole , ah, that's a shame to hear. My organisation is quite good at deploying Spring (Boot) services, but less proficient at deploying anything different than that. What kind of deployment for the zipkin server would you recommend in such a case, if the Spring Boot annotation will be deprecated?

[adriancole]

>

Hi @adriancole https://github.com/adriancole , ah, that's a shame to
hear. My organisation is quite good at deploying Spring (Boot) services,
but less proficient at deploying anything different than that. What kind of
deployment for the zipkin server would you recommend in such a case, if the
Spring Boot annotation will be deprecated?

technically, how do you deploy your server? At the end of the day you are
creating a jar right? How would it be different to deploy a jar you didn't
make vs one you did make?
https://search.maven.org/remote_content?g=io.zipkin.java&a=zipkin-server&v=LATEST&c=exec

The deprecation note will mention that this annotation will be "use at your
own risk". Basically the problem you are running into now could be in
reverse. For example, what if we moved to spring boot 2 ahead of your
schedule :) It is only by accident it is this way. So, the text in the
description will alert users of potential problems such you are facing, and
more importantly to not expect to come onto our support channel about
classpath conflicts which usually take hours to unwind and have very little
to do with zipkin. This literally prevents the project from moving forward
when we have to take hours we don't have to help people with IDE setup.
This is not about "preventing" a custom server, it is about highly
discouraging it and making expectations crystal clear. Make sense?

[bodiam]

@adriancole Technically, it might sound the same, but it's not. We don't deploy jars, wars, we commit source code, and our build pipeline triggers everything. So, having an Annotated Spring Boot app, pushing that to our git repo, will trigger a build, deploy, etc. For a jar, it's very different. We don't put them in source control, there's no source to commit, etc. I'm not saying it's ideal, but I don't think it's unique either, and it's not something I can change overnight. Rolling out new infrastructure, like Eureke, Zipkind, Zuul, etc, is far more complex than rolling out a new Microservice.

Could you explain me why this dependency on a particular Spring version exists? Afaik, something like @EnableEurekaServer works on Spring Boot 1.x and Spring Boot 2.x. Would it be hard to support both versions?

[shakuzen]

Could you explain me why this dependency on a particular Spring version exists?

The Zipkin Server is built using Spring Boot. It uses some Spring Boot classes that were changed in a backwards-incompatible way from 1 → 2, as can be expected in a major version.

Afaik, something like @EnableEurekaServer works on Spring Boot 1.x and Spring Boot 2.x.

It's possible some annotation could work fine across major versions of a dependency if underneath nothing depended on a class that changed in a breaking way. However, this would be somewhat by chance. That is to say, Spring Cloud's release train Finchley supports Spring Boot 2 and does not support Spring Boot 1. Release trains prior to Finchley support Spring Boot 1 and not Spring Boot 2. Something that works with both does so without any guarantee from the Spring Cloud team.

While in some cases it's _technically_ possible (and feasible) to support multiple major versions of libraries (e.g. Hibernate 4 and 5) at the same time using things like reflection to check for available classes/methods, in most cases a given version of a library will support one set of major (and possibly minor) version of each dependency due to maintenance costs. This is particularly true for something as core as a framework dependency. Look at the example in this issue: the class in Spring Boot 2 that needs to be used doesn't exist in Spring Boot 1 and vice versa.

Would it be hard to support both versions?

Yes, it is a major version release that made breaking changes. I doubt there are many libraries simultaneously supporting Spring Boot 1 and 2 in the same version.

@bodiam
That all being said, even given your preference to push source code, you don't need to use Spring Boot 2 right now. Your applications can run on Spring Boot 1 or 2 and still send spans to Zipkin whether it is running on Spring Boot 1 or 2 (with the version that supports each, of course). These are, by design, independent choices. If you want to maintain source code with @EnableZipkinServer that works with Spring Boot 1, then you can still deploy a Zipkin Server in your preferred manner. When Zipkin switches to Spring Boot 2, you will have to switch your custom Zipkin Server to Spring Boot 2 when upgrading to that version of Zipkin or later.

A custom server can work and might be the right solution for you, but we don't have enough resources to troubleshoot all the many things that can go wrong when making a custom Zipkin Server. Especially when we expect the packaged Zipkin Server to meet the majority of people's needs. If there is a need it doesn't fulfill, we would love to hear about that (thank you for sharing your situation about your deployment pipeline/process). Hence the short answer is: right now Zipkin is only compatible with Spring Boot 1. When it changes to Spring Boot 2, it will only run with that. This is all versioned, of course, so if you must make a custom server, you will have to use the version that works with the version of Spring Boot you choose to run. The same way you have to pick the compatible version of Spring Cloud or any other library that builds on Spring Boot.

---

@bodiam
I just noticed that you mentioned @EnableZipkinStreamServer. The stream server was deprecated and has been removed from Sleuth in 2.0 (Finchley release train) as mentioned in the migration guide and in the documentation (1.3.x and 2.0.x). As mentioned there, a vanilla Zipkin server and recent versions of Sleuth work fine sending and receiving spans over Kafka, RabbitMQ, or HTTP.

[bodiam]

@shakuzen Thanks for the amazing writeup. Very clear! (A small note though: it's not that I _want_ to deploy source code per se this way, but it's a big organisation, and it can't support all possible deployment types, and the current approach, while maybe limited for some usecases, works well for the services we're building.

Thanks for the note on @EnableZipkinStreamServer btw, I'll change that to @EnableZipkinServer, and more than happy to run it on Spring Boot 1.x.

[adriancole]

Tommy is awesome, indeed!

Erik, by your response you might have missed that EnableZipkinStreamServer
isnt just 1.x pinned, but it is dead (not moving to 2.x) and never lived
here in this repo anyway (lived in sleuth). Not trying to restart a
conversation circle, just highlighting in case that wasnt a typo.

The specific annotation here is EnableZipkinServer, the one we are flashing
warnings about. We will keep this annotation for the foreseeable future as
well test (via invoker tests) like we have been.. just we are being more
explicit about this being use at your own risk.

[bodiam]

@adriancole Ah, thanks for pointing that out, it was a typo indeed! I've updated my comment!

[adriancole]

https://github.com/openzipkin/zipkin/pull/1968 will deprecate the annotation and hopefully shorten some of these conversations in the future

[robachmann]

My apologies for gatecrashing but just to be sure: Zipkin does and will keep the support of receiving Spans through RabbitMQ, right?
I'm in the exact same situation as @shakuzen - even having the same arguments - but I'm willing to use the provided jar even though then I won't be able to enhance the server with additional code for e.g. metrics collection.

[adriancole]

Hi, Roman.

The reason we are pessimistic about features is that we try our best to
keep them indefinitely. For example, we spent inordinate time trying to
keep some continuity in things we documented moving from Boot 1.5 to 2.0.
Supporting something is different than something being possible. Things we
don't support mean "you are on your own" Let's contrast this with RabbitMQ
which is in the main repo, has integration tests, and has corresponding
code on some of the tracers. This is a supported feature and to the best of
our ability we'll keep it working.

Hope this answers you!

[robachmann]

I appreciate your quick response, Adrian, I see that you give your best and I'm thankful for your hard word you invested in this project. It's just that I tried integrating Zipkin into our own architecture and thought sending spans asynchronously through RabbitMQ is a wonderful feature. When I now run zipkin-server-2.8.4-exec.jar, bind it to an elasticsearch- & rabbitmq-service and add the spring-cloud-starter-zipkin dependency to an existing Boot 2.0 app, I would assume it to work out-of-the-box (with sleuth.sampler.probability: 1.0) because the consumer's (Zipkin) framework (based on SB 1.5) shouldn't affect the producer's (based on SB 2.0) and vice-versa because of RabbitMQ as a decoupling-layer. Unfortunately this isn't the case as I don't even see a queue getting created in RabbitMQ-mgmt-ui as I would assume Zipkin would do during startup in order to listen to it and consume incoming messages. So before wasting your time investigating in a different issue, I wanted to clarify whether this use-case is intended to work and if so, what to configure. From a developer's perspective, sending spans asynchronously makes more sense than using a synchronous HTTP interface.

[adriancole]

I think you should probably use gitter for conversations like this.

Indeed "zipkin" setup for rabbit (not spring cloud stream) works between
1.5 apps using sleuth and various versions of zipkin.

Our server integration tests only look at the rabbit queue and payload and
these haven't changed between recent versions of zipkin.

tldr probably there is something subtle going on if you are using the
standard zipkin distribution and rabbit acts differently between minor
versions of the server. If we had more but reports I would guess maybe
server is problem but I think you should maybe vet the issue first. then
follow up on gitter vs this issue ok?

[robachmann]

Agreed, will do and thank you!

[Sinchana94]

The client application with spring boot version 2.0 .x, is configured with sleuth-zipkin and when http request made it's logging the trace id in console, but same trace id in zipkin server is not found. Any suggestions ?

[shakuzen]

The client application with spring boot version 2.0 .x, is configured with sleuth-zipkin and when http request made it's logging the trace id in console, but same trace id in zipkin server is not found. Any suggestions ?

@Sinchana94 This question is unrelated to this GitHub issue. Please use the Spring Cloud Sleuth chat on Gitter at https://gitter.im/spring-cloud/spring-cloud-sleuth. Probably the trace is not sampled and so it is not reported to Zipkin.

[Sinchana94]

The client application with spring boot version 2.0 .x, is configured with sleuth-zipkin and when http request made it's logging the trace id in console, but same trace id in zipkin server is not found. Any suggestions ?

@Sinchana94 This question is unrelated to this GitHub issue. Please use the Spring Cloud Sleuth chat on Gitter at https://gitter.im/spring-cloud/spring-cloud-sleuth. Probably the trace is not sampled and so it is not reported to Zipkin.

Thanks @shakuzen will post the query there, However the exportable field is true in log