Spring-cloud-sleuth: Support AWS X-Ray

[jramsdale]

Much like Zipkin, Amazon now provides AWS X-Ray, their hosted tracing service: https://aws.amazon.com/xray/

Adding X-Ray support to Spring Cloud Sleuth would be a nice addition for users preferring to use AWS managed services.

[codefromthecrypt]

Here are some links of interest

description of data that can be sent: http://docs.aws.amazon.com/xray/latest/devguide/xray-api.html#xray-api-segments
the Api model: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-models/src/main/resources/models/xray-2016-04-12-model.json
doesn't seem the servlet filter mentioned in docs is on github https://github.com/search?utf8=%E2%9C%93&q=AWSXRayServletFilter
here's an example of using their sdk http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-subsegments.html

Here's some notes @ryangardner toook at reinvent

A few tidbits from talking to the amazon people about their implementation. One reason they went with their own stuff was because they felt they needed to put the timing stuff in the header to make it easier to scale it up
They shuttle the data over UDP to a daemon that runs on localhost and that daemon sends it to the collector. It’s a well-known endpoint that everyone in AWS will share for a given region. The data you see is based on the account you send the data in from. If you want to trace data across multiple aws accounts you have to use a cross-account linking and assuming the role of the central “x-ray” account for your org. In the future they may be able to make this easier by leveraging metadata made available by the “orgs” feature that amazon just announced

[codefromthecrypt]

so I think next step would be either get a hold of the instrumentation SDK code and/or a spec for how headers propagate, etc. Particularly, if this is related to and/or compatible with the recent announcement on ALB, which seems to imply "cookie crub" trace id header. Ex ids are tacked on.

[nicmunroe]

I took a quick look at this yesterday for Wingtips. The docs are sort of broken with how to pull in libs, where classes are, etc. But I did find the AWSXRayServletFilter class. It's in this dependency: com.amazonaws:aws-xray-recorder-sdk-core:1.0.0-beta. The source jar had no java code in it though, so it's basically fun with decompiled code.

In particular, a bunch of info is propagated with a X-Amzn-Trace-Id header. I think trace ID, "segment" (span) ID, parent ID, and sampled are embedded in that one header value.

There also looks to be several magic attributes you can add to the segment/span tags/metadata - I'm assuming the AWS tools will do useful things with those.

I'm not sure what a meaningful integration or support of XRay looks like in relation to other instrumentation libraries like spring sleuth/zipkin/etc or what it accomplishes. Feels like there should be some good use cases here somewhere, just not sure what they are yet.

[davidnortonjr]

perhaps an alternative to the X-Ray daemon would be a Zipkin-compatible service, that translates the Zipkin traces and publishes to X-Ray using the AWS SDK? Similar in concept to Stackdriver Trace's Zipkin integration:

https://github.com/GoogleCloudPlatform/stackdriver-zipkin

This would allow AWS storage/retrieval of trace data while using the Zipkin header standards and the superior Zipkin client libraries - a big problem with X-Ray currently is that it only supports a few frameworks on a few platforms.

[codefromthecrypt]

ps detailed docs were just published about X-Ray's out-of-band format http://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html

[hakanson]

FYI - until they release the Java source, you can npm install aws-xray-sdk and get the JavaScript version of the source code to look through. I had been digging though that code when I was porting a sample app from zipkin-js to x-ray and trying to wrap/patch node-fetch (see https://forums.aws.amazon.com/thread.jspa?threadID=247252&tstart=0).

[codefromthecrypt]

keep the feedback coming! if you or any others end up making a sample app
that works, do ping back a gist or otherwise link.

[chenrui333]

+1

[howardem]

+1

[metaruslan]

Hi guys, did you have any luck in getting the java source code from x-ray? I did ask them and waiting now
https://forums.aws.amazon.com/thread.jspa?threadID=260671&tstart=0

[andrewpowell]

This may be of use: https://aws.amazon.com/blogs/devops/aspect-oriented-programming-for-aws-x-ray-using-spring/

[GrahamLea]

@metaruslan:
Hi guys, did you have any luck in getting the java source code from x-ray? I did ask them and waiting now
https://forums.aws.amazon.com/thread.jspa?threadID=260671&tstart=0

I know this issue is superseded by the Brave re-write, but just in case someone ends up here from Google like I did, and is wondering if AWS's Java SDK for X-Ray ever did go open source, it's here: https://github.com/aws/aws-xray-sdk-java

[mteodori]

is there a test or an example with a configuration of sleuth with brave sending to xray?

[GrahamLea]

Here's the main config that I used in the @SpringBootApplication of a test (Kotlin) app:

    @Bean
    fun tracing(@Value("\${spring.application.name:spring-tracing}") serviceName: String, spanReporter: Reporter<Span>): Tracing {
        return Tracing.newBuilder()
            .localServiceName(serviceName)
            .sampler(Sampler.ALWAYS_SAMPLE)
            .spanReporter(XRayUDPReporter.create())
            .traceId128Bit(true)
            .build()
    }

Plus these dependencies:

org.springframework.cloud:spring-cloud-starter-zipkin 
io.zipkin.aws:zipkin-reporter-xray-udp

[codefromthecrypt]

cheers.. it could maybe look similar to this also (not tried in a while) https://github.com/openzipkin/sleuth-webmvc-example/compare/xray