Sentry-java: HTTP request body not being reported for application/json encodings

[dlopuch]

If your HTTP request contains a json-encoded body (eg POST requests on modern API's), the body doesn't show up in sentry.

The lib extracts the body using ServletRequest.getParameterMap(): https://github.com/getsentry/raven-java/blob/v7.8.3/raven/src/main/java/com/getsentry/raven/event/interfaces/HttpInterface.java#L55

This works fine for form url encodings but returns nothing for JSON encodings. eg http://stackoverflow.com/questions/3831680/httpservletrequest-get-json-post-data

The simplest way to get the raw body text is something like servletRequest.getReader().lines().collect(Collectors.joining()) (but getReader() has limitations that it can only be called once, so sentry will have to be clever not to hurt any downstream code). Ideally this could be sent up to sentry.

[bretthoerner]

We discussed this previously here: https://github.com/getsentry/raven-java/issues/223#issuecomment-232905105

The fact that getReader() can only be called once seemingly kills any chance for Sentry to do anything automatically. Do you have any ideas for getting around that? I think that users will have to set the body manually... 😬

[bretthoerner]

This also comes up some here: https://github.com/getsentry/raven-java/issues/273

[dlopuch]

Yes, there are tons of stackoverflow's asking for workarounds to the fact that an HttpServletRequest's input can be read only once. Someone over at sun once long ago probably had a good reason for it, but today as soon as you add a bunch of logging modules upstream from something like jackson that converts your json into java, it becomes a major pia.

My solution has been to write a request filter that replaces the HttpServletRequest with a custom HttpServletRequestWrapper which caches the body and lets getInputStream() work multiple times for any downstream plugin to not have to worry about what happened before. Basically http://www.myjavarecipes.com/tag/how-to-read-request-twice/. Spring users can easily adopt that for a spring filter bean.

With that caching filter in place, workaround is I just dump the body into the MDC where sentry auto picks it up. An API like in #273 would let me directly inject it.

---

I think sentry automatically replacing the HttpServletRequest with a request cache wrapper is kinda dangerous (lots of concerns around out-of-scope of sentry, eg request size limits, what if someone is already doing their own cache, etc.), but given these limitation of j2ee, I think it would be awesome if you guys at least provided a default implementation that acts as an out-of-the-box opt-in-if-you-accept-the-risks.

273's API would help a lot, as well as updating the docs to make it clear that only form-encoded request payloads are included by default (to include application/json, opt-in to this this silly request-caching filter kludge).

[bretthoerner]

I agree about having an opt-in solution, I'll check it out. I did the start of #273 (making a field available so it's easier to set a body -- you'd still have to use your own RavenFactory and subclass the HttpEventBuilderHelper for now) if you want to check it out: #334

[bretthoerner]

A String body field was added in 7.8.5. This doesn't automate anything, but it should reduce the amount of customization required for now: https://github.com/getsentry/raven-java/commit/10ac4a5a6959ccd2c23a239a9646297477dd2334

[dineshsriram]

If this were to be done automagically by Sentry, that would be great.
Are there any plans to have a custom filter integrated as @dlopuch suggested?

[tienhm0202]

@bretthoerner Can you please provide some example how to set request body manual? Can I define it in ServletContextInitializer as a Spring bean? Or I have to set body every time error happen?

[charlie-world]

Is it fixed completely? i can't see body data on sentry error event now with sentry-java 1.7.23

[arpanagr]

Is this fixed or is there any workaround if I am using spring-sentry?

[kaixin0571]

I hava fixed this issue using spring-boot
step1:

public class HttpEventBuilderHelperWithBody extends HttpEventBuilderHelper {

    @Override
    public void helpBuildingEvent(EventBuilder eventBuilder) {
        ServletRequestAttributes requestAttributes = (ServletRequestAttributes)RequestContextHolder.getRequestAttributes();
        ContentCachingRequestWrapper wrapper =
                WebUtils.getNativeRequest(requestAttributes.getRequest(), ContentCachingRequestWrapper.class);
        String body = null;
        if (wrapper != null) {
            byte[] buf = wrapper.getContentAsByteArray();
            if (buf.length > 0) {
                try {
                    body = new String(buf,0,buf.length,wrapper.getCharacterEncoding());
                } catch (UnsupportedEncodingException e) {
                }
            }
        }
        eventBuilder.withSentryInterface(new HttpInterface(requestAttributes.getRequest(),super.getRemoteAddressResolver(),body),false);
        addUserInterface(eventBuilder,requestAttributes.getRequest());
    }

    private void addUserInterface(EventBuilder eventBuilder, HttpServletRequest servletRequest) {
        String username = null;
        if (servletRequest.getUserPrincipal() != null) {
            username = servletRequest.getUserPrincipal().getName();
        }

        UserInterface userInterface = new UserInterface(null, username,
                super.getRemoteAddressResolver().getRemoteAddress(servletRequest), null);
        eventBuilder.withSentryInterface(userInterface, false);
    }
}

step2:

public class SentryClientCustomFactory extends DefaultSentryClientFactory {

    @Override
    public SentryClient createSentryClient(Dsn dsn) {

        try {
            SentryClient sentryClient = new SentryClient(createConnection(dsn), getContextManager(dsn));
            try {
                // `ServletRequestListener` was added in the Servlet 2.4 API, and
                // is used as part of the `HttpEventBuilderHelper`, see:
                // https://tomcat.apache.org/tomcat-5.5-doc/servletapi/
                Class.forName("javax.servlet.ServletRequestListener", false, this.getClass().getClassLoader());
                                // sentryClient.addBuilderHelper(new HttpEventBuilderHelper());
                sentryClient.addBuilderHelper(new HttpEventBuilderHelperWithBody());
            } catch (ClassNotFoundException e) {
                log.debug("The current environment doesn't provide access to servlets,"
                        + " or provides an unsupported version.");
            }

            sentryClient.addBuilderHelper(new ContextBuilderHelper(sentryClient));
            return configureSentryClient(sentryClient, dsn);
        } catch (Exception e) {
            log.error("Failed to initialize sentry, falling back to no-op client", e);
            return new SentryClient(new NoopConnection(), new ThreadLocalContextManager());
        }
    }

}