Spring-cloud-netflix: Zuul returning 400 bad request when changing body with HttpServletRequestWrapper

[eutiagocosta]

Personal, the zuul receives a request with the encrypted body, decrypts the body and passes the request to the internal services with the payload open.

It does the right job, however I have identified a bug, the first call works, the second gives 400 error, the third one works, the fourth gives 400, and so on, that is, one request works, the other does not.

public class SimpleFilter extends ZuulFilter {

    private static Logger log = LoggerFactory.getLogger(SimpleFilter.class);
    private UrlPathHelper urlPathHelper = new UrlPathHelper();

    @Override
    public String filterType() {
        return "pre";
    }

    @Override
    public int filterOrder() {
        return 1;
    }

    @Override
    public boolean shouldFilter() {
        return true;
    }

    @Override
    public Object run() {

        RequestContext ctx = RequestContext.getCurrentContext();
        HttpServletRequest request = ctx.getRequest();
        final String requestURI = this.urlPathHelper.getPathWithinApplication(ctx.getRequest());

        if (RequestMethod.valueOf(request.getMethod()) == RequestMethod.POST)
            ctx.setRequest(DecryptionFilter.getInstance().parse(request));

        return null;
    }
}

public class DecryptionFilter {

    private static DecryptionFilter instance;
    private JsonObject json;
    private String body;

    public HttpServletRequest parse(HttpServletRequest request) {

        try {
            InputStream is = request.getInputStream();
            this.body = IOUtils.toString(is);
        } catch (IOException e) {
            e.printStackTrace();
        }

        this.json = new JsonParser().parse(body).getAsJsonObject();

        try {
            this.body = Cryptography.decrypt(new Param(
                    json.get("param").getAsString(), 
                    json.get("param1").getAsString()), "src/main/resources/private.pem");
        } catch (Exception e) {
            throw new RuntimeException();
        }

        return modifyRequest(request, body);
    }

    private static HttpServletRequestWrapper modifyRequest(HttpServletRequest request, String body) {
        System.out.println("body: " + body);
        return new HttpServletRequestWrapper(request) {

            @Override
            public ServletInputStream getInputStream() {
                return new ServletInputStreamWrapper(body.getBytes());
            }

        };
    }

    public static DecryptionFilter getInstance() {
        if (instance == null)
            instance = new DecryptionFilter();
        return instance;
    }

}

buildscript {
    ext {
        springBootVersion = '1.5.6.RELEASE'
    }
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
        classpath "se.transmode.gradle:gradle-docker:1.2"
        classpath('io.spring.gradle:dependency-management-plugin:0.5.4.RELEASE')
    }
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'docker'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

jar {
    baseName = 'app'
    version = '0.0.1-SNAPSHOT'
    archiveName = 'app.jar'
}

ext {
    springCloudVersion = 'Dalston.SR2'
}

dependencies {
    compile('org.springframework.cloud:spring-cloud-starter-zuul')
    compile('org.springframework.boot:spring-boot-starter-web')
    compile group: 'com.google.code.gson', name: 'gson', version: '2.8.0'
    compile('org.springframework.cloud:spring-cloud-starter-config')
    compile('org.springframework.cloud:spring-cloud-starter-bus-amqp')
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

dependencyManagement {
    imports {
        mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
    }
}

[ryanjbaxter]

Rather than put the code in the issue, could you perhaps put the code in a GitHub repo and provide us with instructions on how to reproduce the problem using that repo?

[eutiagocosta]

@ryanjbaxter https://github.com/eutiagocosta/ZuulApiGateway

[ryanjbaxter]

The README isnt very detailed on how to reproduce the problem, can you provide more detailed instructions?

[eutiagocosta]

@ryanjbaxter updated

[ryanjbaxter]

What does this mean

You need to create an endpoint that will reply to http://localhost:8010/card-store because cros-domain problem

Do I need to create my own app that is going to respond to POST requests running at http://localhost:8010/card-store?

[eutiagocosta]

Yes, a simple endpoint at this address that answers httpstatus 200 OK

[ryanjbaxter]

To eliminate any potential issues could you create the app and put it in the repo? I don't want to do something differently than what you are doing and waste time trying to recreate the issue.

[eutiagocosta]

Done -> https://github.com/eutiagocosta/CardDemo
I added more informations in https://github.com/eutiagocosta/ZuulApiGateway

[ryanjbaxter]

Thanks.

I haven't found the root of the problem yet, something tells me it is in you Zuul filter, but here is what I have found.

When you get a 200 back the request coming into the Card Demo app looks like

2017-08-21 15:55:53.248 DEBUG 66360 --- [nio-8010-exec-3] o.a.coyote.http11.Http11InputBuffer      : Received [POST /card-store/ HTTP/1.1
content-type: application/json; charset=utf-8
user-agent: Paw/3.1.3 (Macintosh; OS X/10.12.6) GCDHTTPRequest
x-forwarded-host: localhost:8080
x-forwarded-proto: http
x-forwarded-prefix: /api/wallet/card-manager
x-forwarded-port: 8080
x-forwarded-for: 127.0.0.1
Accept-Encoding: gzip
Content-Length: 666
Host: localhost:8010
Connection: Keep-Alive

{"card":{"cvv":"123","expMonth":12,"expYear":2031,"favorite":false,"holderName":"AAAAAA","isSelected":false,"cardNumber":"1111111111111117"},"document":"12345678909","email":"[email protected]"}]

When you get 400 the request looks like

2017-08-21 15:56:01.803 DEBUG 66360 --- [nio-8010-exec-3] o.a.coyote.http11.Http11InputBuffer      : Received [derName":"AAAAAA","isSelected":false,"cardNumber":"1111111111111117"},"document":"12345678909","email":"[email protected]"}]

Which results in

java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
    at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:422) ~[tomcat-embed-core-8.5.16.jar:8.5.16]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:683) ~[tomcat-embed-core-8.5.16.jar:8.5.16]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.16.jar:8.5.16]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-8.5.16.jar:8.5.16]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) [tomcat-embed-core-8.5.16.jar:8.5.16]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.16.jar:8.5.16]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.16.jar:8.5.16]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]

because it is completely malformed.

[ryanjbaxter]

Here is a good example to base your filter on https://github.com/spring-cloud-samples/sample-zuul-filters/blob/master/src/main/java/org/springframework/cloud/samplezuulfilters/UppercaseRequestEntityFilter.java

I modified the code as follows and it works now

    public Object run() {

        RequestContext ctx = RequestContext.getCurrentContext();
        HttpServletRequest request = ctx.getRequest();

        if (RequestMethod.valueOf(request.getMethod()) == RequestMethod.POST)
            try {
                ctx.set("requestEntity", parse(request));
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }

        log.info(String.format("%s request to %s ", request.getMethod(), request.getRequestURL().toString()));

        return null;
    }

    public ByteArrayInputStream parse(HttpServletRequest request) throws UnsupportedEncodingException {
        String body = null;
        try {
            InputStream is = request.getInputStream();
            body = IOUtils.toString(is);
        } catch (IOException e) {
            e.printStackTrace();
        }

        JsonObject json = new JsonParser().parse(body).getAsJsonObject();

        try {
            body = Cryptography.decrypt(
                    new Param(json.get("param").getAsString(),
                            json.get("param1").getAsString()), "src/main/resources/private.pem");
        } catch (Exception e) {
            ZuulException zuulException = new ZuulException("Erro ao decriptar o payload", 400,
                    "Erro ao decriptar o payload");
            throw new ZuulRuntimeException(zuulException);
        }

        return new ByteArrayInputStream(body.getBytes("UTF-8"));
    }

[denisoliveira]

@ryanjbaxter This is a good example but another right way using the HtttpServletRequestWrapper as @eutiagocosta used on your sample as follows.

 private static HttpServletRequestWrapper modifyRequest(HttpServletRequest request, String body) {

    HttpServletRequestWrapper wrapper = new HttpServletRequestWrapper(request) {
        @Override
        public byte[] getContentData() {            
            byte[] data = null;
            try {
                data = body.getBytes("UTF-8");
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }
            return data;
        }

        @Override
        public int getContentLength() {
            return body.getBytes().length;
        }

        @Override
        public long getContentLengthLong() {
            return body.getBytes().length;
        }

        @Override
        public BufferedReader getReader() throws IOException {
            return new BufferedReader(new InputStreamReader(new ByteArrayInputStream(body.getBytes("UTF-8"))));
        }

        @Override
        public ServletInputStream getInputStream() throws UnsupportedEncodingException {
            return new ServletInputStreamWrapper(body.getBytes("UTF-8"));
        }
    };

    return wrapper;
}

@eutiagocosta the issue with HttpServletRequestWrapper occurs when you change request content length. In other words, when you change the request content you need rewrite the other descriptors as content length and other reader methods from your request.

[rashad-farajullayev]

@daniellavoie , Thank you very much.
You made my day :)
It solved my problem.

[RohitDevSwami]

@rashad-farajullayev is it working for you, because i am getting the same request body which i am sending from postman, not in UPPER case. will you attach your project.
Thank you in advance

[RohitDevSwami]

@denisoliveira i tried your way but not getting expected body but, i am getting the same request body which i am sending from postman, not in UPPER case. will you attach your project.

[rashad-farajullayev]

I am not eligible to attach my project, sorry.
But the wrapper solved the problem. On the whole I came out with conclusion if you override the HttpSeretResponse it will change the transferred data in “pre” filter completely. Simply don’t forget to override the getContentLength and getContentLengthLong functions to transfer the correct length of the new data.

[RohitDevSwami]

@rashad-farajullayev I am overriding the getContentLength and getContentLengthLong as shown in above code but how to override HttpSeretResponse ? and what implementation should i do for HttpSeretResponse. could you attach a sample code that also will be very usefull

[rashad-farajullayev]

What do you want to achive? If you want to change transferred data before it reached the endpoint service then you should write “pre” filter and override HttpServletRequest. If you want to change the response of endpoint service after the transferred data reached it and has been processed, then you should write a “post” filter and override HttpServletResponse.

[RohitDevSwami]

I just want to add AuthStatus "A" or "U" to either request body or request header and get in my controller.

[rashad-farajullayev]

In the previous comment I made mistake speciying HttpServletResponse for “pre” filter. Actually in filter you can change request, in post filter you can change response. On the other hand in “pre” filter if you if you forward the request though serviceId and eureka then it would be enough to add “httpEntity” key with HttpServletRequest instance as value to the ctx.put(). I will attach some sample code in workday. You may remind me if you cannot solve the problem.

[RohitDevSwami]

Actualy i am new to microservices and zuul gateway, and i not using eureka, i'll remind you on monday for sample code. Thank you so much

[RohitDevSwami]

@rashad-farajullayev is it possible today to attach some sample code?

[rashad-farajullayev]

package your_organization.cloud.agent.filters;

import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;
import org.apache.commons.codec.binary.Base64;
import org.springframework.stereotype.Component;
import org.springframework.util.StreamUtils;
import javax.servlet.http.*;
import java.io.*;
import java.nio.charset.Charset;

import static com.netflix.zuul.context.RequestContext.getCurrentContext;
import static org.springframework.util.ReflectionUtils.rethrowRuntimeException;

@Component
public class EncryptFilter extends ZuulFilter {
    @Override
    public String filterType() {
        return "pre";
    }

    @Override
    public int filterOrder() {
        return 20;
    }

    @Override
    public boolean shouldFilter() {
        HttpServletRequest request = getCurrentContext().getRequest();
        boolean wsdlRequest = request.getRequestURI().toLowerCase().endsWith(".wsdl");
        boolean postSoap = request.getMethod().equalsIgnoreCase("post") && request.getContentType().toLowerCase().startsWith("text/xml");
        return wsdlRequest || postSoap;
    }

    @Override
    public Object run() {

        try {
            RequestContext context = getCurrentContext();
            InputStream in = (InputStream) context.get("requestEntity");
            if (in == null) {
                in = context.getRequest().getInputStream();
            }

            String body = StreamUtils.copyToString(in, Charset.forName("UTF-8"));
            context.setRequest(new YourRequestWrapper(context.getRequest(), new String(Base64.encodeBase64(body.getBytes()))));
        }
        catch (IOException e) {
            rethrowRuntimeException(e);
        }
        return null;
    }
}

[rashad-farajullayev]

package your_organization.cloud.agent.filters;

import com.netflix.zuul.http.ServletInputStreamWrapper;

import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.*;

public class YourRequestWrapper extends HttpServletRequestWrapper {
    public YourRequestWrapper  (HttpServletRequest request, byte[] data)
    {
        super(request);
        dataBytes = data;
    }

    public YourRequestWrapper (HttpServletRequest request, String body)
    {
        super(request);
        try {
            dataBytes = body.getBytes("UTF-8");
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    private byte[] dataBytes;
    //@Override
    public byte[] getContentData() {
        return dataBytes;
    }

    @Override
    public int getContentLength() {
        return dataBytes.length;
    }

    @Override
    public long getContentLengthLong() {
        return dataBytes.length;
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(new ByteArrayInputStream(dataBytes)));
    }

    @Override
    public ServletInputStream getInputStream() throws UnsupportedEncodingException {
        return new ServletInputStreamWrapper(dataBytes);
    }
}

[rashad-farajullayev]

The above code handles SOAP message requests, converts the entire body into base64 and forwards further - to another Zuul proxy.

The other Zuul proxy in its own turn, handles the base64 encoded request, decodes it and forwards the normal SOAP request to the real Spring WS server.

[RohitDevSwami]

@rashad-farajullayev thank you so much brother, You saved my lot of work. I am getting expected result, but i have decode it on each controller if i am passing Encoded value
: new tring(Base64.encodeBase64(body.getBytes()))));
and if I am passing String : context.setRequest(new YourRequestWrapper(context.getRequest(),body));
then i am getting null in my controller. so id there any way to pass without Base64.encodeBase64(body.getBytes()) ?

[rashad-farajullayev]

I added base64 encoding just to demonstrate body modification. You dont have to encode body. I just wanted to demonstrate where you can specify request headers.

[RohitDevSwami]

@rashad-farajullayev yes i tried it with context.setRequest(new YourRequestWrapper(context.getRequest(),myBody)); but i am getting null in controller because i am wrapping my body request to my Model(User)

[RohitDevSwami]

and is it possible to add in "authStatus":"A" or "U" in request header?

[rashad-farajullayev]

Actually to set only request headers you don't have to alter
Try using RequestContext.addZuulRequestHeader.
Upload your project so that I could look at it.

[RohitDevSwami]

@rashad-farajullayev yes i tried RequestContext.addZuulRequestHeader this approach, but how to get that header in controller because @RequestHeader(value = "authStatus" ) is coming null

code is:

package com.example.apigateway;
import java.io.InputStream;
import java.nio.charset.Charset;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.util.StreamUtils;

import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;

@Component
public class PreFilter extends ZuulFilter {

private final Logger logger = LoggerFactory.getLogger(this.getClass());

@Override
public String filterType() {
    return "pre";
}

@Override
public int filterOrder() {
    return 1;
}

@Override
public boolean shouldFilter() {
    return true;
}

public Object run() {
    try {
        RequestContext context = RequestContext.getCurrentContext();    
        context.addZuulRequestHeader("authStatus", "A");
    }
    catch (Exception ex) {
        logger.error("Error", ex);
    }
    return null;
}

}

and controller method is:

@RequestMapping(value = "/create", method = RequestMethod.POST)
public JSONObject createUser(
@RequestHeader(value = "authStatus") String p_authStatus ) throws Exception {
system.out.prinln(p_authStatus );
// p_authStatus is coming null
// so how to get that header
}

[rashad-farajullayev]

Zuul by default strips some headers. You can specify in the application.yml which headers are sensitive from security point of view. For example if you want Zuul not to strip any haders, you can specify empty array in the application.yml like below:

zuul:
   routes:
        web:
        path: /html/**
        sensitiveHeaders:
        url: http://www.google.com