Micronaut-core: Provide a way to read the request body in a filter when the route does not read it

[claudiodegio]

This is a enhancement request

I want to implement a custom HttpServerFilter, to get POST request and body (json format).
But i have the issue when try to read body with method request.getBody it's always empty,
to solve a have defined an action with @Post and @Body annotation into controller.
But i don't want to create action for each api i need.
regs

I need a filter this a method like this:

Publisher<HttpHttpResponse<T>> readBody(HttpRequest<?> pRequest, ServerFilterChain pChain, Class<T> type)

[jameskleeh]

@claudiodegio Can you provide an example of what isn't working?

[claudiodegio]

Here a example my filter

@Filter("/**")
public class MyFilter implements HttpServerFilter {

    @Override
    public Publisher<MutableHttpResponse<?>> doFilter(HttpRequest<?> request, ServerFilterChain chain) {
        Flowable<Boolean> flowable = Flowable.fromCallable(() -> {


            System.out.println("------- My filter ------");

            Optional<String> body = request.getBody(String.class);


            if (body.isPresent()) {
                System.out.println("BODY PRESENT:" + body.get());
            } else {
                System.out.println("BODY NOT PRESENT");

            }

            System.out.println("------- My filter ------");

            return true;
        }).subscribeOn(Schedulers.io());

        return flowable.switchMap(aBoolean -> chain.proceed(request));
    }

}

My controller

@Controller("/testBug/")
public class MyContoller {

    @Post(value = "/postJson")
    @Consumes({MediaType.APPLICATION_JSON})
    public String testParamsFrom(@Body Map<String, Object> body) {


        System.out.println(body);

        return "OK";
    }


    @Post(value = "/postJsonNoBody")
    @Consumes({MediaType.APPLICATION_JSON})
    public String testParamsFrom() {

        return "OK";
    }
}

Performing a reset in testBug/postJson with json body, the logs print this line

------- My filter ------
BODY PRESENT:{"key":"value"}
------- My filter ------

Performing a reset in testBug/postJsonNoBody with json body, the logs print this line

------- My filter ------
BODY NOT PRESENT
------- My filter ------

[jameskleeh]

@claudiodegio You still haven't explained what isn't working? Is what you described above what you expect to happen or what is happening?

What is the point of the Flowable.fromCallable? Just to move the work onto a different thread?

[claudiodegio]

Hi james,

The example above contain an example working as expected ad an example dosent' work.

The case "postJson" work as expected, the message body it's available on HttpFilter and into controller by @Body annotation.

The case "postJsonNoBody" don't work, the message body it is not available on HttpFilter also i read it.

------- My filter ------
BODY NOT PRESENT
------- My filter ------

This is just an example on my issue. On my application i have to read the body of all http request using an http filter.
The body of request is always empty to solve the issue i need to add a fake action controller with body annotation, but I don't want the add the controller because the source request it's dynamic for my application.

regs

[jameskleeh]

@claudiodegio I see now. I don't think we can support this type of use case because the route, and thus the filters are executed as soon as the route is "available", meaning the arguments are satisfied. In the second case, the body is never read at all. I don't think there is anything we can do about this issue without adding in some additional way for you to tell Micronaut you want the body read.

It seems like an easy enough solution to simply add a body argument given this is an atypical use case.

[claudiodegio]

Thank you for response, i understand the motivation.

But how a can build a small gateway api with micronaut where the api can be configured dinamically using a external configuration like mongo or json file ?
EX:
inputApi: /api/ exposed my application
mapped on outApi: /api/ using service discovery XXX

I don't wont create a controller for each input api to map an other microservice, but i want to use a configuration file.

It's possibile to do this with micronaut ?

I used a HttpServerFilter to do this for intercept all input request a java map with routing information (output service and output api), all work fine but i blocked with empty body on post request.

regs

[adamkobor]

I have this problem too :( We want to do request forwarding on our gateway with minimal side effects, and service discovery, which works just fine, unless we want to forward a request with body.

There is a not so nice workaround, you have to provide fake controller actions for every request with a body:

@Controller("/")
class FakeController {

    @Put("/full-path-to-match/user/{userId}/search-agent")
    fun upsertSearchAgent(userId: String, @Valid @Body searchAgent: Any): Single<Any> {
        return Single.just(null)
    }
}

Big +1 for this feature request, I think it would be super helpful in a lot of cases, if we could use the request body in HttpServerFilter without the @Body annotation.

[venkyvb]

Have a similar use-case as well.

[nhoughto]

So it appears you can workaround this with a dummy controller and an @Error annotation, the original intent of the @Error is for a fallback if a controller isn't found, but we can co-opt it into forcing the body generation for us, an incomplete example:

@Controller
public class DummyController {

    @Error(global = true, status = HttpStatus.NOT_FOUND)
    public HttpResponse<String> handleError(HttpRequest<Object> request, @Body String body) {
        return null; <-- probably want to delegate to default handling here?  depends on scenario..
    }

}

[nhoughto]

After a bit more poking around, it appears the above hack only works if you never intent to call the controller (if you are always handling the request in a filter) or will only ever handle requests with a body, as you can only have 1 global 404 handler, and you have to have the @Body argument, which will fail for a GET request without a body. I tried Optional<String> body but it didn't work, it also failed the GET request. 🤷‍♂

[jameskleeh]

@claudiodegio @adamkobor @venkyvb @nhoughto I'm open to any ideas you may have to allow for this behavior.

[ddrao]

Working on a similar use case. An example on how to build a transparent API Gateway that forwards requests and responses and logs the request and response bodies would go a long way.

[claudiodegio]

@jameskleeh Add support of @Body to filter level ?

[graemerocher]

I think what makes the most sense to expand the FilterChain interface so that proceed(..) behaves as it does today and then have another method that allows reading the body.

My only concern is how a filter could then interfere and change what a controller could do with the body. For example a global filter that reads the body into a string would never work with controller actions that need to stream the body.

I guess this is the same as servlet filters that call getInputStream and read the body. They then have to subsequently buffer the input stream and allow it to be "re-read" in downstream servlets.

What may be better is to simply implement the use cases that people are asking for this feature. For example what seems to be common is the need to forward or proxy the request. So instead of adding the ability to read the body we would add methods are the ability to proxy the request to another service.

Maybe users can chime in on what other use cases they have for reading the request body in a filter.

[nhoughto]

So my use case is more using the body then proxying it, so the servlet style input stream approach would suit me more.

My only reason for using filters was that it is the only (?) way to get micronaut to handle wildcard style requests, where the path and method can be anything at runtime.

[graemerocher]

@nhoughto So again that sounds like a limitation which should take away rather than trying to add something that can break downstream controllers. Something like:

@Controller
class CatchAllController {
   @Any("{+path}")
    void handle(HttpRequest request) {
          ...
     }
}

[claudiodegio]

I have the same problem, i want to proxy any request using filter but the
body of request it was empty.
To solve I have added a fake controller and action to have the
body available at filter level.

On Tue, Nov 12, 2019 at 11:01 AM Graeme Rocher [e-mail]
wrote:

@nhoughto https://github.com/nhoughto So again that sounds like a
limitation which should take away rather than trying to add something that
can break downstream controllers. Something like:

@Controllerclass CatchAllController {
@Any("{+path}")
void handle(HttpRequest request) {
...
}
}

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/micronaut-projects/micronaut-core/issues/1113?email_source=notifications&email_token=AAG74DZ7KRK6S6D4O2WE7RTQTJ5GDA5CNFSM4GQC5BTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDZWKWA#issuecomment-552822104,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAG74D3DBRSEOT5IVY5KMODQTJ5GDANCNFSM4GQC5BTA
.

--
Claudio Degioanni
Skype: claudiodegio

[MikiLoz92]

Same use case here, trying to implement a filter that can act as a proxy for an API gateway. Not having this makes using Micronaut unable for this kind of projects.

[gyulavoros]

My understanding is that folks would need something like the following (in terms of functionality):

• Spring Cloud Gateway
• Nginx reverse proxy
• HAProxy

[lordbuddha]

Well another use case is for a Caching HttpServerFilter. If say we wanted to use the post body as the cache key (well an MD5 of the post body bytes), it would be nice to be able to always get that post body as bytes in a consistent way. Currently for application/json the body is available as an ObjectNode, but for a application/json-rpc the body is available as a CompositeByteBuf. It really should be possible to always get it as bytes (or ByteBuffer etc) via an Optional.

[graemerocher]

My current view on this issue is that there are 2 distinct use cases:

• Low level access to the raw request / response bytes
• Proxying gateway

Filters are not really designed for either of these on their own. They work at the level of objects that have already been decoded or are to be encoded.

To support these use cases my current intention is to allow users to extend Micronaut's Netty pipeline so that low level access to the raw byte buffers can be obtained for use cases like caching, tracing etc.

For proxying / gateway a new interface ProxyClient or GatewayClient will be introduced which will allow filters and/or controllers to proxy requests to another service or services.

[graemerocher]

PR for low level request/response access for logging, tracing etc. #3061

[graemerocher]

Low level access to the Netty pipeline part of this issue is done. Example here (using logbook):

https://docs.micronaut.io/snapshot/guide/index.html#nettyPipeline

[graemerocher]

PR submitted including an API to proxy requests from a filter https://github.com/micronaut-projects/micronaut-core/pull/3068

[claudiodegio]

Thank you @graemerocher fro support

[tandel-pratik]

What's the recommended way for filters to access the body in a filter? An example would be great -- i'm trying to implement a function that verifies that the signed version of the body matches one of the headers (this is to verify that the webhook is authentic).

[muravel]

hi @pratik-brex,
I have the same case.
Have You found a solution?

[tandel-pratik]

I ended up not writing a filter for it and instead performed the check in the handler.

[graemerocher]

@pratik-brex that is the correct way to do it