Spring-cloud-netflix: Support Spring Data Pageable in Feign Client

[edethoor]

I am using Feign for requesting a MicroService who support Spring Data Pageable functionnality.

This is my FeignClient Interface :

 @FeignClient(FeignServiceId.SERVICE_ID)
 @RequestMapping(value = "api/document")
 public interface DocumentApi {

    @RequestMapping(method = RequestMethod.GET, value = "user/{userLogin}")
    Page<DeclarationDT> getDeclarationsByUserLogin(@PathVariable("userLogin") String userLogin, Pageable pageable);

}

When i make a call to the getDeclarationsByUserLogin, Feign do a POST request whereas i specify RequestMethod.GET.

I think this is due to the fact that Feign not support the Spring Data Pageable functionnality.

Is it possible to implement support for this feature ?

[spencergibb]

Actually, it thinks your Pageable is the body. Feign assumes post with a body.

[spencergibb]

@adriancole is there a way to add custom requests parameter mappings or some such?

[codefromthecrypt]

I think in denominator I just made a custom iterator which kept a marker.

Like this:
https://github.com/Netflix/denominator/blob/master/route53/src/main/java/denominator/route53/Route53AllProfileResourceRecordSetApi.java#L120

[Dreampie]

When can solve this problem?

[damienpolegato]

Hi, I'm also interested in this issue. I didn't quite get @adriancole example. Can we register a conversion somewhere ?
Thanks !

[damienpolegato]

My workaround to get it work is :

1. Use PageRequest instead of Pageable
2. On backend side register a converter in ConverterRegistry in a @Configuration class to convert String to PageRequest
   @Bean public Void addPageRequestConverter(ConverterRegistry converterRegistry) { converterRegistry.addConverter(new StringToPageRequestConverter()); return null; }
3. Define my own Page POJO since PageImpl doesn't have noargs constructor and setters
4. Implement a custom Jackson deserializer for Sort

So far at least that works.

Would 3 and 4 be handled by using a HATEOAS PagedResources ?

[codefromthecrypt]

ps sorry, my earlier response was about how to do pagination explicitly, like you have a response that includes a pagination marker, and use that for follow-up requests.

The issue here is how to implement Spring Data Pageable. I've not used that api, so don't have insight into it. Apologies for the distraction.

[dsyer]

@damienpolegato you will eventally be able to use Spring HATEOAS to reduce the burden here. I've been doing this:

@RequestMapping(value="/foos", method=RequestMethod.GET)
PagedResources<Foo> foos(@RequestParam("page") int page);

and it works fine out of the box to get the page data, so I can imagine you can extend to the other page parameters easily, or via a converter as you suggest. You could also use PageMetadata from Spring HATEOAS and write a converter for that instead of your own POJO.

To get the embedded contents as well you need to register an HttpMessageConverter with the Spring HATEOAS ObjectMapper. It's available in the context somewhere (and registers itself with any RestTemplates it finds), but is not registered with Feign yet (see #856).

[damienpolegato]

@adriancole no problem, I get your comment now, which is also uselful by the way.

@dsyer right, I haven't setup HATEOAS in my backend service yet. it will probably make things easier, I'll try that.

Also it is fine to use Page interface in the request return but to make it work I had to define a Jackson Mixin for my front service to define its own Page implementation. Makes things a bit cleaner.

Will try switching to PagedResources<T> next.

Thanks for the help.

[cforce]

Following Spring data rest approach into the client it would be the next logical step to have spring create a proxy on the remote (crud ) repository from remote repository interface. Spring cloud (feign) therefor can provide discovery of the rest Ressource, resilient calls and wrapping of the server side rest Ressource on the client side

[spencergibb]

@cforce, that's way beyond the scope of support pageable.

[xyloman]

We had implemented a system a year ago which leveraged spring data rest for backend services and feign client in the front end. We had a lot of success and have been working on upgrading to Brixton. We had gone with @dsyer suggestion of passing the request parameters explicitly which resulted in interface signatures such as the following:
@RequestMapping(value = "/messages/search/findByTenantIdAndCategory", method = RequestMethod.GET) PagedResources<Message> findByTenantIdAndCategory(@RequestParam("tenantId") String tenantId, @RequestParam("category") Integer category, @RequestParam(value = "sort", required = false) Collection<String> sort, @RequestParam("page") int page);

This would produce http requests that look like the following:
GET http://messages-service/messages/search/findByTenantIdAndCategory?tenantId=db1cd64a-56ae-4471-9cb0-e759a1aa7363&category=2&sort=firstName&sort=lastName%2CDESC&page=0 HTTP/1.1

This follows a convention that we saw posted on stack overflow and have numerous tests for in our spring data rest repositories: http://stackoverflow.com/questions/33018127/spring-data-rest-sort-by-multiple-properties

In the prior release of spring cloud netflix this worked as we would expect. After upgrading to Brixton.SR1 the following http request is produced which is not supported by spring data rest:
GET http://messages-service/messages/search/findByTenantIdAndCategory?tenantId=db1cd64a-56ae-4471-9cb0-e759a1aa7363&category=2&sort=firstName%2ClastName%2CDESC&page=0 HTTP/1.1

As you can see this GET request appears to be affected by the change of using the spring type conversion service which is included in the brixton release. This is a great enhancement but breaks for these cases because it converts the Collection to a comma delimited string.

Let me know if an additional issue needs to be logged or it should be considered apart of this issue. At the moment I am struggling to find a workaround. Right now this is blocking our ability to upgrade our system to the Brixton release train.

[spencergibb]

@xyloman this is a feature request issue, please open a new one if you think there is a bug.

[xyloman]

Per @spencergibb I believe this is a regression bug and I have logged a new issue: https://github.com/spring-cloud/spring-cloud-netflix/issues/1115

[joevalerio]

I've been wrestling with this for some time, and finally got a solution that doesn't suck.

My issue with @dsyer 's suggestion about just using the @RequestParam annotations for paging, quickly explodes if you want all combination of options, not to mention defaults. Combine that with the projection support, and you quickly have 4! overloaded methods for each resource method to account for every combination of paging options and projection.

What we really want here is a @QueryMap like the Default Feign Contract, but the Spring contract does not recognize it. So I added an AnnotatedParameterProcessor to recognize @QueryMap. Now for all methods where I want paging, projection, or both, I can add a map annotated with @QueryMap (I wish the Feign BuildTemplateByResolvingArgs class accounted for null QueryMaps so I didn't have to provide an empty one, but that is a small annoyance.)

Here is the AnnotatedParameterProcessor:

public class QueryMapParameterProcessor implements AnnotatedParameterProcessor {

  @Override
  public Class<? extends Annotation> getAnnotationType() {
    return QueryMap.class;
  }

  @Override
  public boolean processArgument(AnnotatedParameterContext context, Annotation annotation) {
    MethodMetadata data = context.getMethodMetadata();
    data.queryMapIndex(context.getParameterIndex());
    return true;
  }

}

Unfortunately you have to instantiate the SpringMvcContract using the List<AnnotatedParameterProcessor> contructor, and since they don't expose the default list, you have to do something like this.

    List<AnnotatedParameterProcessor> annotatedArgumentResolvers = new ArrayList<>();
    annotatedArgumentResolvers.add(new PathVariableParameterProcessor());
    annotatedArgumentResolvers.add(new RequestParamParameterProcessor());
    annotatedArgumentResolvers.add(new RequestHeaderParameterProcessor());
    annotatedArgumentResolvers.add(new QueryMapParameterProcessor());
    SpringMvcContract contract = new SpringMvcContract(annotatedArgumentResolvers);

To make the client signatures a little more user friendly, I created some classes to account for available options for each client method.

Pageable, Projection, and PageableProjection

Here is my Pageable impl... Projection just has the name, and PageableProjection is a combination of the two. You could validate the key on put, but right now, I don't care to.

public class Pageable implements Map<String, Object>{

  public static final String PAGE = "page";
  public static final String SIZE = "size";
  public static final String SORT = "sort";

  @Delegate
  protected Map<String, Object> delegate = new HashMap<String, Object>();

  public Integer getPageNumber() {
    return (Integer) delegate.get(PAGE);
  }

  public void setPageNumber(Integer pageNumber) {
    delegate.put(PAGE, pageNumber);
  }

  public Integer getPageSize() {
    return (Integer) delegate.get(SIZE);
  }

  public void setPageSize(Integer pageSize) {
    delegate.put(SIZE, pageSize);
  }

  @SuppressWarnings("unchecked")
  public List<Sort> getSortOrder() {
    return (List<Sort>) delegate.get(SORT);
  }

  public void setSortOrder(List<Sort> sortOrder) {
    delegate.put(SORT, sortOrder);
  }
}

Now you can have a method signature that looks like this, and it accounts for all combinations of paging options and projections:

  @RequestMapping(method = GET, value = "/items")
  PagedResources<Resource<Item>> getItems(@QueryMap PageableProjection pp);

This only works because the Feign MethodMetadata allows one @QueryMap parameter, and it must be a Map. The Spring AnnotatedParameterProcessors don't really have an answer for creating multiple parameters from one annotated argument, or at least I couldn't find it. You can then add some logic or a factory to construct these objects from the PageMetadata returned from the HATEOAS package.

Hope this helps,

Cheers

[cforce]

Would be great if your can provide an pull request with tests.

[tjuchniewicz]

@joevalerio Any working example? Test case?

[IsNull]

This bugged me as well, and I wanted to use the same method signature on the client (feign) as on the server side, i.e. Page<T> and Pageable in the feign interface.

I've come up with the following solution:

Pageable support in Feign

/**
     * This encoder adds support for pageable, which will be applied to the query parameters.
     */
    private class PageableQueryEncoder implements Encoder {

        private final Encoder delegate;

        PageableQueryEncoder(Encoder delegate){
            this.delegate = delegate;
        }

        @Override
        public void encode(Object object, Type bodyType, RequestTemplate template) throws EncodeException {

            if(object instanceof Pageable){
                Pageable pageable = (Pageable)object;
                template.query("page", pageable.getPageNumber() + "");
                template.query("size", pageable.getPageSize() + "");

                if(pageable.getSort() != null) {
                    Collection<String> existingSorts = template.queries().get("sort");
                    List<String> sortQueries  = existingSorts != null ? new ArrayList<>(existingSorts) : new ArrayList<>();
                    for (Sort.Order order : pageable.getSort()) {
                        sortQueries.add(order.getProperty() + "," + order.getDirection());
                    }
                    template.query("sort", sortQueries);
                }

            }else{
                delegate.encode(object, bodyType, template);
            }
        }
    }

This encoder can be added to your current encoder by composition:

Sample configuration

@Configuration
@EnableFeignClients
public class FeignClientConfig {

    @Autowired
    private ObjectFactory<HttpMessageConverters> messageConverters;

    @Bean
    public Encoder feignEncoder() {
        return new PageableQueryEncoder(new SpringEncoder(messageConverters));
    }
}

Page<T> support

This is a simple Jackson mapper issue, so it can be solved by adding a mixin for Page

Register (.mixIn(Page.class, PageMixIn.class)) the MixIn

    @JsonDeserialize(as = SimplePageImpl.class)
    private interface PageMixIn{ }

public class SimplePageImpl<T> implements Page<T> {

    private final Page<T> delegate;

    public SimplePageImpl(
            @JsonProperty("content") List<T> content,
            @JsonProperty("page")int number,
            @JsonProperty("size") int size,
            @JsonProperty("totalElements") long totalElements){
        delegate = new PageImpl<>(content, new PageRequest(number, size), totalElements);
    }


    @JsonProperty
    @Override
    public int getTotalPages() {
        return delegate.getTotalPages();
    }

    @JsonProperty
    @Override
    public long getTotalElements() {
        return delegate.getTotalElements();
    }

    @JsonProperty("page")
    @Override
    public int getNumber() {
        return delegate.getNumber();
    }

    @JsonProperty
    @Override
    public int getSize() {
        return delegate.getSize();
    }

    @JsonProperty
    @Override
    public int getNumberOfElements() {
        return delegate.getNumberOfElements();
    }

    @JsonProperty
    @Override
    public List<T> getContent() {
        return delegate.getContent();
    }

    @JsonProperty
    @Override
    public boolean hasContent() {
        return delegate.hasContent();
    }

    @JsonIgnore
    @Override
    public Sort getSort() {
        return delegate.getSort();
    }

    @JsonProperty
    @Override
    public boolean isFirst() {
        return delegate.isFirst();
    }

    @JsonProperty
    @Override
    public boolean isLast() {
        return delegate.isLast();
    }

    @JsonIgnore
    @Override
    public boolean hasNext() {
        return delegate.hasNext();
    }

    @JsonIgnore
    @Override
    public boolean hasPrevious() {
        return delegate.hasPrevious();
    }

    @JsonIgnore
    @Override
    public Pageable nextPageable() {
        return delegate.nextPageable();
    }
    @JsonIgnore
    @Override
    public Pageable previousPageable() {
        return delegate.previousPageable();
    }
    @JsonIgnore
    @Override
    public <S> Page<S> map(Converter<? super T, ? extends S> converter) {
        return delegate.map(converter);
    }

    @JsonIgnore
    @Override
    public Iterator<T> iterator() {
        return delegate.iterator();
    }

Maybe that helps someone :)

[tjuchniewicz]

Works for me. Thanks a lot @IsNull !

Pageable must be annotated with @RequestBody. Looks like Feign Encoder is only applied for body parameters.

My Jackson configuration for Spring Boot:

public class MyJacksonModule extends SimpleModule {

    @Override
    public void setupModule(SetupContext context) {
        context.setMixInAnnotations(Page.class, PageMixIn.class);
    }
}

@Configuration
public class MyJacksonConfiguration {

    @Bean
    public Module myJacksonModule() {
        return new MyJacksonModule();
    }
}

[berinle]

Thanks @IsNull and @tjuchniewicz! I have been banging my head against my desk for the last 2 days to resolve this and a combination of your solutions helped solve the issue.

[spencergibb]

@IsNull interested in creating a PR?

[IsNull]

@spencergibb Actually yes. I'll give it a try.

[IsNull]

Here is the pull request #1604

[jkubrynski]

Looking forward to merging this PR :)

[spencergibb]

It is waiting on @IsNull

[shahbour]

+1 for this

[GarimaJainATOS]

Hi Guys,
I am trying to call below method through feignClient
ResponseEntity> getTransactionList(
@Valid TransactionSearchRequest categorizedTransactionSearchRequest,
@ApiParam("Pageable information") @PageableDefault(size = 05, page = 0) Pageable p);
But i am getting below exception

Caused by: java.lang.IllegalStateException: Method has too many Body parameters: public abstract org.springframework.http.ResponseEntity com.worldline.fpl.banking.budget.controller.IBudgetController.getTransactionList(
com.worldline.fpl.banking.budget.json.TransactionSearchRequest,org.springframework.data.domain.Pageable)
at feign.Util.checkState(Util.java:128)
at feign.Contract$BaseContract.parseAndValidateMetadata(Contract.java:114)
at org.springframework.cloud.netflix.feign.support.SpringMvcContract.parseAndValidateMetadata(SpringMvcContract.java:133)
at feign.Contract$BaseContract.parseAndValidatateMetadata(Contract.java:64)
at feign.hystrix.HystrixDelegatingContract.parseAndValidatateMetadata(HystrixDelegatingContract.java:34)
at feign.ReflectiveFeign$ParseHandlersByName.apply(ReflectiveFeign.java:146)
at feign.ReflectiveFeign.newInstance(ReflectiveFeign.java:53)
at feign.Feign$Builder.target(Feign.java:209)
at org.springframework.cloud.netflix.feign.HystrixTargeter.target(HystrixTargeter.java:48)
at org.springframework.cloud.netflix.feign.FeignClientFactoryBean.loadBalance(FeignClientFactoryBean.java:146)
at org.springframework.cloud.netflix.feign.FeignClientFactoryBean.getObject(FeignClientFactoryBean.java:167)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
... 44 common frames omitted

When i debugged it, i got know that these
interface org.springframework.data.web.PageableDefault
interface javax.validation.Valid
are not the feign supported annotations and that's why it is failing
But i am not getting the solid fix for it .
Can anyone help me on this as soon as possible ?

[jokefaker]

+1 for this

[gregturn]

I'm a little boggled about the concept of using Feign when talking a REST-based service. Feign is an RPC-based technology while Spring Data REST is about exposing hypermedia, i.e. links. I mean you literally hard code each and every method call to the exact path in the remote service, hence no leveraging of all the links built up by Spring Data REST. This makes the whole thing quite brittle and hard to change on the server side.

For an example of an evolvable system built on Spring HATEOAS, the underlying web tech of Spring Data REST, look at this example of API evoluation: https://github.com/spring-projects/spring-hateoas-examples/tree/master/api-evolution

I can't imagine evolving such a system using Feign without lots of effort.

[steventong]

+1 for this feature

[ryanjbaxter]

Moving this issue over to the openfeign project https://github.com/spring-cloud/spring-cloud-openfeign/issues/26

[luizhenriquesantana]

Had the same issue, worked when I added this to the controller:

@Bean public Module pageJacksonModule() { return new PageJacksonModule(); }