Hystrix: Incorrectly getting error "Collapser method must have one argument" from CollapserMetaHolderFactory

[ppetrosyan]

I am having a strange issue. I have a method which is annotated with HystrixCommand

`
@Override
@HystrixCommand(fallbackMethod = "getDataFallback",
commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "60000"),
@HystrixProperty(name="execution.isolation.thread.interruptOnTimeout", value="TRUE"),
@HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"),
@HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"),
@HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50")
},
threadPoolProperties = {
@HystrixProperty(name = "coreSize", value = "10")
})

public List getData(String id, Date date) {
return null;
}
`

it works most of the times, but rarely without changing anything I get the following exception

java.lang.IllegalStateException: Collapser method must have one argument: public java.util.List getData(String,Date)

I checked the code of Hystrix and found that the meta holder factory checks if the HystrixCommand annotation is present then it initializes CommandMetaHolderFactory, otherwise CollapserMetaHolderFactory. In my case, I have the HystrixCommand but it still tries to use CollapserMetaHolderFactory which complains about method not having 1 argument.

This happens rarely and it's possible no one has seen it yet, I was just wondering if there is a recent fix related to this issue. I am using hystrix 1.5.0.

Thanks in advance.

[mattrjacobs]

I haven't heard any reports of this yet. @dmgcodevil - any idea what this might be?

[ppetrosyan]

Just to add more info, I continued looking at the code of Hystrix and it seems unbelievable what I am seeing in the logs because I can see it uses Java reflection and checks if HystrixCommand is present.

return method.isAnnotationPresent(HystrixCommand.class) ? COMMAND : COLLAPSER;

but the fact is I got this error on 2 out of 5 servers running the exact same code. I restarted one of the failed servers and problem went away, still have the other server in "bad" state so I can reproduce the problem with every request but as soon as I restart it, the problem most likely will go away.

This is not the first time we see this problem, about 2 months ago I noticed the same thing but restarted and couldn't reproduce. It's really interesting what's going on there.

[dmgcodevil]

Hi, will take a look at all javanica issues on this weekend

[dmgcodevil]

@ppetrosyan it's really weird, agree. It might be anything: JIT optimization that causes that issue, code cached since previous deployment (unlikely), concurrency issue and etc. I wonder if Collapser annotation is present in the method at this point either. Please continue updating the issue with your findings, I'll do the best from my side.

[matterche]

I'm facing a similar issue with this code (Javanica 1.5.6, Spring Boot 1.4.2):

@HystrixCommand(
            threadPoolKey = "get-purchase-orders-thread-pool",
            ignoreExceptions = {IllegalArgumentException.class, ResourceNotFoundException.class},
            commandProperties = {
                    @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "10000")
            }, threadPoolProperties = {
            @HystrixProperty(name = "maxQueueSize", value = "100"),
            @HystrixProperty(name = "queueSizeRejectionThreshold", value = "100")
    }
    )
    public Future<Optional<PurchaseOrder>> getPurchaseOrderByCode(final String code) {
       ...
    }

We have several nodes running the application. All worked fine for several days. Today requests bound to the command failed on all nodes with

...
Caused by: java.lang.NullPointerException: null
at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect$CollapserMetaHolderFactory.create(HystrixCommandAspect.java:169) ~[hystrix-javanica-1.5.6.jar!/:1.5.6]
at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect$MetaHolderFactory.create(HystrixCommandAspect.java:144) ~[hystrix-javanica-1.5.6.jar!/:1.5.6]
at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect.methodsAnnotatedWithHystrixCommand(HystrixCommandAspect.java:90) ~[hystrix-javanica-1.5.6.jar!/:1.5.6]
...

A restart solved the issue but I couldn't reproduce it.

As we don't use HystrixCollapser in our code my guess was that

return method.isAnnotationPresent(HystrixCommand.class) ? COMMAND : COLLAPSER;

doesn't find any annotation at all so that hystrixCollapser is null when getDeclaredMethod is called in:

 public MetaHolder create(Object proxy, Method collapserMethod, Object obj, Object[] args, final ProceedingJoinPoint joinPoint) {
            HystrixCollapser hystrixCollapser = collapserMethod.getAnnotation(HystrixCollapser.class);
            if (collapserMethod.getParameterTypes().length > 1 || collapserMethod.getParameterTypes().length == 0) {
                throw new IllegalStateException("Collapser method must have one argument: " + collapserMethod);
            }

            Method batchCommandMethod = getDeclaredMethod(obj.getClass(), hystrixCollapser.batchMethod(), List.class);

But I don't know how this could happen.

One additional detail: getPurchaseOrderByCode implements an interface method which is not annotated.

[matterche]

What do you think about adding a sanity check to get more insights:

  private enum HystrixPointcutType {
        COMMAND,
        COLLAPSER;

        static HystrixPointcutType of(Method method) {
            if(method.isAnnotationPresent(HystrixCommand.class)){
                return COMMAND;
            } else if(method.isAnnotationPresent(HystrixCollapser.class)){
                return COLLAPSER;
            } else {
                throw new IllegalStateException("no valid annotation found for ...");
            }            
        }
    }

[dmgcodevil]

@ppetrosyan, @matterche could you please provide a bit more information:

1. Java version + JVM parameters
2. Does a class with hystrix command implement a generic interface ?
3. What kind of proxy do you use in your project, e.g. :

@EnableAspectJAutoProxy(proxyTargetClass = true) - CGLIB proxy
@EnableAspectJAutoProxy(proxyTargetClass = false) - JDK proxy

1. How big a command method is : 5, 10 or 100 lines
2. Is a method executed concurrently under heavy load

the idea to add an extra info could help us but I want to reproduce the issue
Thanks

[matterche]

Java version + JVM parameters

openjdk:8u91-b14-1-22

Does a class with hystrix command implement a generic interface ?

It implements an interface but not a generic one

What kind of proxy do you use in your project, e.g. :

JDK proxy

How big a command method is : 5, 10 or 100 lines

~ 30 lines

Is a method executed concurrently under heavy load

The method is run concurrently

[dmgcodevil]

I gave up trying to reproduce the issue. I tried to analyze JIT compilation log but didn't find anything suspicious, tried different computation modes: parallel and sequential under heavy load, both are working fine. I added a sanity check to get more insights, it should help to find out the root cause in a next time.
PR

[matterche]

👍

[ppetrosyan]

Thanks for looking at this. During this time I encountered this problem couple more times but no solid clue was found because it was really rare and random. I changed the code to not use Hystrix until this problem is solved, so I won't have new updates for a while. Here is the info you requested from my side.

Java version + JVM parameters
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Does a class with hystrix command implement a generic interface ?
It implements an interface but not a generic one

What kind of proxy do you use in your project, e.g. :
JDK proxy

How big a command method is : 5, 10 or 100 lines
The method itself is less than 5 lines, but it calls other methods which exceed 100+ lines

Is a method executed concurrently under heavy load
The method is run concurrently

Hope the logs you added will help to find this issue.

Thanks

[ashumeet]

The log statement added above leads me to this discussion: Hope my logs can help to find the root cause. Please let me know if you see any obvious solution:

'https://github.com/Netflix/Hystrix/issues/1458' - no valid annotation found for:
Method signature:
public java.lang.String com.blurb.orderservice.client.bookserve.PlaceOrderClient.placeOrder(com.blurb.orderservice.domain.order.SubOrder,java.util.Map) throws java.rmi.RemoteException,com.blurb.common.service.api.ApiException
Declaring class:
com.blurb.orderservice.client.bookserve.PlaceOrderClient
Flags:
Bridge=false
Synthetic=false
Final=false
Native=false
Synchronized=false
Abstract=false
AccessLevel=public
Return Type:
ReturnType=class java.lang.String
GenericReturnType=class java.lang.String
Parameters:
parameter [0]:
ParameterType=class com.blurb.orderservice.domain.order.SubOrder
GenericParameterType=class com.blurb.orderservice.domain.order.SubOrder
parameter [1]:
ParameterType=interface java.util.Map
GenericParameterType=java.util.Map
Exceptions:
exception [0]:
ExceptionType=class java.rmi.RemoteException
GenericExceptionType=class java.rmi.RemoteException
exception [1]:
ExceptionType=class com.blurb.common.service.api.ApiException
GenericExceptionType=class com.blurb.common.service.api.ApiException
Annotations:empty

[ashumeet]

@mattrjacobs @dmgcodevil ... Below is the histrix and system info:

Histrix:
com.netflix.hystrix:hystrix-javanica:1.5.12
org.springframework.cloud:spring-cloud-starter-netflix-hystrix:1.4.3.RELEASE

Java + JVM Info:
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

Does a class with hystrix command implement a generic interface?
It doesn't implement a generic interface

What kind of proxy do you use in your project, e.g. :
Apache Access

How big a command method is : 5, 10 or 100 lines
It's about 20 lines but the code underneath is around 100 lines

Is a method executed concurrently under heavy load
it's not executed concurrently and not under heavy load

Below is the Histrix annotation we are using on the method:

@SuppressWarnings({ "rawtypes", "unchecked" }) @TimeMeasured(tag = "bookserve#place_order") @HystrixCommand(ignoreExceptions = { IllegalArgumentException.class }, commandProperties = { @HystrixProperty(name = "execution.isolation.strategy", value = "THREAD"), @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = PLACE_ORDER_TIMEOUT_IN_MS), @HystrixProperty(name = "fallback. ", value = "false") }) public String placeOrder(SubOrder subOrder, Map<Long, AddBookResponse> addBookResponses)

Any insight will be helpful. Thank you in advance.

[dmgcodevil]

@asamra-blurb thanks for the update, I'll take a look.
but Annotations:empty looks bad...

[shaswat09]

The log statement added above leads me to this discussion: Hope my logs can help to find the root cause. Please let me know if you see any obvious solution:

'#1458' - no valid annotation found for:
Method signature:
public java.lang.String com.blurb.orderservice.client.bookserve.PlaceOrderClient.placeOrder(com.blurb.orderservice.domain.order.SubOrder,java.util.Map) throws java.rmi.RemoteException,com.blurb.common.service.api.ApiException
Declaring class:
com.blurb.orderservice.client.bookserve.PlaceOrderClient
Flags:
Bridge=false
Synthetic=false
Final=false
Native=false
Synchronized=false
Abstract=false
AccessLevel=public
Return Type:
ReturnType=class java.lang.String
GenericReturnType=class java.lang.String
Parameters:
parameter [0]:
ParameterType=class com.blurb.orderservice.domain.order.SubOrder
GenericParameterType=class com.blurb.orderservice.domain.order.SubOrder
parameter [1]:
ParameterType=interface java.util.Map
GenericParameterType=java.util.Map
Exceptions:
exception [0]:
ExceptionType=class java.rmi.RemoteException
GenericExceptionType=class java.rmi.RemoteException
exception [1]:
ExceptionType=class com.blurb.common.service.api.ApiException
GenericExceptionType=class com.blurb.common.service.api.ApiException
Annotations:empty

Did you found any solution? I'm having same exception

Thanks

[dmgcodevil]

@shaswat09 tomorrow I'll figure it out, be online .

[shaswat09]

@shaswat09 tomorrow I'll figure it out, be online .

Below Error I'm getting

java.lang.IllegalStateException: 'https://github.com/Netflix/Hystrix/issues/1458' - no valid annotation found for:
Method signature:
public final java.lang.String com.sun.proxy.$Proxy124.whoami(java.lang.String)
Declaring class:
com.sun.proxy.$Proxy124

Flags:
Bridge=false
Synthetic=false
Final=true
Native=false
Synchronized=false
Abstract=false
AccessLevel=public

Return Type:
ReturnType=class java.lang.String
GenericReturnType=class java.lang.String

Parameters:
parameter [0]:
ParameterType=class java.lang.String
GenericParameterType=class java.lang.String

Exceptions:empty

Annotations:empty

at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect$HystrixPointcutType.of(HystrixCommandAspect.java:293) ~[hystrix-javanica-1.5.12.jar:1.5.12]
at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect.methodsAnnotatedWithHystrixCommand(HystrixCommandAspect.java:94) ~[hystrix-javanica-1.5.12.jar:1.5.12]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_172]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_172]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_172]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_172]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at com.sun.proxy.$Proxy125.whoami(Unknown Source) ~[na:na]
at com.example.demo.rest.controller.FeignController.getUsername(FeignController.java:21) ~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_172]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_172]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_172]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_172]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:891) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851) ~[spring-webmvc-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) ~[spring-boot-actuator-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:155) ~[spring-boot-actuator-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:123) ~[spring-boot-actuator-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108) ~[spring-boot-actuator-2.0.6.RELEASE.jar:2.0.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.34.jar:8.5.34]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_172]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_172]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.34.jar:8.5.34]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_172]

My Code:

@FeignClient("config-client")
public interface OrganizationFeignClient {

@HystrixCommand(
        threadPoolKey = "licenseByOrgThreadPool",
        commandProperties=
        {@HystrixProperty(name="execution.isolation.thread.timeoutInMilliseconds",value="12000"),
        @HystrixProperty(name="circuitBreaker.requestVolumeThreshold",value="10"),
        @HystrixProperty(name="circuitBreaker.errorThresholdPercentage",value="75"),
        @HystrixProperty(name="circuitBreaker.sleepWindowInMilliseconds",value="7000")
        },
        fallbackMethod = "buildFallbackLicenseList",
        threadPoolProperties =
        {@HystrixProperty(name = "coreSize",value="30"),
        @HystrixProperty(name="maxQueueSize", value="10")}
        )
@RequestMapping(
          value = "/whoami/{username}", 
          method = RequestMethod.GET)
public String whoami(@PathVariable("username") String username);

public default String buildFallbackLicenseList(String username){
    return "defaultUsername"; // This method is invoked if call to remote microservice will failed with the help of circuit breaker 
}

}

[dmgcodevil]

Method signature: public final java.lang.String com.sun.proxy.$Proxy124.whoami(java.lang.String)

it should not be a method from java Proxy class. Looks like there is a problem with getMethodFromTarget

cc: @shaswat09

[dmgcodevil]

The funny thing is that when I use an interface that contains a method annotated with @HystrixCommand then it wont be intercepted by HystrixCommandAspect: test

[dmgcodevil]

I think that a temporary solution would be switch to CGLIB proxies, instead of JDK: @EnableAspectJAutoProxy(proxyTargetClass = true)

[sullii]

today,i have faced this question ,do you solved this.give me some suggestion.below is the trace infomation:
Method signature:
public final java.util.List com.sun.proxy.$Proxy227.addEmployeeGenerateAccount(java.util.List) throws java.lang.Exception
Declaring class:
com.sun.proxy.$Proxy227

Flags:
Bridge=false
Synthetic=false
Final=true
Native=false
Synchronized=false
Abstract=false
AccessLevel=public

Return Type:
ReturnType=interface java.util.List
GenericReturnType=interface java.util.List

Parameters:
parameter [0]:
ParameterType=interface java.util.List
GenericParameterType=interface java.util.List

Exceptions:
exception [0]:
ExceptionType=class java.lang.Exception
GenericExceptionType=class java.lang.Exception

Annotations:empty
at com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect$HystrixPointcutType.of(HystrixCommandAspect.java:282)

my springboot version is 2.1.6
i want to give the feign interface method timeout setting

[dmgcodevil]

@sullii https://github.com/Netflix/Hystrix/issues/1458#issuecomment-531019345

[maccamlc]

I just came across this error, and link in my Spring Boot application.

Hopefully this might help someone else that gets directed here later.

I was updating a library that was using spring cloud feign (@EnabledFeignClients) and then on the actual @FeignClient it would add @HystrixCommand to the method. For example:

@FeignClient(name = "service-client", url = "${services.url}")
public interface ServiceClient {

    @RequestMapping(value = "/v1/service/{id}", method = RequestMethod.GET, consumes = MediaType.APPLICATION_JSON_VALUE)
    @HystrixCommand
    ResponseEntity<Value> getValue(@NonNull @PathVariable("id") final String id);

}

This was working as they expected (ie. using Hystrix) as they turned on feign.hystrix.enabled: true in the configuration.

Once a new library was added, that actually used @EnableHystrix everything started to break, with the above error ... no valid annotation found for:

Basically, removing the @HystrixCommand from the feign methods, fixes the problem. The proxy created by Feign, does not have any annotations, which seems to be why this error occurs. It is also not required as feign-hystrix when enabled, will use HystrixInvocationHandler that gives you the fallback functionality, etc.

Using HystrixCommand on other beans, now also works.

So from what I learned, if you get this exception in your logs, and you have the HystrixCommand on a Feign Client, then remove it and just use feign-hystrix.

[shaswat09]

You can use @HystrixCommand annotation only on @Service or @Component class bean

[jicui]

You can use @HystrixCommand annotation only on @service or @component class bean
HystrixCommand is only target on method level ,something seems a break between feign integrate with hystrix if we need to configure like groupKey for matrrix in dashboard

[Clopma]

Why is this issue closed if there is no fix?