Spring-cloud-netflix: Read Timeout/ Socket timeout in Zuul

[mirak94]

I've read some issues while researching for the problem of java.net.SocketTimeoutException: Read timed out but all of them are saying the proposed solutions, which are the following: if you are using ribbon so increase the ribbon.Readtimeout and if you are using zuul routes so increase zuul.host.socket-timeout-millis.
I am using zuul routes and I am getting the above exception, but I am seeing that the default value for zuul.host.socket-timeout-millis is 10000, and this is enough I guess.
I can't figure out what may be causing the problem, my internal services are not taking more than 10 seconds for example to send data.
I need help in identifying the possible causes for this problem, because I guess just going and increase the socket timeout is not the solution.
I also need also some insights about the problems that may occur as result of high socket timeout, and when should I say that socket timeout of x mills is considered high, and y is considered low.
Actually I need somehow to be able to define the parameters that I should take in consideration when I see this problem (Read timed out).

Thanks!

[ryanjbaxter]

Can you tell us what version of Spring Cloud you are using and how you are configuring your Zuul routes?

[tmelhiser]

We are seeing this same behavior, using Edgware.RELEASE.

Zuul is 1.4.0, using Ribbon 2.2.4, with Eureka Client 1.7.0.

Ribbon is dynamically populating the routes via Eureka Discovery. The discovery and population of the routes works perfectly. However on the first run against the routes, we receive a SocketTimeoutException.

Eventually, subsequent runs are successful, but the initial few requests fail.

I have tried these settings inside the zuul spring-boot application.yml , but the timeout does not seem to change:

zuul:
  host:
    connect-timeout-millis: 60000
    socket-timeout-millis: 60000
    max-total-connections: 200
    max-per-route-connections: 20
    time-unit: MILLISECONDS
    time-to-live: 10000

ribbon:
  eureka:
    enable: true
  isSecure: false
  readTimeout: 60000
  connectTimeout: 60000

hystrix:
  command:
    default:
      execution:
        isolation:
          thread:
            timeoutInMilliseconds: 60000

[ryanjbaxter]

Could you provide the exception? There is a known issue with the hystrix timeout when using Zuul and Ribbon, see https://github.com/spring-cloud/spring-cloud-netflix/issues/2606

[spencergibb]

The first time timeout can be mitigated by eager loading https://github.com/spring-cloud/spring-cloud-netflix/blob/master/docs/src/main/asciidoc/spring-cloud-netflix.adoc#zuul-eager-application-context-loading

[tmelhiser]

Attached are the logs for the stack trace:
zuul.log

@spencergibb I will take a look. Is there a mechanism to seed the eager loading based off of eureka's distribution of the hosts? These instances are hosted inside a docker swarm and would have unique names on every deploy. Docker's internal DNS will return multiple hosts for one hostname, would this be a viable way to seed this value?

[tmack8001]

I also find these SocketTimeoutExceptions need to be messaged to get the actual service which is having the problem. Unlike the exceptions wrapped in HystrixRuntimeException that contain either the hostname / ip / ribbon service name. Using Zipkin or similar tracing library in logs to connect this back to the original request log is complicated.

I'd rather like to see this exception occur with the following text as an example:

Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: service-a read timed out
    at rx.exceptions.Exceptions.propagate(Exceptions.java:58)
    at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:464)
    at rx.observables.BlockingObservable.single(BlockingObservable.java:341)
    at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClient.java:112)
    ... 132 more
Caused by: java.net.SocketTimeoutException: Read timed out

Update: it might even make sense to add the service key from the RequestContext further up in the "Forwarding Error" error string instead then at least it is always present in error logs.

[ryanjbaxter]

@tmack8001 Can you open a separate issue for your request?

[ryanjbaxter]

@tmelhiser I am unsure what the "distribution of the hosts" has to do with eagerly loading the routes.

[lantenac]

@tmelhiser did you find any solution to your issue ?

[thinksky-sourcecode]

@ryanjbaxter I have the same issue when I set [router-name].ribbon.listOfServers to HTTPS, but it works well on HTTP. Do you have any idea?
e.g.
HTTPS ===> ica.ribbon.listOfServers=https://www.google.com
HTTP ===> ica.ribbon.listOfServers=http://www.google.com

[ryanjbaxter]

@thinksky-sourcecode completely different issue see http://cloud.spring.io/spring-cloud-static/Edgware.SR3/single/spring-cloud.html#spring-cloud-ribbon-without-eureka

[rodrigokuba]

@tmelhiser I had this same issue. I noticed that in your ribbon configuration, the parameters readTimeout and connectTimeout are in lower case.
I tried in upper case (ReadTimeout and ConnectTimeout) and this worked for me.