Guava: Retry tool

[gissuebot]

_Original issue created by Nerther on 2010-12-03 at 02:00 AM_

---

Hello

I implement a retry tool which can retry task(wrapped in Callable) easily, whether it's valuable to be added in guava library?

Only 2 classes: Retry, RetryBuilder.

Example:

Retryer retryer = new RetryerBuilder()
   .times(3)
   .interval(10, SECONDS)
   .when(timeout())
   .build();

Response response = retryer.callWithRetry(new Callable<Response>() {
   public Response call() throws Exception {
      return removeService.upload(request);
   }
});

response.doSomething();
Retry condition:

private static Predicate<Exception> timeout() {
   return new Predicate<Exception>() {
      public boolean apply(Exception exception) {
         if (exception instanceof TimeoutException) {
            return exception.getMessage().startsWith("Retryable");
         }
         return false;
      }
   };
}

Project url : http://code.google.com/p/google-guava-retryer/

All java sources & test has been attached.

[gissuebot]

_Original comment posted by Java2Enterprise on 2010-12-03 at 02:14 AM_

---

retryer.retry is more concise than retryer.callWithRetry :)

[gissuebot]

_Original comment posted by Nerther on 2010-12-03 at 02:28 AM_

---

I refer to the interface TimeLimiter of guava when design the signature, the usage and structure of which is similar to Retryer, the method is named callWithTimeout, it is simple and clear.

[gissuebot]

_Original comment posted by [email protected] on 2011-01-12 at 10:37 PM_

---

FYI, we are experimenting with one or two approaches internally at the moment.

---

Status: Accepted
Labels: Type-Enhancement

[gissuebot]

_Original comment posted by [email protected] on 2011-07-13 at 06:18 PM_

---

_(No comment entered for this change.)_

---

Status: Triaged

[gissuebot]

_Original comment posted by [email protected] on 2011-07-16 at 08:32 PM_

---

_(No comment entered for this change.)_

---

Status: Accepted

[gissuebot]

_Original comment posted by Nerther on 2011-08-18 at 09:09 AM_

---

Update sources!

[gissuebot]

_Original comment posted by cgdecker on 2011-11-20 at 04:30 PM_

---

_Issue #797 has been merged into this issue._

[gissuebot]

_Original comment posted by [email protected] on 2011-12-10 at 03:58 PM_

---

_(No comment entered for this change.)_

---

Labels: Package-Concurrent

[gissuebot]

_Original comment posted by jnizet on 2011-12-25 at 08:42 AM_

---

Sorry for the successive posts. My earlier design had serious issues, so I'm reposting it.

I had the same idea, and built a solution which is a bit more complex, but also more configurable. The differences are:
 - configurable stop strategy (number of times, delay, or custom)
 - configurable wait strategy (fixed delay, random delay, incrementing delay, or custom)
 - different exception handling

Usage example:

Retryer<Response> retryer =
    RetryerBuilder.<Response>newBuilder()
                  .withStopStrategy(StopStrategies.stopAfterAttempt(4))
                  .withWaitStrategy(WaitStrategies.fixedWait(1L, TimeUnit.SECONDS))
                  .retryIfRuntimeException()
                  .retryIfExceptionOfType(IOException.class)
                  .retryIfResult(Predicates.<Response>isNull())
                  .build();
try {
    return retryer.call(callable);
}
catch (ExecutionException e) {
    // the call threw a checked exception which didn't cause a retry
    // encapsulated in the execution exception
    Throwables.propagateIfPossible(e.getCause(), SomeCheckedException.class);
    throw new RuntimeException("unexpected", e.getCause());
}
catch (RetryException e) {
    // the retry was aborted because the call didn't succeed
    // it's also possible to get the last attempt result or exception
    // from the RetryException
    throw new RuntimeException("Call never succeeded", e);
}

[gissuebot]

_Original comment posted by wasserman.louis on 2011-12-25 at 12:42 PM_

---

This is clearly a nontrivial problem. I am curious if this is the best way to solve it...

[gissuebot]

_Original comment posted by adrian.f.cole on 2012-01-04 at 10:09 PM_

---

We are currently using this for years:
https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/predicates/RetryablePredicate.java
https://github.com/jclouds/jclouds/blob/master/core/src/test/java/org/jclouds/predicates/RetryablePredicateTest.java

It has limitations including no support for non-trivial shapes (ex. wait initially 10 seconds, then increase period over time), and sometimes you want to receive both the boolean result as well the last result tested. The latter functionality is sketched here:

https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/predicates/Retryables.java
https://github.com/jclouds/jclouds/blob/master/core/src/test/java/org/jclouds/predicates/RetryablesTest.java

Clearly, we'd prefer something like this in guava as opposed to our codebase, so really excited about progress we can make here.

[gissuebot]

_Original comment posted by adrian.f.cole on 2012-01-04 at 10:11 PM_

---

personally, I like the customizability of comment 11

[gissuebot]

_Original comment posted by raymond.rishty on 2012-01-18 at 09:09 PM_

---

It might be nice if the "Retryer" were a RetryingExecutorService and the response a ListenableFuture

[gissuebot]

_Original comment posted by [email protected] on 2012-01-18 at 09:19 PM_

---

FYI, internally, we have:

• RetryingCallable (similar to the original suggestion and comment 11)
• RetryingFuture (repeated calls to a Supplier<Future>)
• Retry (bytecode magic to turn an object into a retrying proxy; unlikely to be open-sourced due to its dependencies)
• RetryingExecutor (still under review; similar to comment 15)

There is clearly demand, but there are a lot of possible approaches and a lot of work left to do, so I recommend using one of the existing solutions in the meantime.

[gissuebot]

_Original comment posted by adrian.f.cole on 2012-05-15 at 06:33 AM_

---

Here's an addition to add the list, resulting from jclouds proliferation of LoadingCache :)

   // deal with eventual consistency delay between bucket resources and their acls
   CacheLoader<String, AccessControlList> loader = RetryingCacheLoaderDecorator.newDecorator()
        .on(ResourceNotFoundException.class).exponentiallyBackoff()
        .decorate(
            new CacheLoader<String, AccessControlList>() {
               @Override
               public AccessControlList load(String bucketName) {
                  return client.getBucketACL(bucketName);
               }

               @Override
               public String toString() {
                  return "getBucketAcl()";
               }
            });

[gissuebot]

_Original comment posted by [email protected] on 2012-05-15 at 07:02 AM_

---

+1 for comment 11 and 17's approach. I'm not sure I like the particulars of the builder pattern though, I'd like to save/serialize the builder so I could use it later.

.withStopStrategy() and .withWaitStrategy() could probably have variants directly in the builder, although having the ability to wire in your own strategies is very useful.

[gissuebot]

_Original comment posted by [email protected] on 2012-05-30 at 07:43 PM_

---

_(No comment entered for this change.)_

---

Labels: -Type-Enhancement, Type-Addition

[gissuebot]

_Original comment posted by schnitzi on 2013-02-04 at 05:15 AM_

---

This would be useful for me too, but so as not to make this a "me, too" post, here are a few links to others who have implemented this sort of thing, which can be used reference for a Guava version (which is what I'd prefer):

https://github.com/rholder/guava-retrying - not Guava but built on top of Guava; hence the name

http://grepcode.com/file/repo1.maven.org/maven2/org.springframework.batch/spring-batch-infrastructure/1.0.0.FINAL/org/springframework/batch/retry/ - one from the Spring framework

[gissuebot]

_Original comment posted by toellrich on 2013-05-03 at 06:15 AM_

---

There's also a .Net API called the transient fault handling application block which does the same thing:
http://msdn.microsoft.com/en-us/library/hh680905(v=pandp.50).aspx

[gissuebot]

_Original comment posted by [email protected] on 2013-05-11 at 12:43 AM_

---

I have implemented both approaches:

1) simple: http://code.google.com/p/spf4j/source/browse/trunk/src/main/java/org/spf4j/base/Callables.java

this implementation allows you to retry a callable based on Exception or returned result, and execute a callable before retry.

public static <T> T executeWithRetry(final Callable<T> what, final Callable<Boolean> doBeforeRetry,
        final Predicate<? super T> retryOnReturnVal, final Predicate<Exception> retryOnException)
        throws InterruptedException 

2) sophisticated:http://code.google.com/p/spf4j/source/browse/trunk/src/main/java/org/spf4j/concurrent/RetryExecutor.java

this is a executor based implementation that allows you to use your threads more efficiently.

[gissuebot]

_Original comment posted by loisel.jerome on 2013-06-05 at 07:14 AM_

---

I have tried an implementation too, trying to keep it simple:

https://github.com/jloisel/retrying-callable

I like the approach in #11 but i felt the need to separate concerns between retry on exception and retry on returned result.

[akormushin]

Aren't you gonna merge this one? Waiting since 31 Oct 2014 and highly anticipated