Caffeine: Wondering why there is no public Optional<V> get(K k)

[landawn]

I'm Wondering why where is no public Optional get(K k) defined. It's it's much more flexible, comparing to _V get(K key, Function mappingFunction)_ or _V getIfPresent(Object key)_ because Optional provides much more operations to handle existing/non-existing keys. for example:
cache.get("key").orElse("default"); cache.get("key").orElseGet("default"); cache.get("key").orElseThrow("default"); cache.get("key").map(...);

[ben-manes]

In a prior thread, I replied with the main reasons, which you might find interesting. A few main points here...

A concern with Optional is that it requires an allocation in the happy path of the value being present. This might go away in JDK10 with value types, but is an expensive choice until then as it puts a lot of pressure on the garbage collector. If one was to store the wrapped optional instead, this adds memory overhead and is incompatible with weak/soft values.

The preference is to compute atomically by using a mapping function or a CacheLoader. Most often you want to load the value into the cache, the api gives an alternative to a racy get-compute-put approach. In general a get(key) computes and most often the value is returned. If the value isn't computable some might return null for the caller to handle, throw an exception (e.g. JAX-RS BadRequestException), or negatively cache by storing a dummy value (perhaps an Optional).

An Optional makes the most sense when your orElse won't feed back into the cache. The difference is then pretty small compared to a null check and becomes stylistic. E.g. you could use Optional.ofNullable(cache.get(key)) or, if lookup only, cache.asMap().get(key, defaultValue).

You could also write a custom Cache interface with your preferred API that delegates to Caffeine, as Scaffeine did. In this way Caffeine should feel like a natural extension to the Collections Framework and those that love/hate Optional shouldn't feel too unhappy.

[landawn]

Here are my opinions:

"puts a lot of pressure on the garbage collector_"...
What drives you to make the this conclusion? Personally, I don't see any pressure on performance/memory by my test/data/experiences.

Secondly, Using the mapping function or CacheLoader to generate the return value when the key is not found in cache, it could be a good idea. but most likely not. Why? Usually it's the application responsibility to handle this scenario when the key is not cached. and there could be different strategies to handle null value even in the same system for the same cache, For example, some call may query the record from db, some call may create a new object, some call may just return 'not found', in other words, the mapping function/CacheLoader won't help developers too much to handle non-cached key.

"...Collections Framework and those that love/hate Optional shouldn't feel too unhappy“
You know, what make me having this question? it's get(k, mappingFunction) and getIfPresent(k), the Java 8 style. I won't ask this question if it's the traditional V get(key). Since Caffeine is targeting to Java 8 and above, Since Optional is widely used in JDK 8 or other libraries, avoiding it only makes the people who love Optional/Java 8 style unhappy. Meanwhile won't make the people who hate Optional happy because they have already been flooded by JDK 8.

[landawn]

I also looked into the main reasons. Except the first one : _Incompatibility with reference caching_...(I don't understand it). My thoughts are pretty simple: Either V get(K key) or Optional<V> get(K key)is perfect to me. But the combination V get(K key, Function<? super K, ? extends V> mappingFunction) and V getIfPresent(K key) doesn't give me the best taste.

[ben-manes]

What drives you to make the this conclusion? Personally, I don't see any pressure on performance/memory by my test/data/experiences.

An Optional requires a heap allocation and could be collected in the young generation. In a JMH benchmark that occurs in the eden space making it very efficient, but on an active server it would likely reach the survivor space before being collected. In the most minimal benchmark,

@State(Scope.Benchmark)
public class OptionalBenchmark {
  static final Integer value = 1;
  @Benchmark public void integer(Blackhole blackhole) {
    blackhole.consume(value);
  }
  @Benchmark public void optional(Blackhole blackhole) {
    blackhole.consume(Optional.of(value));
  }
}

```
Benchmark Mode Cnt Score Error Units
OptionalBenchmark.integer thrpt 10 339097139.113 ± 11957961.677 ops/s
OptionalBenchmark.optional thrpt 10 241541639.809 ± 7378157.795 ops/s

This indicates a 29% penalty on the G1 collector. This doesn't mean you should avoid allocations, but that very hot paths might see a performance boost by being more watchful of GC usage.

> Secondly, Using the mapping function or CacheLoader to generate the return value when the key is not found in cache, it could be a good idea. but most likely not, at least to me. 

Unfortunately we'll have to disagree here because my experience, and the vast amount of code that I've seen internally or on Github, is the opposite. Most local caches have a narrow scope and are encapsulated, so there is only one or two answers given. That is often to memoize the value on a miss (to avoid [dog piling](https://en.wikipedia.org/wiki/Cache_stampede)) or throw an exception if not computable (e.g. 404 for a missing resource). A remote cache, like redis, is more likely to encounter the many different usages and strategies due being a shared collection of many types.

> it's get(k, mappingFunction) and getIfPresent(k), the Java 8 style....

We defined an earlier version of this API in Java 5 for Guava, and reviewed by Josh Bloch. An issue that Charles had was the passive nature of `Map.get()`, which didn't convey side effects or that the data structure might change underneath them. Therefore the `Cache` interface is more active where a `get` corresponds to JDK8's `computeIfAbsent` and the passive calls appear more special case. When you see `cache.getIfPresent` or `cache.estimatedSize` it indicates a warning that the user is inspecting the cache to do work, rather than delegating to it.

JDK8 does not strongly advocate Optional, there was no attempt to retrofit it everywhere, its not highly used like in Scala. Like Guava's, it is useful when conveying the absence in an opaque API call, e.g. querying a time range class where an end date isn't required. Java collections very much rely on nulls as indicators and don't appear destined for a redesign, so I think conforming to their conventions is a positive.

A `Cache.get(k)` could return a value, null (miss), or throw an exception (not computable). There is no Result<V, Exception> type in Java, so `Optional<V>` or an exception might be more awkward. If only used on `getIfPresent`, then it means a mixture of null and Optional which is inconsistent.

But if you find the API is not an ideal fit for your code, then I don't see a drawback for writing a custom facade.

> Incompatibility with reference caching...(I don't understand it)

Weak references can be a convenient strategy to clean-up resources when there are no more strong reference usages. This often when a size or time based policy could result in incorrect behavior, but a usage based policy is correct. There are many scenarios, but a simple generic one would be a dynamic stripe of locks,

```java
Cache<Key, Lock> stripe = Caffeine.weakValues().build(key -> new ReentrantLock());
public void process(Key key) {
  Lock lock = stripe.get(key);
  lock.lock();
  try {
    // Perform some work scoped to the key that cannot be done concurrently
    // This is similar to synchronized (key), except interned to not be instance-based.
  } finally {
    lock.unlock();
  }
}

In this case the value is a weak reference and collected when no one holds the lock. A weak reference of Optional<Lock> wouldn't work, so storing the Optional in the cache could break this use-case.

[ben-manes]

I forgot to show a run of OptionalBenchmark with concurrency, to show a busy system causing GC delays. In that case I added @Thread(8) to the methods.

Benchmark                    Mode  Cnt           Score          Error  Units
OptionalBenchmark.integer   thrpt   10  1659249867.451 ± 23404098.681  ops/s
OptionalBenchmark.optional  thrpt   10   742319002.910 ± 12702515.401  ops/s

In this case there is a 2.2x gain of avoiding allocations. This is slightly unfair because most systems are not cpu-bound and the GC able to mask the penalty by stealing free cycles concurrently. So we know that there is a 1.4-2.2x speedup, though that may not matter too much for most usages.

[landawn]

Just talk about the benchmark, Could you run benchmark for below code?

public String get() {
    return UUID.randomUUID().toString();
}

public Optional<String> optionalGet() {
    return Optional.of(UUID.randomUUID().toString());
}

In most real use cases/scenarios where cache is used, if not all, the load will be much bigger than the code you ran for benchmark above. Probably there is only 0.0...1 speedup, if it's not zero. I think the benchmark you used to draw the conclusion is meaningless.

[ben-manes]

Your benchmark has, as expected, equal runtimes at 430k/s. In that case the computation is limiting the thread and the GC can steal spare cycles, as it is designed to do.

My benchmark wasn't to demonstrate real work, but merely that garbage is created that adds to the system. One hopes that escape analysis could remove it, but that is a finicky optimization. It also wasn't meant to argue against creating garbage, but that some applications struggle with GC churn and good data structures try to minimize their contributions. Similar to why Java's HashMap uses power-of-two sizes for shifts and AND masks rather than multiplication, division, and modulus. Its not an optimization that is critical or should bleed into the API, but an implementation consideration.

As I said originally, its a short-term concern until value types. That's not meant as a strong argument, e.g. as Bloch advises...

Consider the performance consequences of API design decisions, but don’t warp an API to achieve performance gains. Luckily, good APIs typically lend themselves to fast implementations.

The primary arguments are where we disagree. In those aspects, I think you would have to provide more examples to make a stronger case for the proposed API change.

[landawn]

Thanks. You use HashMap as an example, but how about HashSet? back implemented by HashMap and more unnecessary objects (Map.Entry...) are created...
My point is that performance/GC should NOT be a reason of using Optional or not in this API design because the cost of creating/recycling Optional instance is so tiny and it can be totally ignored.
Do you agree?

[ben-manes]

Yes. I don't think performance should be "the" reason to reject an API design, but I do think it should be "a" reason to better scrutinize the options. In this case there are much stronger arguments that, when evaluating, led me against it in that API. It is used in Policy (cache.policy()) where I thought it was the better design choice.

[landawn]

Good.
Then let's talk about Optional<T>. Personally, I disliked it too at the begin because I didn't feel it's as useful/good as what a few people described. After learned more, I found it's a good type to improve the design as long as it's used appropriately and start to use it in my own API design. I strongly agree with the quote: Java 8 Optional: How to Use it

_The JSR-335 EG felt fairly strongly that Optional should not be on any more than needed to support the optional-return idiom only._

As you can see, Optional is adopted in the new introduced APIs in Java 8 (e.g. Stream API). And I think Map.get(K key) is a perfect example where Optional should be used. (But it can't be changed due to backward compatibility). We could continue to debate other items one by one... My point is actually very simple: Optional<V> get(K key) is not necessary to be better than V get(K key) in the design of Cache. But it's a way better than the combination of V get(K key, Function<? super K ,? extends V> mappingFunction) and V getIfPresent(K key).

[ben-manes]

Java's Optional is similar to Guava's, so I've used both quite a bit since being introduced. Both authors try to clarify it should be used judiciously.

V get(key) is provided on a LoadingCache. I don't see Optional<V> get(key) allowing for removal of any of those methods. The get(key, mappingFunction) is equal to Map.computeIfAbsent, so I don't see why it would go away.

Java's Optional is not as rich as in other languages like Scala and Haskell. In those languages it would be the sure idiom but Java it does not fit as neatly into the language's design or the conventions adopted by the community. The Cache APIs try to feel like an extension to Java Collections, which return null values to indicate not present. Returning an Optional is only in Java 8's Collections Framework for a few methods on Stream (e.g. findAny). Those methods are probably infrequently used compared to others. This may change over time, but if so then there is a slow adoption by the language authors and community. So we risk violating one of Bloch's rules,

APIs must coexist peacefully with the platform, so do what is customary. It is almost always wrong to “transliterate” an API from one platform to another.

Because of the language's constraints, there is a penalty for using Optional. To its favor it is explicitly designed to follow Bloch's rule,

Avoid return values that demand exceptional processing.
Clients will forget to write the special-case code, leading to bugs. For example, return zero-length arrays or collections rather than nulls.

However, Java is biased towards a structured programming style rather than functional. This means that it has typically been more readable to use if and for loops than nested function chains. Java 8 removes that for many of the simple cases, e.g. filtering by an anonymous class predicate was less readable than a for-if loop. But there are still holes such as a lack of Optional flattening, pattern matching, and for comprehension. This means developers are coerced into calling Optional.get() to drop back into the richer structured programming style.

That leads to another of Bloch's rules,

You can’t please everyone so aim to displease everyone equally. Most APIs are overconstrained.

In this case, I think the majority of users would be displeased by using Optional here. Their usages of the Cache are encapsulated and they may use Optional at their API boundaries, but want succinct idiomatic implementation logic. Those like you who want Optional are perhaps slightly annoyed at writing a simple adapter to their preferred API. They might also be frustrated enough by Java's limitations to choose a language with stronger support, so a language adapter would be desirable regardless. To be pragmatic we have to gauge both the current users frustrations and the trend, which I think favors displeasing those in your camp.

Unfortunately Guava's Optional has been around since 2011 and neither it nor Java's seem to have caused a big shift in programming styles or usage in their camps. Newer languages like Kotlin, Ceylon, and Dotty (Scala 3), and C# are favoring union types to replace Optional with stronger type information (e.g. nullable types). That might be a better fit for Java.

Overall, I don't think there is a strong argument for or against Optional.

[landawn]

Very Good!
I think there are couple of reasons we don't see the big shift. 1) Keep backward compatibility, that means the existing JDK API can't be modified to adopt it and any libraries which need to support JDK 7 or older version can't use it either. 2) Without the lambdas support, There is no conviction for developer to adopt the functional interfaces/style, including Function/Predicate/Consumer and Optional. for example:

// Traditional for loop 
List<Account> result = new ArrayList<>();
for (Account e : accounts) {
    if (e.getFirstName().equals("jack")) {
        result.add(e);
    }
}

// Stream API without Lambdas
result = accounts.stream().filter(new Predicate<Account>() {
    @Override
    public boolean test(Account account) {
        return account.getFirstName().equals("jack");
    }            
}).collect(Collectors.toList());

// Stream API with Lambdas
result = accounts.stream().filter(e -> e.getFirstName().equals("jack")).collect(Collectors.toList());

I saw the some libraries which only support Java 8 and above start to use Optional and other Function APIs more and more. (e.g. Spring 5) and developers are writing more and more functional style code with Java 8.

[ben-manes]

Have you seen Optional pipeline terminators used very often? (min, max, findAny, findFirst, and reduce) I haven't, and these methods was the reason for Java adding the new type. It seems that Optional is finding its way into application code mostly because developers are exploring it. How API designs evolve will be interesting to watch, because it could be that the other JDK8 features are adopted heavily, but Optional is used sparingly.

Scala's Optional is very nice to work with, but a lot of that is missing here. Theirs is Iterable to be a zero or one sized collection, flattens, matches, etc. The code comes out very neat with fewer ifs and loops. Java's hasn't been as pleasant for me.

[ben-manes]

Unless there is additional feedback, I am planning on closing this issue. I don't think this would introduced into the API. But happy to leave open if there is opposition favoring a reconsideration.