Sdk: Legitimate use case to detect if a Future is completed.

[marcglasberg]

I'm the creator of https://pub.dev/packages/async_redux and we let devs create sync or async reducers depending on the return type of their reducer:

FutureOr<AppState> reduce();

In other words, they may return AppState or Future<AppState>, and I process it like this:

var result = action.reduce();

if (result is Future<St>) {
  return result.then((state) => registerState(state));
} else if (result is St) {
  registerState(result);
}

This works fine, unless they try to return a completed Future. For example, this is WRONG:

Future<AppState> reduce() async { return state;}

The solution is returning only Future which are NOT completed, for example:

Future<AppState> reduce() async { await someFuture(); return state;}

Which is totally fine. The reason completed futures don't work is because the new returned state must be registered in the same microtask. If it waits even a microtask, the state may get changed by some other reducer, and that previous state will be written over with stale data.

In other words, I need the then to be executed in the same microtask of the returned value, which ONLY happens if the Future returned by the reducer is NOT completed.

I read here https://github.com/dart-lang/sdk/issues/14323 the following:

Another argument against immediate callback on listen is that it gives you two programming models - immediate callback and eventual callback. You can write code that assumes immediate callback, and that breaks if the callback comes later. And you can write code that assumes a later callback, and breaks if the callback comes immediately (which did happen in the beginning). (...) Sticking with one model: callbacks are always later, working with futures becomes much simpler.

This is all fine, but the problem is: Callbacks are always later, yes, but you can never be sure that the returned value is within the same microtask or not. You made it easy to reason about when the callback is called (later), but you made it impossible to reason about if the value it is getting is recent or stale. If you assume it's from the same microtask you may be wrong, and if you assume it's from the previous microtask you also may be wrong. There's absolutely no way of knowing, so you don't know if that returned value is stale or not.

At the moment I have to trust the developers to do the right thing, and there is absolutely no way for me to know if they did something wrong.

What I need to do is this:

if (result is Future<St>) {
  if (result.isComplete) throw AssertionError("Don't return completed Futures.");
  return result.then((state) => registerState(state));
}

Is there any reason why _isComplete is private in _Future in future_impl.dart?

I'd like to ask this information to be made public, for the sake of us, framework developers. That's not a breaking change, and that's not a feature which can be abused. It's just a way to issue a warning when a completed Future is not acceptable.

[lrhn]

A listener on a future is always notified in a later microtask than the one where it was registered.
It may or may not be notified in the same microtask where the future completes (a sync completer can complete it immediately, an async one will do so in a later microtask). The future returned by an async function is currently completed synchronously if the function returns a value, at least if that happens after the first await - if there is no previous await, then the completion is actually delayed. Also, there is no promise that the future is completed synchronously, so depending on it is not something I recommend.

Futures are asynchronous. Relying on them for synchronous communication is not a good idea.

Example:

import "dart:async";
main() async {
  f() async {
    await 0;
    scheduleMicrotask(() {
      print("mt");  // prints second
    });
    return "ar";  // prints first
  }
  f().then(print);
  // Flush microtask queue.
  await new Future(()=>0);
  g() async {
    scheduleMicrotask(() {
      print("mt2");  // prints third
    });
    return "ar2";  // prints fourth
  }
  g().then(print);
}

So you are saying that you depend on futures being completed synchronously, which is not something the Future class actually promises. There are many situations where it's not true, even for downstream handlers of a sync completer future (for example if one future handler throws, then all other completed futures are postponed to a later microtask).

As for _isComplete (and getting the value if the future is complete with a value), it is very deliberate that it is not available in the Future API. The incentives are such that many, many users would be tempted to use it. We really, really don't want that. Not only does it make the class harder to use properly (you shouldn't use isComplete and value, but the incentives are wrong for that), it also makes people more dependent on the precise timing. Some would start assuming that the future is complete at some point, because it almost always is, except that one time where something got postponed to a later microtask for some unrelated reason, and the code breaks.

Even if we wanted to add something to Future, it's not practically possible. There are many, many implementations of Future in the wild, some mocks, some wrappers, some I have no idea what does. If we add anything to the Future interface, all those classes will become invalid. We can't do that.

The one way to access a future's value is to add a listener and wait for the value. That value may come at any later time, and assuming differntly is unsound. That is the provided API.

What you can do, if you really want a future with a synchronous access to the value, is to wrap the future:

import "package:async/async.dart";
class FutureValue<T> implements Future<T> {
  Future<T> _future;
  Result<T> _result;
  FutureValue(Future<T> future) : _future = future {
    Result.capture(_future).then((result) { _result = result; });
  }
  bool get isComplete => _result != null;
  T get value => _result.asValue?.value ?? (throw _result.asError.error); 
  // Forward all future functions
  R then<R>(FutureOr<R> action(T value), {Function onError}) => 
      _future.then<R>(action, onError: onError);
  ...
}

If you wrap the Future immediately you get it, then it will complete as you expect. You can't do this for other people, but you can make sure the futures you return are wrapped.

Or, if you don't want to wrap the future, you can register it instead:

class FutureValueRegister<T> {
  Expando<Result<T>> _registry = Expando();
  void register(Future<T> future) {
    Result.capture(future).then((result) { _registry[future] = result; } );
  }
  Result<T> resultOf(Future<T> future) => _registry[future];
}

Then you can always check whether a registered future has completed yet, and with what.

(There have been previous requests for something like this, e.g., https://github.com/dart-lang/sdk/issues/29026)

[marcglasberg]

Lasse, thanks a lot for your fast and precise answer, I appreciate it.

You say: "That value may come at any later time, and assuming differntly is unsound."
I agree, and that's what I'm doing. I don't care if the value comes right away or takes some time to arrive. I actually don't need a future with a synchronous access to the value.

All I'm asking is a way to KNOW if the value I'm getting may be STALE or not.
It may be stale if there was a microtask between the return and the then.
What you are basically saying is that I should never assume to get a value which may NOT be stale.
Which is the same as saying I must always assume to get potentially stale values.
That's kind of terrible. It means I can never trust a value returned by a future to be still valid
when I get it, even if it was guaranteed to be valid when the Future returned it.

Currently you not only don't know when the information will arrive, but you also don't know if its current. The only thing you can say is that that information was valid at some point in the past.

I see this as unrelated to the question of assuming that the value may come at any later time,
which, as I said, I don't care about.

I also understand when you say that getting the value if the future is complete may present the wrong incentives. But that's also unrelated to KNOWING if the Future is already complete. If you know that
but you can't get the value, there's nothing you can do, and there are no wrong incentives in this case.
You still have to wait for it to arrive later.

The rule to add something to any interface is that old code is guaranteed to run just the same,
without any modifications. That's the case with adding the a getter with the following contract
to the Future interface:

/// May return true if the Future is completed, and false otherwise.
/// Return null if the information is unavailable, or if the specific Future subclass
/// that implements this interface doesn't disclosure this information.
bool get isCompleted => null;

In other words, Futures would not be forced to return this value, unless they declare they will do so, in their contract. This is useful because of async/await. The regular Futures returned by functions marked
"async" are the important ones, because frameworks will always let devs use this one to have a clean API. This is the only one that would need to implement this interface.

Now, regarding your FutureValue class, could you please help me understand it a little bit better?
You say: "If you wrap the Future immediately you get it, then it will complete as you expect".

I'm getting the Future from an async function:

class Action {
   Future<AppState> reduce() async => return state;
}

var result = action.reduce();

Are you saying that I should do this?:

var result = FutureValue(action.reduce());

Would that make sure the state returned by reduce() would be given synchronously to result.then?
Even if reduce() is returning a completed Future?

[natebosch]

What you are basically saying is that I should never assume to get a value which may NOT be stale.

There is no notion of a "stale" value from a Future. A Future only ever completes with one value, and that value has no notion of stale or fresh - it is always the value that was used to complete the Future.

The only thing you can say is that that information was valid at some point in the past.

This is intentional. As far as I know we don't make any guarantees that a callback on a Future is invoked in the same microtask as the completion of the Future, regardless of whether it was added to a "completed" future or not.

In other words, you are trying to solve a different problem than Future is trying to solve. If you need to have a way to communicate a value which may come later, but where the callback needs to be synchronous to when the value is available, you'll likely need a different concept than a future.

I think this is what Lasse meant when he said "Futures are asynchronous. Relying on them for synchronous communication is not a good idea."

If you are using Futures, don't rely on any particular interleaving of execution other than what is explicit in your callbacks.

[marcglasberg]

Thanks for your answer, Nate. The notion of "stale" value is not from the Future class. It's a word which means something in the real world. It also is not necessarily related to synchronous communication. It's stale if it's not the most recent info that the application has. It means you cannot use it without risking overwriting a more recent value. You actually can guarantee that with async communications, generally, but not with Dart Futures.

Futures having or not having the notion of "stale" is irrelevant. It just means a Dart Future, as defined, cannot be used when you need to make sure you are using the most recent value. It means it's useless in that situation, because it will only work 99,9% of the time, say. Now, you may be OK with that, but it means Dart Futures can't be used like Java Futures and C# Futures to do stuff like dealing with important financial data, and lots of other applications that must make sure you apply the information in the exact order you receive them, even asynchronously. If you write your own Futures it's OK. But if you get them from client code (or from a colleague's code) you can't guarantee freshness anymore. That may be OK for UI code, Flutter and stuff, but picking Dart for server software doing financial stuff, for example, would be a bad decision.

This is unfixable, since you can't really avoid Futures. You gave this advice: I should avoid Futures, which I appreciate, but that's not a realistic advice. You get Futures from functions declared with async, and you can't avoid async/await in practice. Packages in pub will use Futures, and I can't try and find some "Futureless" version of them.

Given time, languages are used to do what they do well, and I guess Dart will be used for UI, but won't be used to do stuff that needs to guarantee information freshness. Dart is that language which Futures cannot be used to do this and that, because of its limitations created not to foster the wrong incentives (Just to make it clear, I agree that avoiding wrong incentives is a good thing, but I think the price being paid is a Future which is useless in some real world situations which are taken for granted in some other popular languages. Maybe you could have created hurdles for using Futures in ways you don't want it to be used, but still allowed some backdoor to those uses).

I truly appreciate you guys taking the time to answer me, thanks once again.

[lrhn]

You generally cannot assume that any value remains current over an asynchronous boundary. Dart interleaves code, so code can always happen between the return statement of an async function and the receiver getting that value.

I would recommend not passing the value, but a notification that there is a value. The receiver can then either check for intermediate changes when it gets the notification, or it can fetch the most recent value directly.

For example, if you always keep the most recent value in a variable mostRecent, then instead of just returning the current value of mostRecent, you could return a ChangeNotification:

class ChangeNotification<T> {
  T _triggerValue;
  T Function() _currentValue;
  ChangeNotification(this._triggerValue, this._currentValue);
  T get value => _triggerValue;
  T get mostRecentValue => _currentValue;
  bool get isStale => _triggerValue != _currentValue();
}

Then you do return ChangeNotification<int>(mostRecent, () => mostRecent); and the user getting the change can choose to use the value at notification start, or the current value when the notification is received, or even check if the value has changed.
Or you can be even more clever and keep a change counter, so you can see that the value is stale, even if it has changed back to the same value again (if that matters, say because you do nothing for stale values because you know there is a more recent ChangeNotification underway with the newest value).

Or, if ordering is important, have a queue and send notifications saying that there are new events in the queue, but not the values. Then handlers must take the events in the correct order.