the wasm-bindgen-futures crate provides for consuming promises as futures, and returning futures to JS as promises, but not executing futures in wasm.
I'm currently experimenting with borrowing the strategy for passing futures to js for my use-case.
I'm not sure it's actually possible to execute futures in wasm without the support of JS? Executing futures involves some degree of blocking but JS/wasm can't block right now I think? Do you have a particular API in mind though for this?
I'm playing with creating an indexeddb library. Eventually, I'd like to make it integrate into a virtual DOM library so an action like a click is processed into an immediate DOM update (loading), and a delayed DOM update (once the promises have resolved).
+------------+ +------------+
|click action+----->future chain|
+-----+------+ +-----+------+
| pending |
| | future chain is complete
+-----v------+ |
|state change<-----------+
+------------+
You should be able to get this behavior by specifying your state, and functions describing how the state changes when the future is ready or rejected. This applies equally to fetch as much as indexedDB. The beauty of futures is that you can have quite a complex operation with many steps but treat it as a single atomic operation.
I'm just struggling to get my head around how js and wasm interact here. Do we need an executor in wasm using something like setTimeout(() => {..}, 0), or can we arrange to be woken up by the JS when a promise completes.
I always find the lack of determinism in js a bit frustrating (w.r.t. the order that things will happen in), and I feel like futures would help me get a handle on it, but the big picture is just eluding me.
I'm thinking I probably have to pass the function I want to run into the promise, so that JS knows what code to run. Maybe this means I can't use futures (at least not the standard rust ones).
An example of the code I'd like to be able to write: (pseudo code)
#[derive(Default, Debug, ...)]
struct State {
name: Option<String>,
age: Option<u32>,
loading: bool,
}
enum Action {
LoadData,
LoadFailed { reason: String },
LoadSucceeded {
name: String,
age: u32
}
}
fn merge(action: Action, state: &mut State) {
match action {
Action::LoadData => {
state.loading = true;
}
Action::LoadFailed { reason } => {
eprintln!("loading failed: {}", reason);
}
Action::LoadSucceeded { name, age } => {
state.name = Some(name);
state.age = Some(age);
state.loading = false;
}
}
}
fn on_click() {
state.merge(Action::LoadData);
indexeddb::open("my_db", 1)
.then(|db| db.start_transaction())
.then(|(db, trans)| db.object_store("people"))
.then(|store| store.get(0))
.then(|record| {
state.merge(Action::LoadSucceeded {
name: record.name,
age: record.age
});
}); // todo pass this future somewhere so it gets run.
}
The way that this is handled in stdweb is that it provides a spawn_local function, which lets you spawn a Future:
spawn_local(some_rust_future);
You can then write your code like this:
fn on_click() {
state.merge(Action::LoadData);
spawn_local(indexeddb::open("my_db", 1)
.then(|db| db.start_transaction())
.then(|(db, trans)| db.object_store("people"))
.then(|store| store.get(0))
.then(|record| {
state.merge(Action::LoadSucceeded {
name: record.name,
age: record.age
});
}));
}
Or even easier with async/await!:
fn on_click() {
state.merge(Action::LoadData);
spawn_local(async {
let db = await!(indexeddb::open("my_db", 1))?;
let trans = await!(db.start_transaction())?;
let store = await!(db.object_store("people"))?;
let record = await!(store.get(0))?;
state.merge(Action::LoadSucceeded {
name: record.name,
age: record.age
});
});
}
Internally spawn_local uses a Rust dequeue of Tasks, and it uses JavaScript's microtask queue to delay the Future execution by one tick.
The same sort of function should be implementable in wasm-bindgen as well.
@Pauan do you have to be running an event loop for that to work?
@derekdreery In asynchronous code, an event queue is mandatory (though it's hidden from the user).
JavaScript provides two built-in event queues: macrotask and microtask (Promises always use the microtask event queue). You can read more here:
https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/
So the question is whether spawning Futures directly uses the JS microtask queue, or whether it uses a Rust dequeue.
Using the JS microtask queue is a lot simpler, but using a Rust dequeue is multiple orders of magnitude faster.
Here is a very old stdweb Executor which doesn't use a Rust dequeue, it spawns Futures directly using the JS Promise microtask queue.
It was written for Futures 0.1, it's quite old, probably buggy, and quite slow, but it is also simple and easy to understand, so hopefully it's educational.
All of this event queue stuff is an implementation detail: the user just calls spawn_local and at some unknown point in the future the Future will run.
I say "unknown point", but it's usually very fast (a few milliseconds at most). The "unknown" part just means you can't rely upon it running at a deterministic point in time, since it's asynchronous.
Thanks for the microtask reading - I feel like I understand all this stuff much better now!
I am currently writing a small HttpRequest crate and ended up creating a wrapper struct with a custom Dropimplementation.
pub struct Request<T: Future + 'static = JsFuture>(Option<T>);
/// snip ..
impl<T: Future + 'static> std::ops::Drop for Request<T> {
fn drop(&mut self) {
if let Some(future) = self.0.take() {
future_to_promise(future.and_then(|_| {
future::ok(JsValue::null())
}).or_else(|_| {
future::err(JsValue::null())
}));
}
}
}
Now the browser is doing most of the work.
Usage:
Request::new(Method::Get, "example.org/test")
.header("Accept", "text/plain").send()
.and_then(|resp_value: JsValue| {
let resp: Response = resp_value.dyn_into().unwrap();
resp.text()
})
.and_then(|text: Promise| {
JsFuture::from(text)
})
.and_then(|body| {
println!("Response: {}", body.as_string().unwrap());
future::ok(())
});
@lcnr Running an asynchronous Future in the Drop impl seems pretty hacky to me.
If the goal is to cancel a fetch, you should use the standard AbortController, something like this:
pub struct Request<'a> {
url: &'a str,
init: RequestInit,
}
impl<'a> Request<'a> {
pub fn new(url: &'a str) -> Self {
Self {
url,
init: RequestInit::new(),
}
}
pub fn send(mut self) -> RequestFuture {
let controller = AbortController::new().unwrap();
let init = self.init.signal(Some(&controller.signal()));
let future = window().unwrap().fetch_with_str_and_init(self.url, &init).into();
RequestFuture {
controller,
future,
}
}
}
pub struct RequestFuture {
controller: AbortController,
future: JsFuture,
}
impl Drop for RequestFuture {
#[inline]
fn drop(&mut self) {
self.controller.abort();
}
}
impl Future for RequestFuture {
type Output = Result<JsValue, JsValue>;
#[inline]
fn poll(self: Pin<&mut Self>, waker: &LocalWaker) -> Poll<Self::Output> {
self.future.poll_unpin(waker);
}
}
Untested, but it should be close to correct.
If your goal is instead to workaround the lack of spawn_local (or similar)... that's a very hacky solution.
Instead the correct solution is for us to add in spawn_local, which you would use in your main function.
@Pauan I am not even sure what exactly I am doing. :laughing: I want to make requests during which I return control of the main thread back to to browser/javascript and don't help to store the future anywhere. This means I am currently using a global state.
If your goal is instead to workaround the lack of spawn_local (or similar)... that's a very hacky solution.
jup
I want to make requests during which I return control of the main thread back to to browser/javascript and don't help to store the future anywhere.
That happens naturally as part of the Promises/Futures system, you don't need to do anything special for that.
The best thing to do is to have Request::send return a Future (similar to what I showed in my previous post), and then you can use future_to_promise to spawn it in main:
#[wasm_bindgen(start)]
pub fn main() {
future_to_promise(
Request::new(Method::Get, "example.org/test")
.header("Accept", "text/plain").send()
.and_then(|resp_value: JsValue| {
let resp: Response = resp_value.dyn_into().unwrap();
resp.text()
})
.and_then(|text: Promise| {
JsFuture::from(text)
})
.and_then(|body| {
println!("Response: {}", body.as_string().unwrap());
future::ok(JsValue::UNDEFINED)
})
);
}
No global state needed.
And then when spawn_local is supported you can just replace future_to_promise with spawn_local.
It's even nicer with async/await:
#[wasm_bindgen(start)]
pub fn main() {
future_to_promise(async {
let resp_value = await!(
Request::new(Method::Get, "example.org/test")
.header("Accept", "text/plain").send()
)?;
let resp: Response = resp_value.dyn_into().unwrap();
let body = await!(JsFuture::from(resp.text()?))?;
println!("Response: {}", body.as_string().unwrap());
Ok(JsValue::UNDEFINED)
});
}
Naturally you don't want to put everything into main, but that's not a problem: you can just split your program into multiple async functions:
async fn get_text(url: &str) -> Result<String, JsValue> {
let resp_value = await!(
Request::new(Method::Get, url).header("Accept", "text/plain").send()
)?;
let resp: Response = resp_value.dyn_into().unwrap();
let body = await!(JsFuture::from(resp.text()?))?;
Ok(body.as_string().unwrap())
}
async fn do_something() -> Result<(), JsValue> {
let body = await!(get_text("example.org/test"))?;
println!("Response: {}", body);
Ok(())
}
#[wasm_bindgen(start)]
pub fn main() {
future_to_promise(async {
await!(do_something())?;
Ok(JsValue::UNDEFINED)
});
}
When you use await! on a Promise, it will yield control to the browser.
So in the above example, await!(Request::new(Method::Get, url).header("Accept", "text/plain").send()) and await!(JsFuture::from(resp.text()?)) will yield control.
await!(get_text("example.org/test")) and await!(do_something()) also yield control, since they call get_text (and get_text uses await! on Promises).
Using async/await in Rust is similar to async/await in JavaScript.
P.S. async/await support will require #1105 to be fixed first (or I guess you can use the 0.1 to 0.3 Futures compatibility shim to make it work?).
Even without async/await, my point about using future_to_promise in main still stands.
FWIW JS at the fundamental level can't block, so it's almost always queueing up callbacks to execute at some later date. If you do blocking work at the base level there's likely some callback that gets invoked when the operation is finished (either successfully or not). In that sense we can queue up callbacks to run on events, and those callbacks could drive another event queue in Rust (much like futures work today with tokio and such).
Some of this may belong in the wasm-bindgen-futures crate, but otherwise much of this is largely stock futures and other crates which in theory already work.
Are there still points though that want to be clarified before closing this?
@alexcrichton what do you think about @Pauan's suggestion to add a spawn_local method to wasm-bindgen-futures?
It may be a good idea! I'll admit though that I don't fully understand the motivation after skimming over this issue again. Could you remind me the motivation though for adding a function like that?
(er also good to mention the context that future_to_promise is sort of like spawn_local, but it seems to me like spawn_local would likely have different performance characteristics, so I'm unclear if it's just that or if it also adds expressivity)
@alexcrichton It's just a way to spawn a Future.
So it is indeed almost identical to future_to_promise, except it doesn't return a Promise, so as you say it can be faster.
In particular, it would have this signature:
pub fn spawn_local<F>(future: F) where F: Future<Output = ()> + 'static
I'm assuming Futures 0.3 (it'll be a bit different with Futures 0.1)
This makes it clear to any readers what it is doing, compared to future_to_promise.
Ok just wanted to confirm. That seems reasonable to me to add to wasm-bindgen-futures!
I'll have a go at implementing the Queue and see what it looks like.
Very naive implementation in https://github.com/rustwasm/wasm-bindgen/pull/1148.
Trying to understand where things are at for a newbie perspective... forgive me if I got the terminology wrong
Does wasm_bindgen have a native Future _reactor_ - i.e. a way to run and consume Futures without going through Promises?
Is there an example somewhere of wrapping a js_sys::Function into a Future? At a glance I think all that needs to be done is implement the Poll trait - but I'm not 100% clear on that, and it'd be helpful to see an example that takes some of the ownership issues into consideration.
Specifically, my current goal is to to wrap, say HtmlImageElement's onload/onerror and return a Future<HtmlImageElement,JsValue> when it's ready. Happy to dive in with the examples and figure it out, but I've got some doubts that I'd be doing it right and it'd be helpful to see a pro example
Thanks again!
Caveat emptor I'm not a genius this may be incorrect:
resource, that is a base future that other futures are built on top of. It is the bottom future's responsibility to tell the executor when further progress can be made. The way you do this is (in the poll method) call let handle = task::current() to get a handle, and then call handle.notify() when you can make progress. Here's an example for indexeddb I'm working onimpl Future for IdbOpenDbRequest {
type Item = Db;
type Error = JsValue;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
use web_sys::IdbRequestReadyState as ReadyState;
match self.inner.ready_state() {
ReadyState::Pending => {
let success_notifier = task::current();
let error_notifier = success_notifier.clone();
// If we're not ready set up onsuccess and onerror callbacks to notify the
// executor.
let onsuccess = Closure::wrap(Box::new(move || {
success_notifier.notify();
}) as Box<FnMut()>);
self.inner
.set_onsuccess(Some(onsuccess.as_ref().unchecked_ref()));
self.onsuccess.replace(onsuccess); // drop the old closure if there was one
let onerror = Closure::wrap(Box::new(move || {
error_notifier.notify();
}) as Box<FnMut()>);
self.inner
.set_onerror(Some(&onerror.as_ref().unchecked_ref()));
self.onerror.replace(onerror); // drop the old closure if there was one
Ok(Async::NotReady)
}
ReadyState::Done => match self.inner.result() {
Ok(val) => Ok(Async::Ready(Db {
inner: val.unchecked_into(),
})),
Err(_) => match self.inner.error() {
Ok(Some(e)) => Err(e.into()),
Ok(None) => unreachable!("internal error polling open db request"),
Err(e) => Err(e),
},
},
_ => panic!("unexpected ready state"),
}
}
}
in my case I have a ready_state function I can call to see if I'm ready to make progress. If you only have the result in the callback, you can use a futures::unsync::oneshot to send the result to the original future, so you'd do a rx.receive() and if there is no data yet, then return Async::NotReady.
@dakom
Does wasm_bindgen have a native Future reactor - i.e. a way to run and consume Futures without going through Promises?
Currently no, and executing Futures requires scheduling them on the JS microtask event loop, so it is necessary for it to internally use Promises (or another technique like MutationObserver).
However, once spawn_local is added it will be easy to spawn Futures without worrying about the internal details of Promises.
Specifically, my current goal is to to wrap, say HtmlImageElement's onload/onerror and return a
Future<HtmlImageElement,JsValue>when it's ready.
Normally you would use the oneshot channels for this.
First, let's make it easier to create event listeners:
use web_sys::EventTarget;
use wasm_bindgen::JsCast;
use wasm_bindgen::convert::FromWasmAbi;
pub struct EventListener<'a, A> {
node: EventTarget,
kind: &'a str,
callback: Closure<FnMut(A)>,
}
impl<'a, A> EventListener<'a, A> where A: FromWasmAbi + 'static {
#[inline]
pub fn new<F>(node: &EventTarget, kind: &'a str, f: F) -> Self where F: FnMut(A) + 'static {
let callback = Closure::wrap(Box::new(f) as Box<FnMut(A)>);
node.add_event_listener_with_callback(kind, callback.as_ref().unchecked_ref()).unwrap();
Self {
node: node.clone(),
kind,
callback,
}
}
}
impl<'a, A> Drop for EventListener<'a, A> {
#[inline]
fn drop(&mut self) {
self.node.remove_event_listener_with_callback(self.kind, self.callback.as_ref().unchecked_ref()).unwrap();
}
}
Now you can use oneshot::channel:
use web_sys::{HtmlImageElement, UiEvent};
use futures::Poll;
use futures::sync::oneshot::{Receiver, channel};
pub struct Image {
img: Option<HtmlImageElement>,
_on_load: EventListener<'static, UiEvent>,
receiver: Receiver<HtmlImageElement>,
}
impl Image {
pub fn new(width: u32, height: u32, url: &str) -> Self {
let (sender, receiver) = channel();
let img = HtmlImageElement::new_with_width_and_height(width, height).unwrap();
img.set_src(url);
let _on_load = EventListener::new(&img, "load", {
let mut sender = Some(sender);
let img = img.clone();
move |_| {
sender.take().unwrap().send(img.clone()).unwrap();
}
});
Self { img: Some(img), _on_load, receiver }
}
}
impl Drop for Image {
#[inline]
fn drop(&mut self) {
if let Some(ref img) = self.img {
// Cancels the image download
img.set_src("");
}
}
}
impl Future for Image {
type Item = HtmlImageElement;
type Error = JsValue;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
self.receiver.poll().map(|x| {
if x.is_ready() {
// Prevents the image from being cancelled
self.img = None;
}
x
}).map_err(|_| unreachable!())
}
}
However, in your case you need to send from two different callbacks, so that doesn't work. So instead, like @derekdreery mentioned, you need to manually use the Task stuff.
But rather than mucking around directly with Task, let's instead define some helper types:
use std::sync::{Arc, Mutex};
use futures::task::{Task, current};
use futures::{Async, Poll};
// TODO should use oneshot::Inner
#[derive(Debug)]
struct Inner<T, E> {
completed: bool,
value: Option<Result<T, E>>,
task: Option<Task>,
}
impl<T, E> Inner<T, E> {
#[inline]
fn new() -> Self {
Self {
completed: false,
value: None,
task: None,
}
}
}
pub fn result_channel<T, E>() -> (ResultSender<T, E>, ResultReceiver<T, E>) {
let inner = Arc::new(Mutex::new(Inner::new()));
(
ResultSender {
inner: inner.clone(),
},
ResultReceiver {
inner: inner,
},
)
}
#[derive(Debug, Clone)]
pub struct ResultSender<T, E> {
inner: Arc<Mutex<Inner<T, E>>>,
}
impl<T, E> ResultSender<T, E> {
fn send(&self, value: Result<T, E>) {
let mut lock = self.inner.lock().unwrap();
if !lock.completed {
lock.completed = true;
lock.value = Some(value);
if let Some(task) = lock.task.take() {
drop(lock);
task.notify();
}
}
}
#[inline]
pub fn ok(&self, value: T) {
self.send(Ok(value));
}
#[inline]
pub fn err(&self, value: E) {
self.send(Err(value));
}
}
#[derive(Debug)]
pub struct ResultReceiver<T, E> {
inner: Arc<Mutex<Inner<T, E>>>,
}
impl<T, E> Future for ResultReceiver<T, E> {
type Item = T;
type Error = E;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
let mut lock = self.inner.lock().unwrap();
if lock.completed {
lock.value.take().unwrap().map(Async::Ready)
} else {
lock.task = Some(current());
Ok(Async::NotReady)
}
}
}
Now finally we can define the Image struct:
use web_sys::{HtmlImageElement, UiEvent};
use js_sys::Error;
enum ImageState<'a> {
Initial {
width: u32,
height: u32,
url: &'a str,
},
Pending {
img: HtmlImageElement,
receiver: ResultReceiver<HtmlImageElement, JsValue>,
_on_load: EventListener<'static, UiEvent>,
_on_error: EventListener<'static, UiEvent>,
},
Complete,
}
pub struct Image<'a> {
state: ImageState<'a>,
}
impl<'a> Image<'a> {
#[inline]
pub fn new(width: u32, height: u32, url: &'a str) -> Self {
Self { state: ImageState::Initial { width, height, url } }
}
}
impl<'a> Drop for Image<'a> {
#[inline]
fn drop(&mut self) {
if let ImageState::Pending { ref img, .. } = self.state {
// Cancels the image download
img.set_src("");
}
}
}
impl<'a> Future for Image<'a> {
type Item = HtmlImageElement;
type Error = JsValue;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
match self.state {
ImageState::Initial { width, height, url } => {
let (sender, receiver) = result_channel();
let img = HtmlImageElement::new_with_width_and_height(width, height).unwrap();
img.set_src(url);
let _on_load = EventListener::new(&img, "load", {
let sender = sender.clone();
let img = img.clone();
move |_| {
sender.ok(img.clone());
}
});
let _on_error = EventListener::new(&img, "error", move |_| {
sender.err(Error::new("Failed to load image").into());
});
self.state = ImageState::Pending { img, receiver, _on_load, _on_error };
Ok(Async::NotReady)
},
ImageState::Pending { ref mut receiver, .. } => {
let output = receiver.poll();
match output {
Ok(Async::Ready(_)) | Err(_) => {
self.state = ImageState::Complete;
},
_ => {},
}
output
},
ImageState::Complete => {
panic!("Image polled after completion");
},
}
}
}
imho a runtime exception should be the equivilent of a panic() but Promises swallow exceptions (sortof)... do Futures that come from Promises thereby also swallow panics?
I'm not sure. The interactions between Rust panics and JS exceptions are currently pretty weird, so it may just always panic.
I suggest not relying upon the panic handling, since it will almost certainly change in the future.
In other words when I get a Future - via the wasm_bindgen_futures helpers, if I decide to do nothing with "expected errors" in the course of development, can I be sure that I'm not also accidentally silencing unexpected exceptions?
There are no silent errors. When you use JsFuture::from, the error type is JsValue.
In order to run the Future using spawn_local, it must have an error type of (). So that means the static type system forces you to use map_err to handle the error in some way.
If you instead use future_to_promise, the errors still won't be silent: JS has a mechanism where unhandled Promise errors are automatically reported in the console.
Unlike some other languages, Rust and JS don't have silent errors (which is a wonderful thing).
Whoa, this is all super helpful! Can't wait till the next opportunity where I can sit with it and properly digest what's going on :)
In the meantime, just a small tangential point:
JS has a mechanism where unhandled Promise errors are automatically reported in the console.
Unfortunately "handled" !== "handled" :)
In other words, this is silent:
new Promise(() => { throw new Error("ruhroh") })
.then(
data => {
// happy path
console.log(data);
},
err => {
//Todo - deal with errors later...
});
Sure, _ideally_ we'd always have some proper error handling - even just console.error, but realistically in the course of development sometimes you just don't deal with it while working stuff out.
When errors are _only_ the expected kind - like "wrong password" for a login, or whatever, it's not a big deal (after all - you're just getting started with a hardcoded login/password - it simply won't occur). When errors are also runtime exceptions - not only do they have to be handled, they have to be differentiated by inspecting the type. This suuuuucccckkkkksss.
Would be so nice if wasm_bindgen futures did not swallow exceptions - even in the case of there being an error handler. As a developer I want to know if I caused a panic, even if I am not handling my Promise rejections nicely.
No doubt the futures that are converted from Promises don't have that opportunity (or maybe they do because something is different about the wasm environment) - but I'd suggest this as a design consideration and something to point out in the docs when implementing spawn_local
but realistically in the course of development sometimes you just don't deal with it while working stuff out.
In stdweb there is an unwrap_future function which handles errors by printing them to the console (and then panicking).
You use it like this:
spawn_local(unwrap_future(some_future));
In other words, all you need to do is slap unwrap_future into your main and you're good to go.
So it's not hard at all to handle errors correctly, we just need a helper function like that in wasm-bindgen.
Would be so nice if wasm_bindgen futures did not swallow exceptions - even in the case of there being an error handler. As a developer I want to know if I caused a panic, even if I am not handling my Promise rejections nicely.
My understanding is that there is one of two possibilities:
It panics, in which case there should be an error message in the console (if you're using console_error_panic_hook)
The Promise swallows it, in which case it will then be logged to the console by unwrap_future (or similar).
So the only way it can be silently ignored is if you intentionally ignore it by using .map_err or similar. We can't force developers to handle errors correctly, if they choose to ignore it with .map_err then that's their choice.
Hmm, but I want a rejection to _only_ reject (no panic, i.e. just the Future equivilent of Result's Err) and if there is any thrown exception in the pipeline - to _only_ panic and not get converted into an Err.
If I misunderstood and that is what unwrap_future does, my mistake!
What I'm suggesting is different than the Promise spec so I get that it can't be dealt with via anything that is Future<>Promise - but a pure Future implementation has a chance to do it differently (and imho better).
Also, I understand that sometimes, especially when working with third-party libraries, the above is just not possible, and recovering from rejections/panics should be the default. In that case I think wrapping in a Promise is fine (rather than relying on, I think catch_unwind ?) and it's something that can be decided by the developer.
Fwiw a lot of my take on this is influenced by the JS Fluture library: https://github.com/fluture-js/Fluture
There's a nice summary of the Fluture approach in "Error Handling" here: https://github.com/fluture-js/Fluture/wiki/Comparison-to-Promises#error-handling
Could a native wasm Future take the same philosophy?
Perhaps there's some confusion about how Futures work in Rust.
Unfortunately, Futures 0.1 conflates error handling (similar to Promises in JS). Futures 0.3 is much better, since it makes the Result explicit.
Let me start by explaining the Futures 0.3 model, and then we'll go back to the Futures 0.1 model.
With Futures 0.3, a Future behaves like an ordinary Rust function: it just returns a value.
So a Rust function that returns impl Future<Output = u32> is the same as a Rust function which returns u32 (except the Future is asynchronous, obviously).
Similarly, a Rust function that returns impl Future<Output = Result<u32, SomeError>> is the same as a Rust function which returns Result<u32, SomeError>. So everything is nice and clear.
And the parallels are even more obvious with async/await!:
fn foo() -> Result<u32, SomeError> {
let x = bar()?;
Ok(x + 5)
}
async fn foo() -> Result<u32, SomeError> {
let x = await!(bar())?;
Ok(x + 5)
}
In this case foo() returns impl Future<Output = Result<u32, SomeError>>. Other than the return type and the usage of await! both functions are the same. We get to use Result, just like usual!
With Futures 0.1, it's like as if every Future implicitly returns Result. So foo() would return impl Future<Item = u32, Error = SomeError>.
When a panic happens, that usually causes the Task (or even the entire Executor) to fail. Panics are not swallowed up and converted into Err (unless you use something like catch_unwind).
So generally speaking, everything follows Rust's normal idioms: use panicking for unrecoverable errors (and this will kill the current thread/Task/whatever, as usual). And return Err for recoverable errors.
This mirrors the Fluture link you posted, so I guess the answer is: Rust Futures already follow that error handling philosophy.
In the case of Futures 0.1, you can retrieve the Err of a Future by using the then or or_else methods.
The then method behaves like then with JS Promises, and or_else behaves like catch with JS Promises.
Unlike Promises, Futures are inert: they don't do anything until they are spawned. This spawning usually happens once, in main:
fn main() {
spawn_local(some_future);
}
If you want parallelism, you can use the various Future combinators like join:
fn main() {
spawn_local(some_future.join(some_other_future));
}
The spawn_local function requires the Future to have an error type of (). In other words, it requires you to handle the errors in some way before calling spawn_local:
fn main() {
spawn_local(some_future.join(some_other_future).map_err(|err| {
println!("Something went wrong! {:?}", err);
}));
}
This simulates the way that non-asynchronous code works: when you panic (or return a Result from main), the panic will bubble up to main, and then main will print the error to the console.
So your Future spawning + Future error handling code is all in one place, in main.
If you don't want an error to bubble up to main, you can use then or or_else to handle it:
some_future.or_else(|error| {
// Handle it somehow
Ok(())
})
This is in addition to the error handling in main: the purpose of the error handling in main is to notify you about any uncaught errors.
Unlike JS Promises, because you are required to handle errors before you can call spawn_local, and because Futures do not do anything until they are spawned, this guarantees that errors are handled in some way.
As for the question of mixing JS Promises and Futures, that's really tricky. As you know, Promises swallow all exceptions.
After running some quick tests, it appears that panicking causes it to display a console message (thanks to console_error_panic_hook), then the entire wasm module aborts, and then an unreachable exception is swallowed by JS.
You shouldn't rely upon that, though, since the panic/exception interactions between wasm/JS will likely change in the future.
Could you explain more about what you're trying to accomplish? If you want to avoid ignoring errors, that's what spawn_local + unwrap_future do. If you want to handle errors (similar to how Result is handled in Rust), then you can use then or or_else.
So the question is whether spawning Futures directly uses the JS microtask queue, or whether it uses a Rust dequeue.
there is already (90% of) an executor (FuturesUnordered), could it be used to impl a wasm executor? I thought about impling a custom wasm-waker, which would poll from the FuturesUnordered in a loop in the wake method, and break out of the loop on Poll::Pending returning the control to the js runtime environment?
@chpio From what I can tell, FuturesUnordered isn't an Executor, it's a Stream.
Existing Executors don't work with wasm, because they assume blocking threads, which wasm doesn't have. That's why new Executors have to be written, which use JavaScript's microtask event loop.
Though I suppose it would be possible for an Executor to internally use FuturesUnordered (in addition to other things).
Existing Executors don't work with wasm, because they assume blocking threads, which wasm doesn't have. That's why new Executors have to be written, which use JavaScript's microtask event loop.
yeap, that's why im referring to impling a custom waker/executor, which would run the executor loop until all scheduled tasks would return Poll::Pending, then return the control to the runtime environment.
yeap, that's why im referring to impling a custom waker/executor, which would run the executor loop until all scheduled tasks would return Poll::Pending, then return the control to the runtime environment.
That would probably work, but in that case you can just use a Rust VecDeque, which is probably faster than FuturesUnordered.
Could you explain more about what you're trying to accomplish? If you want to avoid ignoring errors, that's what spawn_local + unwrap_future do. If you want to handle errors (similar to how Result is handled in Rust), then you can use then or or_else.
Primarily trying to learn enough Rust so I can be productive with nighttime hacking ;)
More specifically - in this case, I want to have a library load things and return a Future - and I don't want it to turn panics/exceptions into Err's :)
Looking forward to digging into all the above when I can properly digest it. Thank you so much for taking the time to write up all the examples and explanations!
Btw I had problems last night when I tried using futures w/ 0.3... does that work properly for you in wasm? The error I got was about #![feature(pin)]
Btw I had problems last night when I tried using futures w/ 0.3... does that work properly for you in wasm? The error I got was about #![feature(pin)]
Are u using the nightly compiler? if yes { remove the feature gate, it's stable now (in the nightly compiler)} else {you need the nightly version}
Yep - nightly :)
I'll try again, thanks!
I believe this shoudl be addressed by https://github.com/rustwasm/wasm-bindgen/pull/1148, so closing in favor of that!
I have a bit of time today to pick this back up... @Pauan your examples are _super_ helpful... thanks again!
Just a small followup (so far) - do you mind explaining why Arc/Mutex? At a glance I'm thinking since JS is single-threaded there shouldn't be a need for those...?
@dakom
At a glance I'm thinking since JS is single-threaded there shouldn't be a need for those...?
No, there is a future feature for threading in wasm.
Btw I tried creating a futures example locally in a fork, and I still get the error with Pin:
Cargo.toml:
[package]
name = "futures"
version = "0.1.0"
authors = ["The wasm-bindgen Developers"]
edition = "2018"
[lib]
crate-type = ["cdylib"]
[dependencies]
futures-core-preview = "0.3.0-alpha.11"
futures-channel-preview = "0.3.0-alpha.11"
futures-util-preview = "0.3.0-alpha.11"
wasm-bindgen = "0.2.31"
js-sys = "0.3.8"
[dependencies.web-sys]
version = "0.3.4"
features = [
'console'
]
lib.rs
(doesn't do anything yet)
#![feature(arbitrary_self_types)]
use futures_core::Poll;
use futures_core::future::Future;
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsCast;
use web_sys::{console};
#[wasm_bindgen]
pub fn run() -> () {
console::log_1(&JsValue::from_str("hello world"));
}
Error
error[E0658]: arbitrary `self` types are unstable (see issue #44874)
--> /home/dakom/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-core-preview-0.3.0-alpha.11/src/stream/mod.rs:179:23
|
179 | mut self: Pin<&mut Self>,
| ^^^^^^^^^^^^^^
|
= help: add #![feature(arbitrary_self_types)] to the crate attributes to enable
= help: consider changing to `self`, `&self`, `&mut self`, or `self: Box<Self>`
Oh I see wasm_bindgen_futures is on futures = "0.1.20" ...yet @Pauan your examples like rust-signals and dominator are on futures v. 0.3
o_O
More specifically - in this case, I want to have a library load things and return a Future - and I don't want it to turn panics/exceptions into Err's :)
I don't think you need to worry too much about that. Right now panics are turned into aborts, so there's not really much possibility for a panic to get swallowed into an Err.
I think we can revisit this later, after proper panicking is supported on wasm32-unknown-unknown.
There are ways to guarantee that panics won't be swallowed by Promises, but they have a large latency penalty (it can be as high as 4ms in some cases).
I don't recommend it, at least not until we have more experience with proper panicking support.
Btw I had problems last night when I tried using futures w/ 0.3... does that work properly for you in wasm? The error I got was about #![feature(pin)]
wasm-bindgen only supports Futures 0.1 right now (which doesn't support async/await). I'm hoping it'll upgrade to Futures 0.3 soon.
All of my projects are using stdweb (which does fully support Futures 0.3, and async/await).
Just a small followup (so far) - do you mind explaining why Arc/Mutex? At a glance I'm thinking since JS is single-threaded there shouldn't be a need for those...?
Mostly habit. When working on futures-signals, I specifically designed everything to work with multi-threading (since it can work in any environment, not just wasm).
It's nice creating general-purpose tools which work in all situations, rather than wasm-specific tools.
And as @chpio mentioned, wasm will soon have support for multi-threading.
At the moment, with wasm, Arc and Mutex compile down to basically the same code (or faster) as Rc / RefCell, so there's currently no performance cost to using them.
Oh I see wasm_bindgen_futures is on futures = "0.1.20" ...yet @Pauan your examples like rust-signals and dominator are on futures v. 0.3
Yeah, sorry, most of my work is in stdweb (dominator doesn't support wasm-bindgen at the moment, but I'm porting it over).
@dakom When you return Ok(Async::NotReady) you are saying "hold on, please wait, I don't have a value ready yet, but I'll notify you later when I do have a value".
However, you never actually notify, so it just sits there waiting forever for a value which never arrives.
In order to notify, you first call current() to get the current Task, and then you later call Task::notify, like this:
let task = current();
task.notify();
The above code immediately notifies, which is rarely what you want. Instead you should store the task somewhere else, and then call Task::notify later. Here is an example:
impl Future for AsyncCounter {
type Item = JsValue;
type Error = JsValue;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
if self.value == self.target {
Ok(Async::Ready(JsValue::from_f64(self.value as f64)))
} else {
self.value += 1;
let task = current();
let closure = Closure::wrap(Box::new(move || {
task.notify();
}) as Box<FnMut()>);
// TODO: handle cancellation with clearTimeout
window().unwrap().set_timeout_with_callback_and_timeout_and_arguments_0(closure.as_ref().unchecked_ref(), 1000).unwrap();
closure.forget();
Ok(Async::NotReady)
}
}
}
This will use setTimeout to wait for 1000 milliseconds, and then it notifies the Task (which then causes poll to get called again).
Basically, calling Task::notify is saying "hey, please call poll again".
Also, I noticed you are using JsValue for the Item type, but that is not necessary. You can use u32 for the Item (and then get rid of the JsValue::from_f64).
In general only the Error type needs to match, the Item type can be different for each Future.
I also noticed you are using future_to_promise and then immediately afterwards using JsFuture::from. That is a very inefficient noop (since it converts from Future -> Promise -> Future).
Instead you can just return the Future directly:
#[wasm_bindgen_test(async)]
fn custom_future_steps() -> impl Future<Item = (), Error = JsValue> {
AsyncCounter::new(2)
.map(|x| {
assert_eq!(x, 2);
})
}
I see you are copying what the docs say, but the docs are written that way since it is generating a Promise out of thin air and then converting it into a Future.
In your case you already have a Future, so you don't need to convert into a Promise (or back again).
As for the question about tasks, notification, etc. that's a bit tricky to explain, so I'll just point to some other resources:
http://aturon.github.io/blog/2016/08/11/futures/
http://aturon.github.io/blog/2016/09/07/futures-design/
https://tokio.rs/docs/getting-started/futures/
https://tokio.rs/docs/going-deeper/futures/
They're a bit outdated, but the general concepts of Task and notification is the same.
If you want to see a fast and robust wasm Executor/Spawn/Task implementation, here is the one that stdweb uses. I can go into more detail about it, since I wrote it.
Or if you want to see a simplified Executor/Task implementation, which might be easier to understand, here is the old stdweb implementation. It's severely outdated, and won't work with Futures 0.3
There is also the implementation used by wasm-bindgen. The Package type is the Task. It doesn't have an Executor, instead it just spawns the Tasks directly (which is unusual).
Note: you don't need to understand Executors or Tasks in order to implement or use Futures. They are purely an internal implementation detail.
You only need to understand the Future trait, and then use spawn_local or similar to actually run the Future.
I guess you saw my cross-post here: https://users.rust-lang.org/t/how-to-drive-a-future/24169 :)
Thanks again!!!
I guess you saw my cross-post here: https://users.rust-lang.org/t/how-to-drive-a-future/24169 :)
Actually, I receive e-mail notifications from GitHub, so even if a message is edited or deleted, I still see the message in my e-mail.
If I had seen that thread, I would have posted in there (so that way other people can see it).
Ahh... yeah I had a bug in the code I had posted which was just a silly mistake I fixed... can't remember what it was now ;)
Anyway - Isn't future_to_promise the way to _execute_ a Future?
Linked the post to here just now in case anyone wants to follow along
Anyway - Isn't future_to_promise the way to execute a Future?
It is one way to execute a Future. The other way is to call spawn_local.
You should use future_to_promise if you intend to pass the Future into JavaScript (since that way JavaScript receives a Promise).
But if you just want to execute the Future in Rust (without passing to JS), then spawn_local is better (it is harder to make mistakes, and it can be faster).
In the case of #[wasm_bindgen_test(async)], it handles the spawning for you (internally it uses future_to_promise)
spawn_local for wasm_bindgen (not stdweb, hehe) seems to just call future_to_promise
So is there another way to execute that doesn't involve future_to_promise?
Good to know about the test framework!
In general only the Error type needs to match, the Item type can be different for each Future.
i just noticed that there are no result adopters (and_then) in [email protected] anymore.
Do you know if there are any plans to bring them back, maybe by using the Try trait?
spawn_local for wasm_bindgen (not stdweb, hehe) seems to just call future_to_promise
That will change. That was just an initial quick implementation.
However, it will always be using the same underlying machinery as future_to_promise (since they both execute Futures).
So is there another way to execute that doesn't involve future_to_promise?
Not right now. And even if there was another way, it would still involve the JavaScript microtask queue in some way.
Promises are the only standard cross-platform way to access the microtask queue.
stdweb uses MutationObserver to access the microtask queue (this does not swallow panics). However that only works in browsers, so it falls back to Promises in other environments.
You can use setTimeout (or any other macrotask queue function) to avoid panics being swallowed by Promises. But as I said earlier, that's not necessary right now (since panics are compiled into aborts).
@chpio They were moved into TryFutureExt. All Futures which return Result automatically implement TryFuture.
Might be worthwhile renaming spawn_local to spawn_promise or even having something like spawn_timeout and spawn_request_animation_frame to allow different strategies?
Might be worthwhile renaming spawn_local to spawn_promise or even having something like spawn_timeout and spawn_request_animation_frame to allow different strategies?
spawn_timeout and spawn_request_animation_frame don't work very well, since they are macrotasks, not microtasks (and they have high latency).
spawn_promise makes it sound like it's spawning a Promise (which it isn't).
And as I said, Promises are not the only way to access the microtask queue (MutationObserver is another way, and there are some others too), so the exact way that it spawns the Future is an implementation detail, so it shouldn't be reflected in the name.
In addition, calling it spawn_promise ties it to Promise semantics (e.g. exceptions being swallowed), but we probably don't want to codify that (we might want to switch later to a strategy which doesn't swallow exceptions).
Also, to be clear, regardless of what strategy is used for spawn_local, you can always use things like setTimeout and requestAnimationFrame: you just have to create a Future which wraps them.
So it's always possible to avoid exceptions being swallowed, even if spawn_local uses Promises.
Yes but since wasm/rust gets to decide how to schedule things, I don't see why spawn necessarily equates to microtask queue. By default and recommendation, yes, but being able to choose how to schedule Futures seems like a nice feature.
I don't feel strongly about it, just curious
Also, to be clear, regardless of what strategy is used for spawn_local, you can always use things like setTimeout and requestAnimationFrame: you just have to create a Future which wraps them.
i don't know the difference between a micro- and macrotask, so maybe that's just part of my misunderstanding. isn't the current impl creating a new Promisse in spawn_local/future_to_promise and thus executing the task on the next tick/after RAF?
edit: a microtask seems to be a task, which is added to the running/current task/executed directly after the current task finishes(?)
I don't see why spawn necessarily equates to microtask queue.
There's some rather deep and subtle reasons for this.
The Futures system is designed to be asynchronous, so when you call task.notify() it must not immediately wake up the Task, instead it must be delayed by a small amount.
This isn't specific to wasm, all Future Executors on all platforms must be that way (including tokio). It's a part of the Executor contract, and various Futures rely upon that contract (so they would break on non-compliant Executors).
So since we are in wasm, we need some way to schedule a wakeup in the future. There are only two ways to do that: microtask and macrotask.
We could use macrotasks (e.g. setTimeout), but that introduces an unnecessarily large delay (it can be up to 4ms in some cases).
In addition, the browser rendering is based on macrotasks, so we might end up racing with the renderer! In other words, the browser might render the page before the setTimeout or after the setTimeout.
But we don't want that: we want to guarantee that the Future runs before rendering, so we can avoid the dreaded "flash of unstyled content" (and similar issues).
If we use microtasks (e.g. Promises), then all those problems go away: microtasks have zero latency (they are delayed, but run immediately after the JS code), and they are guaranteed to run before rendering.
This ensures the maximum performance and the minimum issues.
It isn't very surprising that a mechanism specifically designed for asynchronous values (Promises) would also be a good match for Futures.
There's another reason why we need spawn_local (and future_to_promise) to be asynchronous: consistency. Consider this program:
let x = Arc::new(Mutex::new(0));
let y = x.clone();
spawn_local(some_future.map(move |_| {
*y.lock().unwrap() = 5;
}));
println!("{}", *x.lock().unwrap());
The question is: does it print 0 or 5? If we didn't guarantee asynchronousness, then it would depend on the behavior of some_future! But by guaranteeing asynchronousness, it will always be 0.
i don't know the difference between a micro- and macrotask, so maybe that's just part of my misunderstanding. isn't the current impl creating a new Promisse in spawn_local/future_to_promise and thus executing the task on the next tick/after RAF?
Promises execute long before RAF (RAF has very high latency, Promises have zero latency).
The difference between microtasks and macrotasks is rather subtle, this page probably explains it best.
a microtask seems to be a task, which is added to the running/current task/executed directly after the current task finishes(?)
Yes, basically. Microtasks have priority over macrotasks, so they run first.
so when you call task.notify() it must not immediately wake up the Task
Async is sometimes called cooperative multitasking. It's called this because it requires the futures to play by the rules. With threads, a bad thread will still only get their share of the cpu. With futures, we can block everything up if we want.
This thread should probably be turned into a blogpost. There is loads of great information here!
Async is sometimes called cooperative multitasking. It's called this because it requires the futures to play by the rules. With threads, a bad thread will still only get their share of the cpu. With futures, we can block everything up if we want.
According to the Future contract, it is valid to immediately call task.notify() inside of poll, and this is guaranteed to not immediately call poll again.
It might still synchronously call poll, but it must allow other pending Tasks to go first.
You can read more here:
https://github.com/rust-lang-nursery/futures-rs/issues/738
https://github.com/rust-lang-nursery/futures-rs/issues/754
If the scheduling is always driven by the microtask queue, then won't calling task.notify() immediately create a deadlock since it re-schedules it in the _same_ queue?
I put together a little JS demo to show what I mean - using the macrotask queue isn't a problem, but the microtask freezes the window: https://codesandbox.io/s/13n8pxyxz3
I'd imagine that example is synonymous with calling task.notify() immediately - but haven't tested that out yet.
@dakom yeah, but that's a broken future impl, it would break pretty much every executor, i guess.
Oh wow - that issue is exactly what I should have read before ;)
Thanks!
If the scheduling is always driven by the microtask queue, then won't calling task.notify() immediately create a deadlock since it re-schedules it in the same queue?
Yes, however, it won't stop other Futures from running. That is the key difference between an event loop and running task.notify() immediately. So it's a soft deadlock, not a hard deadlock.
Even if we used the macrotask queue, it would still livelock, which isn't much better than deadlock.
Interestingly, it's possible to create a combinator which forces other Futures to run on the macrotask queue, thus converting deadlock into livelock.
I'd imagine that example is synonymous with calling task.notify() immediately - but haven't tested that out yet.
Yeah, it's the same.
The microtask queue behaves the same as most event loop implementations (including Rust event loops).
The macrotask queue is... different.
Just came back to this after a bit of time away...
Still haven't taken on the EventListener thing but I ended up just storing the closure in the Image struct and it seems to work fine so far
e.g. (partial code)
img.set_src(url);
let task = current();
let closure = Closure::wrap(Box::new(move || {
console::log_1(&JsValue::from_str("loaded"));
task.notify();
}) as Box<FnMut()>);
img.set_onload(Some(closure.as_ref().unchecked_ref()));
//closure.forget();
self.img = Some(img);
self.state = ImageState::Loading;
self.closure = Some(closure);
The Image container thing gets dropped somehow once the future resolves, so I think it's all good... not 100% sure though and I don't get how that's _really_ possible since the Future resolves an Image which still has the callback attached to it somehow?
this could throw an exception on the js side if the rust struct is dropped?
also i think, that it's not guaranteed that a task.notify will result in the future being polled (especially if your future is wrapped by adapters).
edit: yeah, i deleted that comment, i thought you're storing the closure inside of the <img> dom element.
I'm not really sure about much of anything here, hehe
The one gotcha I'm having is that the closure can't access the containing struct since the closure outlives it... so not sure how to signal errors yet :\
Tried to workaround by using a channel... but seems I can't store that in a closure?
error[E0277]: expected a `std::ops::FnMut<()>` closure, found `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
--> crates\web_sys_loaders\src\image.rs:72:49
|
72 | let closure_err = Closure::wrap(Box::new(cb) as Box<FnMut()>);
| ^^^^^^^^^^^^ expected an `FnMut<()>` closure, found `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
|
= help: the trait `std::ops::FnMut<()>` is not implemented for `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]`
= note: wrap the `[closure@crates\web_sys_loaders\src\image.rs:64:26: 70:18 sender_err:futures::sync::oneshot::Sender<xhr::wasm_bindgen::JsValue>, task:futures::task_impl::Task]` in a closure with no arguments: `|| { /* code */ }
= note: required for the cast to the object type `dyn std::ops::FnMut()`
code is basically:
let (sender_err, receiver_err):(Sender<JsValue>, Receiver<JsValue>) = channel();
//error callback
let task = current();
let cb = move || {
sender_err.send(JsValue::from_str("unknown error!"));
task.notify();
};
let closure_err = Closure::wrap(Box::new(cb) as Box<FnMut()>);
img.set_onerror(Some(closure_err.as_ref().unchecked_ref()));
Just came across this curious page on MDN: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/queueMicrotask
o_O
@dakom That's very cool, and exactly what we need to remove the Promise overhead. But it's not standardized, so we can't use it right now.
Most helpful comment
This thread should probably be turned into a blogpost. There is loads of great information here!