Async-std: Provide test macro like `#[tokio::test]` or `#[runtime::test]`

[95th]

As as the title says, async_std should provide macro like #[async_std::test] similar to #[tokio::test] or #[runtime::test] for writing tests.

[ghost]

While the test macro is some sweet sugar, I find task::block_on() not much worse when it comes to ergonomics. Compare:

#[async_std::test]
async fn foo() {
    // ...
}

#[test]
fn foo() {
    task::block_on(async {
        // ...
    })
}

That said, I wouldn't be against adding a #[async_std::test] attribute similar to #[std::test].

[95th]

Yes, the block_on is close, but macro feel more natural IMO. Also, block_on version has more boilerplate.

[yoshuawuyts]

@95th compile times would likely spike significantly if we included such a macro. Perhaps it would make sense to create as a separate package (perhaps under the org) that's not necessarily part of the same top-level crate.

In my opinion the main benefit of such macros is when abstracting over different systems; in particular between different execution styles. This isn't quite in scope for the async-std crate yet, but it might be in the future, and getting ahead of that isn't necessarily a bad thing imo. As long as we can stick to the driving vision of: "we provide an async version of std".

[skade]

While I do see the boilerplaty-ness of block_on in simple cases, a lot of async-stds model relies on spawning and joining tasks, in which case such patterns become nice:

fn test() {
   // future construction
   let x = task::spawn(fut1);
   let y = task::spawn(fut2);
   task::block_on(async {
       x.await;
       y.await;
   })
}

The amount of boiler becomes less and less and it avoids running into annoying situations like this being a block_on in a block_on if we had such a test annotation. I agree with @yoshuawuyts that removing the line at the cost of compiling a proc macro seems excessive to me.

[95th]

@skade The pattern you described above is still looks better better with macro IMHO:

#[runtime::test]
fn test() {
   // future construction
   let x = task::spawn(fut1);
   let y = task::spawn(fut2);
   x.await;
   y.await;
}

it avoids running into annoying situations like this being a block_on in a block_on if we had such a test annotation

Agreed.

[ghost]

In my opinion the main benefit of such macros is when abstracting over different systems; in particular between different execution styles.

For us, the only benefit of the #[test] macro is sugar.

But I think #[bench] would have another additional benefit. The following is not the best way of benchmarking things:

#[bench]
fn benchmark(b: &mut Bencher) {
    b.iter(|| {
        task::block_on(async {
            // ...
        })
    })
}

The reason is that block_on currently spawns a task and then awaits it (which is costly!), rather than polling the future on the current thread. But I think we'll change the behavior of block_on so that it does poll the future on the current thread without spawning. That would also remove the F: Send bound from block_on.

We can then try:

#[bench]
fn benchmark(b: &mut Bencher) {
    task::block_on(async {
        b.iter(|| {
            // ...
        })
    })
}

Still, that is not yet there because a future polled outside the runtime interacting with the reactor and the executor will be slower than one that is spawned onto the executor.

So then the proper way of benchmarking tasks becomes:

#[bench]
fn benchmark(b: &mut Bencher) {
    task::block_on(task::spawn(async {
        b.iter(|| {
            // ...
        })
    }))
}

Except one has to really think about this whenever benchmarking stuff in order to not get misleading numbers.

So the summary is that an attribute is not only syntax sugar; it also provides the optimal benchmarking setup.

[yoshuawuyts]

async-attributes now exists, which adds support for all the features discussed in this issue (including @stjepang's correct bench code).

You can get it from crates.io. Thanks all!