Rust: Async fn does not compile if lifetime does not appear in bounds (sometimes)

[dtolnay]

#![feature(async_await)]

use std::marker::PhantomData;

trait Trait {}

async fn f<'a>(reference: &(), marker: PhantomData<dyn Trait>) {}

This fails with:

error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
 --> src/main.rs:7:64
  |
7 | async fn f<'a>(reference: &(), marker: PhantomData<dyn Trait>) {}
  |                                                                ^
  |
note: hidden type `impl std::future::Future` captures the scope of call-site for function at 7:64
 --> src/main.rs:7:64
  |
7 | async fn f<'a>(reference: &(), marker: PhantomData<dyn Trait>) {}
  |                                                                ^^

I believe this should compile because:

1. The non-async function with the same signature does compile;

   fn f<'a>(reference: &(), marker: PhantomData<dyn Trait>) {}
   

2. Seemingly insignificant tweaks, like adding a wrapper struct, make it compile.

   struct Wrapper(PhantomData<dyn Trait>);
   async fn f<'a>(reference: &(), marker: Wrapper) {}
   

Mentioning @cramertj who fixed a similar error message in #59001.
Mentioning @nikomatsakis who touched this error message in #49041 and #61775.

rustc 1.38.0-nightly (c43753f91 2019-07-26)

[nikomatsakis]

Check-in from WG-async-await meeting: I agree this is a bug -- it doesn't necessarily block stabilization (no fwd compat risk) but would definitely be a good one to fix. Marking as blocking + unclear for the moment and will return to it.

[nikomatsakis]

I did a bit of investigation. The error results because we have an inference variable ('_#3r) that seems to have no constraints, but must be equal to one of the captured regions (e.g., 'a). I haven't 100% figured out where this variable comes from, I imagine it's related to 'a though? In any case, the resolver basically winds up in a situation where it could pick any of the lifetimes but has no reason to prefer one over the other, and hence it reports an error.

[nikomatsakis]

OK, I think I see what's going on. It doesn't actually have much to do with the unused lifetime per se. What happens is this:

• The type of the generated result is something like Foo<&'2 (), PhantomData<dyn Trait + '3>> -- i.e., it includes the types of the parameters, though with region variables in place of the known regions. Those types must be supertypes of the actual parameter types.
• We capture the argument marker of type dyn Trait + 'static -- per the subtyping rules, we create the type dyn Trait + '3 for the type of this captured value, with the requirement that 'static: '3 -- as it happens, this is always true, so there is effectively no constraint on '3.
• We do impose the constraint that '2 and '3 must both be equal to one of the free regions -- in this case, 'a or the anonymous lifetime of reference (which I'll call 'b).

So thus the region solvers get in a state where '3 must be equal to 'a, 'b, or 'static -- but any one of them would be a reasonable choice.

Removing the unused lifetime ('a) makes this an easy choice -- it can pick 'b, which is smaller than 'static. (Per the "least choice" rules we usually use.)

Removing the type and putting it into a struct (Wrapper) removes the lifetime variable altogether, as then we have Foo<&'2 (), Wrapper>, and there are no lifetime parameters involved.

I'm not sure what I think is the best fix. It would be nice for the solver to pick 'static, though it complicates the algorithm. You could certainly imagine a tweak that picks 'static if the value is legal and otherwise looks for a least choice. That might indeed be superior to the current approach, though I'd want to think about the full implications -- in particular, when impl Trait is used in places outside of the fn return type (e.g., in a let statement), picking 'static might lead to unnecessary borrow check errors. (The alternative would be to only look for 'static when there is no least choice; that seems safer.)

(Interestingly, at some point at least I thought that a polonius-like approach sidesteps a lot of the challenges here-- if we had confidence that we would transition that might ease some of my concerns.)

[nikomatsakis]

Now that I understand better what is happening, I don't feel this has to block stabilization -- in particular, I don't think that there is a forward compatibility risk from us improving the region inference algorithm. I don't think we would choose an algorithm that makes existing code illegal. =)

As such, I'm going to mark this as Deferred.

[sfackler]

Here's another example of the same problem that doesn't have an unused lifetime (AFAIKT):

struct Foo;

impl Foo {
    async fn wat(&self, _: &'static str, _: Bar<'_>) {}
}

struct Bar<'a>(Box<dyn std::fmt::Debug + 'a>);

error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
 --> src/lib.rs:4:54
  |
4 |     async fn wat(&self, _: &'static str, _: Bar<'_>) {}
  |                                                      ^
  |
note: hidden type `impl std::future::Future` captures the scope of call-site for function at 4:54
 --> src/lib.rs:4:54
  |
4 |     async fn wat(&self, _: &'static str, _: Bar<'_>) {}
  |                                                      ^^

error: aborting due to previous error

Removing any of the three arguments to the function allows it to compile.

[lovasoa]

I am also affected by this bug, where having both a reference and a box as parameters to a function causes the cryptic "hidden type for impl Trait captures lifetime that does not appear in bounds" :

trait Trait {}
async fn f<'a>(_a: &(), _b: Box<dyn Trait>) {}

(playground)

Is there a resolution in sight ?

[sfackler]

'a is unused in that function - if you remove the <'a> it compiles.

[lovasoa]

Ok, let me change it to something closer to what I had in my original code:

trait T {}
struct S;
impl S {
    async fn f(_a: &S, _b:&S, _c: Box<dyn T>) {}
}

(playground)

[netvl]

I think I found another example of this:

trait T {}

struct L<A = Box<dyn T>> {
    value: A,
}

async fn test1(l: L, s1: &str, s2: &str) {
}

(playground)

This example more or less represents what I met in my code (when I tried to pass a structure with a type parameter to an async function with the default value for this type parameter being a trait object), but it can be simplified by removing the default parameter assignment and passing the Box<dyn T> argument to L explicitly:

trait T {}

struct L<A> {
    value: A,
}

async fn test1(l: L<Box<dyn T>>, s1: &str, s2: &str) {
}

(playground)

Curiously, if you replace async fn ... { ... } with fn ... -> impl Future { async { ... } }, the error goes away:

use std::future::Future;

trait T {}

struct L<A> {
    value: A,
}

fn test1(l: L<Box<dyn T>>, s1: &str, s2: &str) -> impl Future<Output=()> {
    async {
    }
}

(playground)

It is important to note that the test1 function has two additional arguments. If you remove one or both of them, it starts compiling again:

trait T {}

struct L<A> {
    value: A,
}

async fn test2(l: L<Box<dyn T>>, s1: &str) {
}

async fn test3(l: L<Box<dyn T>>) {
}

(playground)

[skeet70]

@netvl 's comment is a minimal reproduction of what I ran into. I had a function async f(a: &CacheA, b: &CacheB, c: &CacheC, d: Arc<dyn Client>). If I remove two of the cache arguments it compiles. As soon as I add one back in it gives me the title error.

[nikomatsakis]

Hmm, it is probably worth revisiting this problem now that stabilization has happened.

[swilcox3]

Any word on this? I'm getting this error as well.

pub async fn publish(
    conn: &mut StateConnection,
    file: &FileID,
    user: &UserID,
    latest: ChangeID,
    mut subscribers: Vec<Box<dyn ChangeSubscriber>>,
) -> Result<(), ObjError> {
    let to_publish = gather_changes(conn, file, latest).await?;
    for sub in subscribers {
        sub.get_changes(file, user, latest, &to_publish).await?;
    }
    Ok(())
}

Changing mut subscribers: Vec<Box<dyn ChangeSubscriber>> to subscribers: &mut Vec<Box<dyn ChangeSubscriber>> fixes the issue. Only problem is that I actually want to move that vector in because I want to tokio::spawn each call to get_changes.

[nikomatsakis]

No progress or major updates. The description here is still accurate as to the root cause, but I think we don't yet know the best fix.

[danieleades]

this is going to become a problem now that #72459 is merged.

It makes it impossible to implement IntoFuture for types with lifetimes, when the future that's returned is unnameable.

consider

pub struct Request<'a> {
    ...
}

impl<'a> Request<'a> {
    async fn send(self) -> Result<(), Error> {
        ...
    }
}

// does not compile
// "cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements"
impl<'a> IntoFuture for Request<'a> {
    type Output = Result<(), Error>;
    type Future = impl Future<Output = Self::Output>;

    fn into_future(self) -> Self::Future {
        self.send()
    }
}

// does not compile
// "hidden type for `impl Trait` captures lifetime that does not appear in bounds"
impl<'a> IntoFuture for Request<'a> {
    type Output = Result<(), Error>;
    type Future = impl Future<Output = Self::Output> + 'a;

    fn into_future(self) -> Self::Future {
        self.send()
    }
}

[AndiDog]

Here's another very simple example:

#[allow(dead_code)]
struct SomeStruct<'a> {
    s: &'a str,
}

async fn something_asynchronous(_a: &'static str, _b: &str, _c: &SomeStruct<'_>) {}

error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
 --> src/main.rs:6:82
  |
6 | async fn something_asynchronous(_a: &'static str, _b: &str, _c: &SomeStruct<'_>) {}
  |                                                                                  ^
  |
note: hidden type `impl std::future::Future` captures lifetime smaller than the function body
 --> src/main.rs:6:82
  |
6 | async fn something_asynchronous(_a: &'static str, _b: &str, _c: &SomeStruct<'_>) {}
  |                                                                                  ^

Removing any of the 3 parameters makes the code compile. Alternatively, removing the static lifetime of _a (→ _a: &str) makes it compile as well.

[redragonx]

Hi.

I get the following error that matches what everyone is also getting:

error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
  --> src/profile.rs:91:10
   |
91 |     ) -> Result<(), BoxError> {
   |          ^^^^^^^^^^^^^^^^^^^^
   |
note: hidden type `impl futures::Future` captures lifetime smaller than the function body
  --> src/profile.rs:91:10
   |
91 |     ) -> Result<(), BoxError> {
   |       

pub type BoxError = Box<dyn Error + Send + Sync + 'static>;

    pub async fn run_profile(
        &self,
        app_config: &AppiConfig,
        app_args: ArgMatches<'static>,
    ) -> Result<(), BoxError> {

[pthariensflame]

Another case showed up yesterday:

async fn test(test: &String, test2: &String, test3: &'static String) {}