Rust: impl-trait return type is bounded by all input type parameters, even when unnecessary

[Arnavion]

rustc 1.20.0-nightly (f590a44ce 2017-06-27)
binary: rustc
commit-hash: f590a44ce61888c78b9044817d8b798db5cd2ffd
commit-date: 2017-06-27
host: x86_64-pc-windows-msvc
release: 1.20.0-nightly
LLVM version: 4.0

#![feature(conservative_impl_trait)]

trait Future {
}

impl<F> Future for Box<F> where F: Future + ?Sized {
}

struct SomeFuture<'a>(&'a Client);
impl<'a> Future for SomeFuture<'a> {
}

struct Client;
impl Client {
    fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a /* (1) */ {
        SomeFuture(self)
    }
}

fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
    client.post(&[username])
}

fn main() {
    let client = Client;
    let _f = {
        let username = "foo".to_string();
        login(&client, &username)
    };
}

Since SomeFuture borrows 'a Client, I'd expect impl Future + 'a to be the correct return type, but it gives this error:

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
  --> src\main.rs:21:16
   |
21 |    client.post(&[username])
   |                  ^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the function body at 20:1...
  --> src\main.rs:20:1
   |
20 | / fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
21 | |  client.post(&[username])
22 | | }
   | |_^
note: ...so that expression is assignable (expected &str, found &str)
  --> src\main.rs:21:16
   |
21 |    client.post(&[username])
   |                  ^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 20:1...
  --> src\main.rs:20:1
   |
20 | / fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
21 | |  client.post(&[username])
22 | | }
   | |_^
note: ...so that the type `impl Future` will meet its required lifetime bounds
  --> src\main.rs:20:53
   |
20 | fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a {
   |                                                     ^^^^^^^^^^^^^^^^

error: aborting due to previous error(s)

• Changing _body to have an explicit lifetime like _body: &'b B where 'b is independent of 'a or where 'a: 'b does not change the error. This and the original error make it seem that returning an impl trait is somehow causing the _body parameter to get the 'a lifetime, even though it's clearly unused, let alone used in a way that it would require the 'a lifetime.

• Changing (1) from impl Future + 'a to SomeFuture<'a> fixes it.

• Changing (1) from impl Future + 'a to Box<Future + 'a> and returning Box::new(SomeFuture(self)) fixes it.

[Arnavion]

From some experimentation, it seems to be because fn foo<T>(_: T) -> impl Bar compiles to something like fn foo<T>(_: T) -> impl Bar + 't, where 't is the lifetime of T. (I don't think this relationship can be expressed in regular Rust code, though Ralith on IRC suggested one could consider the anonymous type to contain PhantomData<T>)

Thus in the original code fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a effectively forces the return type to be bounded by B's lifetime in addition to 'a. This is why changing _body: &B to _body: B does not change anything, nor does annotating the parameter as _body: &'b B for an unconstrained 'b

---

In light of that, here's a smaller repro:

#![feature(conservative_impl_trait)]

trait Tr { }

struct S;
impl Tr for S { }

fn foo<T>(_t: T) -> impl Tr {
    S
}

struct S2;

fn main() {
    let _bar = {
        let s2 = S2;
        foo(&s2)
    };
}

which complains that s2 is dropped while still borrowed because the compiler thinks it needs to live as long as _bar.

[cramertj]

This is an intentional restriction. See RFC 1951 for the reasoning.

[Arnavion]

Sure. So can there be a way that I can convince the compiler that Client::post's and foo's results don't depend on all the inputs? Maybe it can be made to not override explicit lifetimes like it does in the Client::post example? (I recognize this won't help for the foo example since there is no way to ascribe a lifetime to T.)

The reasoning in the RFC is that eventually there will be syntax that provides control over which lifetime are and are not part of the returned existential ("Assumption 1"). But for the OP example where there already is an explicit lifetime and bound so it could be made to work today.

[cramertj]

Oh, I understand what you're saying now. Yes, if the return type contains an explicit lifetime bound, the compiler should be able to understand that the returned type outlives that lifetime bound. Currently, it cannot do that if there are type parameters involved. This should be fixed. Thanks for the report!

[Arnavion]

This is fixed by existential_type

• existential type PostFuture<'a>: Future + 'a; fn post<'a, B>(&'a self, _body: &B) -> PostFuture<'a> {

• existential type SomeTr: Tr; fn foo<T>(_t: T) -> SomeTr {

The fact that existential types intentionally don't have the "generic lifetime inheriting" behavior that impl trait has is documented here.

[alexreg]

@cramertj @pnkfelix This should probably be closed for the same reason as https://github.com/rust-lang/rust/issues/53450, if I'm not mistaken?

[cramertj]

No, this is still a bug. The fact that existential type provides a workaround does not mean that the code here shouldn't work-- if you say that your return type outlives some lifetime, then it shouldn't also be bound by the lifetime of a type that cannot be contained in the return type.

e.g. this doesn't compile today and shouldn't:

trait X {}
impl<T> X for T {}
fn foo<'a, T>(x: &'a u8, t: T) -> impl X + 'a {
    (x, t)
}

You have to explicitly add T: 'a in order for the return type to satisfy the impl X + 'a bound. If T didn't already outlive 'a, it couldn't appear in the return type.

[alexreg]

@cramertj Okay, so this is an issue with a lifetime inference, right? (Not actual bounds checking.) Would you mind writing up mentoring instructions so someone can tackle this? (Maybe even me.)

[alexreg]

@nikomatsakis Can we do something about this soon you think? :-) Seems kind of urgent to me, though perhaps this affects me more than most users.

[aidanhs]

Just to check, this is the same issue right?

use std::path::Path;

trait RR {}
impl RR for () {}

fn foo<'a>(path: &'a Path) -> impl RR + 'static {
    bar(path)
}
fn bar<P: AsRef<Path>>(path: P) -> impl RR + 'static {
    ()
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0a65678ab18f34888291c9c514ec2834

gives

error: cannot infer an appropriate lifetime
 --> src/lib.rs:7:9
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static {
  |                               ----------------- this return type evaluates to the `'static` lifetime...
7 |     bar(path)
  |         ^^^^ ...but this borrow...
  |
note: ...can't outlive the lifetime 'a as defined on the function body at 6:8
 --> src/lib.rs:6:8
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static {
  |        ^^
help: you can add a constraint to the return type to make it last less than `'static` and match the lifetime 'a as defined on the function body at 6:8
  |
6 | fn foo<'a>(path: &'a Path) -> impl RR + 'static + 'a {
  |                               ^^^^^^^^^^^^^^^^^^^^^^

(I also think the suggestion is misleading - won't adding a 'a after 'static' be a no-op since it will choose the longer of the two?)

[cramertj]

@aidanhs Yes to both (well, the answer to the second question is a bit more complicated, but "it won't work" is correct ;) ).

[Marwes]

Also encountered this issue with the following (minimised) code. Also found a workaround by specifying a dummy trait and moving the (static) lifetime there.

Errors

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=56e0e8506c1b32ee44d471d5cb6ed215

trait Parser2 {
    type Input;
    type PartialState;
}

struct Test<I>(::std::marker::PhantomData<fn(I)>);

impl<I> Parser2 for Test<I> {
    type Input = I;
    type PartialState = ();
}

fn line<'a, I>() -> impl Parser2<Input = I, PartialState = impl Send + 'static> {
    Test(::std::marker::PhantomData)
}

fn status<'a, I>() -> impl Parser2<Input = I, PartialState = impl Send + 'static> {
    line()
}

fn main() {
}

Works

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=74f4abad271f12df1d79a133007ee07f

trait Parser2 {
    type Input;
    type PartialState;
}

struct Test<I>(::std::marker::PhantomData<fn(I)>);

impl<I> Parser2 for Test<I> {
    type Input = I;
    type PartialState = ();
}

trait Static: Send + 'static {}
impl<T> Static for T where T: Send + 'static {}

fn line<'a, I>() -> impl Parser2<Input = I, PartialState = impl Static> {
    Test(::std::marker::PhantomData)
}

fn status<'a, I>() -> impl Parser2<Input = I, PartialState = impl Static> {
    line()
}

[alexreg]

@nikomatsakis Would be curious to get your thoughts on this along with the other not-too-dissimilar covariant lifetimes issue.

[nikomatsakis]

I've often found it pretty useful to model impl Trait in terms of associated types. If we include the effects of the default keyword, in fact, we get a very close modeling (without the inference), except for "auto trait propagation". I think this modeling is relevant here.

the model

Basically an impl Trait instance like

fn post<'a, B>(&'a self, _body: &B) -> impl Future + 'a { .. }

can be modeled as if there were a "one-off" trait with a single impl:

trait Post<'a, B> {
    //     ^^^^ these are the "captured" parameters, per RFC 1951
    type Output: Future + 'a;
}

impl<'a, B> Post for () {
    default type Output = /* the hidden type that compiler infers */;
}

and the function were then declared like so:

fn post<'a, B>(&'a self, _body: &B) -> <() as Post>::Output { .. }

Why does this matter?

Look at the next function, login:

fn login<'a>(client: &'a Client, username: &str) -> impl Future + 'a { .. }

this function winds up inferring that the hidden type X is <() as Post<'0, B>>::Output for some fresh inference variable '0, and hence we must infer that X: Future and X: 'a. Since the Output is declared as default in the impl, we can't fully normalize it, and must instead rely on the declared bounds (type Output: Future + 'a, where the 'a here will be the variable '0).

The bug report here seems correct: we should be able to prove that X: 'a by adding the requirement that '0: 'a and not requiring B: 'a. And if you try to write out the model I wrote above, you'll find that the compiler does so (at least I expect you will).

How does this work for associated types?

The compiler handles similar problems already for associated types. The rules were outlined in RFC 1214, along with some of the challenges. I think we should probably be able to apply the compiler's existing heuristics to this problem, but it might take a bit of work.

The relevant function is here:

https://github.com/rust-lang/rust/blob/daa53a52a2667533d5fe59bfcc5b8614b79c3d31/src/librustc/infer/outlives/obligations.rs#L357-L362

In particular, I believe this heuristic is the one that will help:

https://github.com/rust-lang/rust/blob/daa53a52a2667533d5fe59bfcc5b8614b79c3d31/src/librustc/infer/outlives/obligations.rs#L445-L459

[chpio]

this stackoverflow question seems related + a funny error message from the compiler about the lifetime, that could be changed?!

[kornelski]

A smaller test case from the forum (fails to compile in 1.39-nightly):

fn foo<T>(_: T) -> impl Iterator<Item = i32> + 'static {
    vec![2].into_iter()
}

fn bar() {
    let input = vec![1];
    let output0 = foo(&input); //output0 should have nothing to do with input
    let output1 = foo(input);
}

[Arnavion]

Updating the workaround I mentioned with the new syntax for trait aliases:

• type PostFuture<'a> = impl Future + 'a; fn post<'a, B>(&'a self, _body: &B) -> PostFuture<'a> {

• type SomeTr = impl Tr; fn foo<T>(_t: T) -> SomeTr {

[Twey]

Smallest test case I've managed to produce:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cf4582e81175a18eac9e2abc39c725a2
rust fn foo<T>(_: T) -> impl Send { } fn bar<'a>(x: &'a ()) -> impl Send { foo(x) }

[alexreg]

This should probably be triaged at this point, so we can decide how much effort to devote to it (and how soon). @Centril?

[ColonelThirtyTwo]

This really sucks for futures. For example, the return value for this won't be 'static at all; it will be whatever lifetime T has.

fn send<T: Serialize>(&self, body: &T) -> impl Future<Output=Result<Response, Error>> + 'static {
    let prepared_body: Vec<u8> = serde_json::to_vec(body);
    // We don't need `body` anymore... but rust thinks we do.
    async move {
        let resp = self.client.send(prepared_body?).await?;
        Ok(serde_json::from_slice(&resp)?)
    }
}

What's especially confusing is that even though I marked the impl as 'static, when I actually tried to use it as such, the compiler was assigning the return type a non-static lifetime.

At the very least, please add a warning; as far as I can tell, bounding the impl lifetime with 'static is useless because of the other implicit lifetimes it gains.

[ColonelThirtyTwo]

Any news on this? Just ran into this again.

[djc]

I wrote up a separate issue on potentially improving the diagnostics for this type of thing: https://github.com/rust-lang/rust/issues/78402.