Rust: Tracking issue for DST coercions (coerce_unsized, unsize) stabilization

[aturon]

DST coercions were the last part of DST to land, and are still undergoing some changes.

Before stabilization, at the very least the API surface here needs some scrutiny from the libs team for naming and other conventions issues.

cc @nrc @Gankro

Things to consider prior to stabilization

• Interaction with pin, see this internals thread and #68015 for details.

[aturon]

@nrc, what's your sense of the status for these features? Do you feel ready to stabilize after the libs team takes a look?

[nrc]

My feeling (and it is really just a feeling, I have no data to back this up) is that DST coercions haven't yet got the use they need to prove themselves for stabilisation. I'd love to be wrong about this - if someone could show me a bunch of code doing stuff with DST coercions I'd be very happy. OTOH, I don't think there is anything fundamentally wrong with them, so it would not be a waste of time for the libs team to take a look.

[photino]

Why is the marker named Unsize? Unsized sounds better, since we already have Sized.

[arielb1]

@photino

It goes the wrong way around - for example, we have [i8; 2]: Unsize<[u8]> (and also [i8; 2]: Unsize<fmt::Debug>). We _can_ swap the order of parameters - U: Unsized<T> if U is thean unsized version of T.

[rphmeier]

Can we move to stabilize this? It's been unchanged for 5 months and I've heard no complaints about it.

[Gankra]

I don't recall having any issues with this in libstd. The correct way to use this is basically a witchcraft incantation (mostly because these are under-the-covers things one never interacts with in real code), but as far as I know, there's know way to use it _dangerously_ wrong? The whole thing Just Works.

I guess one thing I've never really thought about is why one needs to talk about whether the pointed-to value is unsizeable or whatever. Why can't one simply declare that pointer X is an unsizable one, and have all that junk automatic? Is this desirable to have for generic consumers...?

[jnicholls]

It's desirable for generic consumers of e.g. Arena allocators that allocate a Sized type that implements a trait unknown to the allocator, and the consumer wants to coerce the allocated result to an allocated trait object.

See https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs in the tests. It's a decent use case. How would you recommend doing this otherwise?

[stevenblenkinsop]

Is the lack of support for enums deliberate, temporary, or merely overlooked?

[aturon]

I'm nominating this for discussion at the next lang team meeting.

[Gankra]

@jnicholls maybe I'm missing something, but just mark the one pointer field that will be Unsized as coercable: https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs#L41

It seems to me that all the impl<...> CoerceUnsized for ... is trivially derivable from the claim that that field should be coercable.

[jnicholls]

@Gankro Sorry I don't follow what you mean. Can you provide an example?

[Gankra]

Oops, something clobbered my selection, I meant to link to: https://gitlab.com/Bastacyclop/rust_arena/blob/master/src/lib.rs#L59

Basically instead of

pub struct Allocated<'a, 'b, T: 'b + ?Sized> {
    reference: &'b mut T,
    phantom: PhantomData<&'a ()>
}

impl<'a, 'b, T: ?Sized, U: ?Sized> CoerceUnsized<Allocated<'a, 'b, U>> for Allocated<'a, 'b, T>
    where U: 'b, T: 'b + Unsize<U> {}

You would just do _something_ like:

#[coerce_unsized(T)]
pub struct Allocated<'a, 'b, T: 'b + ?Sized> {
    reference: &'b mut T,
    phantom: PhantomData<&'a ()>
}

[aturon]

cc @eddyb

[aturon]

Removed T-libs tag, but note that we'd like to review the naming conventions here before stabilization.

[eddyb]

There's room for generalization and a good chance we might want Source: Coerce<Target>.
See this comment in response to wanting &mut T reborrows working through structs.

[apasel422]

Right now, because T does not implement Unsize<T>, one cannot use CoerceUnsized to support optional coercions in a generic context. (My use case is supporting DSTs with the placement API). Since &T can trivially be coerced to &T, would it make sense to add the identity impl?

[tomaka]

I'm not very familiar with how this works, but shouldn't the T of CoerceUnsized be T: ?Sized?
Right now T is required to be Sized, which doesn't make sense for me. Example.

[arielb1]

@tomaka

You implement CoerceUnsized on pointer types, not on DST types.

e.g.

impl<'a, T: ?Sized+Unsize<U>, U: ?Sized> CoerceUnsized<&'a mut U> for &'a mut T {}

[alexcrichton]

(adding T-libs as this has a library API associated with it which needs stabilization)

[nikomatsakis]

So at a recent @rust-lang/lang meeting we discussed this. I think everyone felt...somewhat comfortable with the idea of going forward, but we also wanted to do a thorough review of the current status of the code, along with a comparison against the existing specifications, to make sure everything is well aligned.

[eddyb]

@nikomatsakis The problem, though, is that the RFC is _intentionally provisional_, mostly to get Rc and Arc working with DSTs. I would like to see an attempt at generalization before stabilizing it (I believe the compiler code would "just work" with more than one field being coerced, and various coercion modes, not just Unsize, with minimal changes).

Maybe with specialization we can even encode Deref coercions in a general Coerce trait.

[arielb1]

@eddyb

Deref coercions involve transitivity, which the trait system does not try to support.

[abonander]

@nrc triage ping

[nrc]

No change. I think we are waiting for some kind of custom DST RFC before moving forward with stabilisation here.

[H2CO3]

@nrc @Gankro Any progress with this? It's really annoying that my beloved custom smart pointers can't point to trait objects in the lack of stable Unsize and CoerceUnsized.

[zakarumych]

What if instead of using Unsize and CoerceUnsized directly in libraries we could implement From for all pointers from std which implement CoerceUnsized.
For example.

impl<T, U> From<Box<T>> for Box<U>
where
  T: Unsize<U>
{
  fn from(value: Box<T>) -> Box<U> {
    value as Box<U>
  }
}

In this case we could implement From for our pointer wrappers relying on From implementation of pointers.
And there will be no issues with converting multiple fields of the struct.

[Osspial]

It's been over a year since anyone's really looked at this. What's stopping this from getting stabilized?

[eddyb]

The current design is perma-unstable. A new RFC is required for a design that we can consider stabilizing.

The Unsize trait can be ignored, as T: Unsize<U> and *mut T: CoerceUnsized<*mut U> are pretty much equivalent.

So what we should do is generalize CoerceUnsized (rename to Coerce and handle more/all kinds of coercions through it).

Then we can allow even impl<T: Coerce<U>, U> Coerce<Option<U>> for Option<T>, which has been requested for a long while, too.

IMO "custom DSTs" was a red herring, as stabilizing Unsize is unnecessary for making coercions work.

[eddyb]

Nominating for discussion, regarding splitting this issue into:

• Unsize trait, reflects unsizing, may want to wait on "custom DSTs", irrelevant for "opt into unsizing coercions", can stay unstable forever (or even removed)
• CoerceUnsized, can be generalized to an "opt into structural coercion" Coerce trait, would cover all the cases "CoerceUnsized with Unsize bounds" cover today (and more), can be stabilized more readily

[Mcat12]

I just wanted to note (if it hasn't been noted already) that this feature makes it possible to write a struct with a generic trait parameter. This was useful for me because I could then require the other generics to implement the generic trait.

Related internals thread I found helpful:
https://internals.rust-lang.org/t/traits-as-generic-type-parameters/4950

Example (a mockable Rocket request guard):

pub enum Service<'r, Trait, Impl, Mock>
where
    Trait: ?Sized + 'r,
    Impl: Unsize<Trait> + FromRequest<'a, 'r>,
    Mock: Deref + FromRequest<'a, 'r>,
    <Mock as Deref>::Target: Unsize<Trait>
{
    Prod(Impl, PhantomData<&'r Trait>),
    Test(Mock)
}

pub type MyServiceGuard<'r> = Service<
    'r,
    MyServiceTrait,
    MyServiceImpl<'r>,
    rocket::State<'r, MyServiceMock>
>;

[mathstuf]

This feature is how I'm currently performing arc_of_t as arc_of_u given T: U (the type needs extra traits in the application, but a library doesn't know about that trait and just requires a "base" trait). Is there some other way of doing this on stable?

[eddyb]

Came across this again and... how did https://github.com/rust-lang/rust/issues/27732#issuecomment-480075569 result in nothing? Not even a comment?

Re-nominating (low priority, but should still not be unnominated before a decision).

[nikomatsakis]

@eddyb we discussed in @rust-lang/lang meeting today -- splitting off into two feature gates makes sense, provided we have a good description of what the two gates cover and the interaction between them. (e.g., as the issue comments)