Rust: [Stabilization] Pin APIs

[withoutboats]

@rfcbot fcp merge
Feature name: pin
Stabilization target: 1.32.0
Tracking issue: #49150
Related RFCs: rust-lang/rfcs#2349

This is a proposal to stabilize the pin library feature, making the "pinning"
APIs for manipulating pinned memory usable on stable.

(I've tried to write this proposal as a comprehensive "stabilization report.")

Stabilized feature or APIs

[std|core]::pin::Pin

This stabilizes the Pin type in the pin submodule of std/core. Pin is
a fundamental, transparent wrapper around a generic type P, which is intended
to be a pointer type (for example, Pin<&mut T> and Pin<Box<T>> are both
valid, intended constructs). The Pin wrapper modifies the pointer to "pin"
the memory it refers to in place, preventing the user from moving objects out
of that memory.

The usual way to use the Pin type is to construct a pinned variant of some
kind of owning pointer (Box, Rc, etc). The std library owning pointers all
provide a pinned constructor which returns this. Then, to manipulate the
value inside, all of these pointers provide a way to degrade toward Pin<&T>
and Pin<&mut T>. Pinned pointers can deref, giving you back &T, but cannot
safely mutably deref: this is only possible using the unsafe get_mut
function.

As a result, anyone mutating data through a pin will be required to uphold the
invariant that they never move out of that data. This allows other code to
safely assume that the data is never moved, allowing it to contain (for
example) self references.

The Pin type will have these stabilized APIs:

impl<P> Pin<P> where P: Deref, P::Target: Unpin

• fn new(pointer: P) -> Pin<P>

impl<P> Pin<P> where P: Deref

• unsafe fn new_unchecked(pointer: P) -> Pin<P>
• fn as_ref(&self) -> Pin<&P::Target>

impl<P> Pin<P> where P: DerefMut

• fn as_mut(&mut self) -> Pin<&mut P::Target>
• fn set(&mut self, P::Target);

impl<'a, T: ?Sized> Pin<&'a T>

• unsafe fn map_unchecked<U, F: FnOnce(&T) -> &U>(self, f: F) -> Pin<&'a U>
• fn get_ref(self) -> &'a T

impl<'a, T: ?Sized> Pin<&'a mut T>

• fn into_ref(self) -> Pin<&'a T>
• unsafe fn get_unchecked_mut(self) -> &'a mut T
• unsafe fn map_unchecked_mut<U, F: FnOnce(&mut T) -> &mut U>(self, f: F) -> Pin<&'a mut U>

impl<'a, T: ?Sized> Pin<&'a mut T> where T: Unpin

• fn get_mut(self) -> &'a mut T

Trait impls

Most of the trait impls on Pin are fairly rote, these two are important to
its operation:

• impl<P: Deref> Deref for Pin<P> { type Target = P::Target }
• impl<P: DerefMut> DerefMut for Pin<P> where P::Target: Unpin { }

std::marker::Unpin

Unpin is a safe auto trait which opts out of the guarantees of pinning. If the
target of a pinned pointer implements Unpin, it is safe to mutably
dereference to it. Unpin types do not have any guarantees that they will not
be moved out of a Pin.

This makes it as ergonomic to deal with a pinned reference to something that
does not contain self-references as it would be to deal with a non-pinned
reference. The guarantees of Pin only matter for special case types like
self-referential structures: those types do not implement Unpin, so they have
the restrictions of the Pin type.

Notable implementations of Unpin in std:

• impl<'a, T: ?Sized> Unpin for &'a T
• impl<'a, T: ?Sized> Unpin for &'a mut T
• impl<T: ?Sized> Unpin for Box<T>
• impl<T: ?Sized> Unpin for Rc<T>
• impl<T: ?Sized> Unpin for Arc<T>

These codify the notion that pinnedness is not transitive across pointers. That
is, a Pin<&T> only pins the actual memory block represented by T in a
place. Users have occassionally been confused by this and expected that a type
like Pin<&mut Box<T>> pins the data of T in place, but it only pins the
memory the pinned reference actually refers to: in this case, the Box's
representation, which a pointer into the heap.

std::marker::Pinned

The Pinned type is a ZST which does not implement Unpin; it allows you to
supress the auto-implementation of Unpin on stable, where !Unpin impls
would not be stable yet.

Smart pointer constructors

Constructors are added to the std smart pointers to create pinned references:

• Box::pinned(data: T) -> Pin<Box<T>>
• Rc::pinned(data: T) -> Pin<Rc<T>>
• Arc::pinned(data: T) -> Pin<Arc<T>>

Notes on pinning & safety

Over the last 9 months the pinning APIs have gone through several iterations as
we have investigated their expressive power and also the soundness of their
guarantees. I would now say confidently that the pinning APIs stabilized here
are sound and close enough to the local maxima in ergonomics and
expressiveness; that is, ready for stabilization.

One of the trickier issues of pinning is determining when it is safe to perform
a pin projection: that is, to go from a Pin<P<Target = Foo>> to a
Pin<P<Target = Bar>>, where Bar is a field of Foo. Fortunately, we have
been able to codify a set of rules which can help users determine if such a
projection is safe:

1. It is only safe to pin project if (Foo: Unpin) implies (Bar: Unpin): that
   is, if it is never the case that Foo (the containing type) is Unpin while
   Bar (the projected type) is not Unpin.
2. It is only safe if Bar is never moved during the destruction of Foo,
   meaning that either Foo has no destructor, or the destructor is carefully
   checked to make sure that it never moves out of the field being projected to.
3. It is only safe if Foo (the containing type) is not repr(packed),
   because this causes fields to be moved around to realign them.

Additionally, the std APIs provide no safe way to pin objects to the stack.
This is because there is no way to implement that safely using a function API.
However, users can unsafely pin things to the stack by guaranteeing that they
never move the object again after creating the pinned reference.

The pin-utils crate on crates.io contains macros to assist with both stack
pinning and pin projection. The stack pinning macro safely pins objects to the
stack using a trick involving shadowing, whereas a macro for projection exists
which is unsafe, but avoids you having to write the projection boilerplate in
which you could possibly introduce other incorrect unsafe code.

Implementation changes prior to stabilization

• [ ] Export Unpin from the prelude, remove pin::Unpin re-export

As a general rule, we don't re-export things from multiple places in std unless
one is a supermodule of the real definition (e.g. shortening
std::collections::hash_map::HashMap to std::collections::HashMap). For this
reason, the re-export of std::marker::Unpin from std::pin::Unpin is out of
place.

At the same time, other important marker traits like Send and Sync are included
in the prelude. So instead of re-exporting Unpin from the pin module, by
putting in the prelude we make it unnecessary to import std::marker::Unpin,
the same reason it was put into pin.

• [ ] Change associated functions to methods

Currently, a lot of the associated function of Pin do not use method syntax.
In theory, this is to avoid conflicting with derefable inner methods. However,
this rule has not been applied consistently, and in our experience has mostly
just made things more inconvenient. Pinned pointers only implement immutable
deref, not mutable deref or deref by value, limiting the ability to deref
anyway. Moreover, many of these names are fairly unique (e.g. map_unchecked)
and unlikely to conflict.

Instead, we prefer to give the Pin methods their due precedence; users who
need to access an interior method always can using UFCS, just as they would be
required to to access the Pin methods if we did not use method syntax.

• [ ] Rename get_mut_unchecked to get_unchecked_mut

The current ordering is inconsistent with other uses in the standard library.

• [ ] Remove impl<P> Unpin for Pin<P>

This impl is not justified by our standard justification for unpin impls: there is no pointer direction between Pin<P> and P. Its usefulness is covered by the impls for pointers themselves.

This futures impl will need to change to add a P: Unpin bound.

• [ ] Mark Pin as repr(transparent)

Pin should be a transparent wrapper around the pointer inside of it, with the same representation.

Connected features and larger milestones

The pin APIs are important to safely manipulating sections of memory which can
be guaranteed not to be moved out. If the objects in that memory do not
implement Unpin, their address will never change. This is necessary for
creating self-referential generators and asynchronous functions. As a result,
the Pin type appears in the standard library future APIs and will soon
appear in the APIs for generators as well (#55704).

Stabilizing the Pin type and its APIs is a necessary precursor to stabilizing
the Future APIs, which is itself a necessary precursor to stabilizing the
async/await syntax and moving the entire futures 0.3 async IO ecosystem
onto stable Rust.

cc @cramertj @RalfJung

[withoutboats]

@rfcbot fcp merge

[rfcbot]

Team member @withoutboats has proposed to merge this. The next step is review by the rest of the tagged team members:

• [x] @Kimundi
• [ ] @SimonSapin
• [x] @alexcrichton
• [x] @dtolnay
• [ ] @sfackler
• [x] @withoutboats

Concerns:

• box-pinnned-vs-box-pin (https://github.com/rust-lang/rust/issues/55766#issuecomment-443371976)
• improving-the-documentation (https://github.com/rust-lang/rust/issues/55766#issuecomment-443367737)
• naming-of-Unpin resolved by https://github.com/rust-lang/rust/issues/55766#issuecomment-445074980
• self-methods (https://github.com/rust-lang/rust/issues/55766#issuecomment-443368794)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[alexcrichton]

Thanks for the detailed writeup here @withoutboats! I've also sort of been historically confused by the various guarantees of Pin, and I've currently got a few questions about the APIs being stabilized here wrt the safety guarantees. To help sort this out in my own head though I figured I'd try to write these things down.

In trying to start writing this down though I keep running up against a wall of "what is Unpin?" I'm sort of confused by what that is and the various guarantees around it. Can you say again what it means for T to and also to not implement Unpin? Also, if Unpin is a safe trait to implement it naively seems like it could be used to easily undermine the unsafe guarantees of Pin<T>, but I'm surely missing something

[tinaun]

if i have got this correct, every type that is not self referential (ie: not a generator) is Unpin

[Nemo157]

It's not just self-referentiality, there are some other use-cases for stable memory addresses that Pin can also support. They are relatively few and far-between though.

How I understand Unpin being safe to implement is that by implementing it you may violate an invariant required of other unsafe code you have written (crucially, only you may write this unsafe code, no external code can be relying on whether you have implemented Unpin). There is nothing you can do with Pin's safe API that will cause unsoundness whether or not you have implemented Unpin. By opting in to using some of Pin's unsafe API you are guaranteeing that you will only implement Unpin when it is safe to do so. That is covered by point 1 of the "Notes on pinning & safety" section above.

[alexcrichton]

Hm I still don't really understand Unpin. I'm at first just trying to understand what it means to implement or not to impelment Unpin.

First off, it's probably helpful to know what types implement Unpin automatically! It's mentioned above that common pointer types (Arc/Rc/Box/references) implement Unpin, but I think that's it? If this is an auto trait, does that mean that a type MyType automatically implements Unpin if it only contains pointers? Or do no other types automatically implement Unpin?

I keep trying to summarize or state what Unpin guarantees and such, but I'm finding it really difficult to do so. Can someone help out by reiterating again what it means to implement Unpin as well as what it means to not implement Unpin?

I think I understand the guarantees of Pin<P> where you can't move out of any of the inline members of P::Target, but is that right?

[withoutboats]

@alexcrichton Thanks for the questions, I'm sure the pinning APIs can be a bit confusing for people who haven't been part of the group focusing on them.

First off, it's probably helpful to know what types implement Unpin automatically!

Unpin is an auto trait like Send or Sync, so most types implement it automatically. Generators and the types of async functions are !Unpin. Types that could contain a generator or an async function body (in other words, types with type parameters) are also not automatically Unpin unless their type parameters are.

The explicit impls for pointer types is to make the Unpin even if their type parameters are not. The reason for this will hopefully be clearer by the end of this comment.

Can someone help out by reiterating again what it means to implement Unpin as well as what it means to not implement Unpin?

Here is the sort of fundamental idea of the pinning APIs. First, given a pointer type P, Pin<P> acts like P except that unless P::Target implements Unpin, it is unsafe to mutably dereference Pin<P>. Second, there are two basic invariants unsafe code related to Pin has to maintain:

• If you unsafely get an &mut P::Target from a Pin<P>, you must never move P::Target.
• If you can construct a Pin<P>, it must be guaranteed that you'll never be able to get an unpinned pointer to the data that pointer points to until the destructor runs.

The implication of all of this is that if you construct a Pin<P>, the value pointed to by P will never move again, which is the guarantee we need for self-referential structs as well as intrusive collections. But you can opt out of this guarantee by just implementing Unpin for your type.

So if you implement Unpin for a type, you're saying that the type opts out of any additional safety guarantees of Pin - its possible to mutably dereference the pointers pointing to it. This means you're saying the type does not need to be immovable to be used safely.

Moving a pointer type like Rc<T> doesn't move T because T is behind a pointer. Similarly, pinning a pointer to an Rc<T> (as in Pin<Box<Rc<T>>) doesn't actually pin T, it only pins that particular pointer. This is why anything which keeps its generics behind a pointer can implement Unpin even when their generics don't.

[withoutboats]

Also, if Unpin is a safe trait to implement it naively seems like it could be used to easily undermine the unsafe guarantees of Pin, but I'm surely missing something

This was one of the trickiest parts of the pinning API, and we got it wrong at first.

Unpin means "even once something has been put into a pin, it is safe to get a mutable reference to it." There is another trait that exists today that gives you the same access: Drop. So what we figured out was that since Drop is safe, Unpin must also be safe. Does this undermine the entire system? Not quite.

To actually implement a self-referential type would require unsafe code - in practice, the only self-referential types anyone cares about are those that the compiler generates for you: generators and async function state machines. These explicitly say they don't implement Unpin and they don't have a Drop implementation, so you know, for these types, once you have a Pin<&mut T>, they will never actually get a mutable reference, because they're an anonymous type that we know doesn't implement Unpin or Drop.

The problem emerges once you have a struct containing one of these anonymous types, like a future combinator. In order go from a Pin<&mut Fuse<Fut>> to a Pin<&mut Fut>, you have to perform a "pin projection." This is where you can run into trouble: if you pin project to the future field of a combinator, but then implement Drop for the combinator, you can move out of a field that is supposed to be pinned.

For this reason, pin projection is unsafe! In order to perform a pin projection without violating the pinning invariants, you have to guarantee that you never do several things, which I listed in the stabilization proposal.

So, the tl;dr: Drop exists, so Unpin must be safe. But this doesn't ruin the whole thing, it just means that pin projection is unsafe and anyone who wants to pin project needs to uphold a set of invariants.

[Matthias247]

generators and async function state machines. These explicitly say they don't implement Unpin and they don't have a Drop implementation, so you know, for these types, once you have a Pin<&mut T>, they will never actually get a mutable reference, because they're an anonymous type that we know doesn't implement Unpin or Drop.

Shouldn't an async state machine have a Drop implementation? Things that are on the "stack" of the async function (which probably equals fields in the state machine) need to get destructed when the async function completes or gets cancelled. Or does this happen otherwise?

[SimonSapin]

I guess what matters in this context is whether an impl Drop for Foo {…} item exists, which would run code with &mut Foo which could use for example mem::replace to "exfiltrate" and move the Foo value.

This is not the same as "drop glue", which can be called through ptr::drop_in_place. Drop glue for a given Foo type will call Drop::drop if it’s implemented, then recursively call the drop glue for each field. But those recursive calls never involve &mut Foo.

[Nemo157]

Also, while a generator (and therefore an async state machine) has custom drop glue, it's just to drop the correct set of fields based on the current state, it promises to never move any of the fields during drop.

[withoutboats]

The terminology I use (though I don't think there's any standard): "drop glue" is the compiler generated recursive walking of fields, calling their destructors; "Drop implementation" is an implementation of the Drop trait, and "destructor" is the combination of the drop glue and the Drop implementation. The drop glue never moves anything around, so we only care about Drop implementations.

[Gankra]

Is someone on the hook to write a nomicon chapter on futures? It seems incredibly necessary given how subtle this is. In particular I think examples, especially examples of buggy/bad implementations and how they're unsound, would be illuminating.

[cramertj]

@Gankro Sure, you can put me down for that.

[Matthias247]

Thanks for the explanation everyone!

I personally am super new to async Rust and the Pin APIs, but I have played a bit around with it during the last days (https://github.com/rust-lang-nursery/futures-rs/pull/1315 - where I tried to exploit pinning for intrusive collections in async code).

During experimentation I got some concerns with these APIs:

• The concept of pinning seems very hard to fully understand, even if one knows that stable memory addresses are required. Some challenges that I faced:
  
  
  
  • Error messages like:
    
    >within impl core::future::future::Future+futures_core::future::FusedFuture, the trait std::marker::Unpin is not implemented for std::marker::Pinned
    
    This one sounds pretty recursive and contradicting (why should something that is pinned be not pinned?)
  
  
  • How can I access mutable fields in my method that takes a Pin as a parameter. It took a bit of search, learning about new terms (Pin projection), trying to copy the methods from pin-utils, until I came to a solution that uses 2 unsafe methods to do what I needed.
  
  
  • How can I actually use my future in an async method and a select case. It turned out I needed to pin_mut!() it to futures stack. Which isn't really a stack, which makes it confusing.
  
  
  • Why does my drop() method now again take &mut self, instead of a Pin.
  
  
  • Overall it was a bit of a feeling of randomly using unsafe pinning related APIs in the hope of getting things to compile, which definitely shouldn't be the ideal situation. I suspect others might try to do similar things.
  
  
• The concept propagates a lot throughout the codebase, which seems to force lots of people to understand what Pin and Unpin actually are. This seems like a bit a leak of implementation details for some more special use-cases.
• Pin seems to be somewhat related to lifetimes, but also somewhat orthogonal. Basically a Pin tells us we might see an object again with exactly the same address until drop() is called. Which gives it some kind of virtual lifetime between the current scope and 'static. It seems confusing to have 2 concepts which are related, but still completely different. I wondered whether it can be modelled by something like 'pinned.

While getting things to compile, I often thought about C++, where most of these issues are avoided: Types can be declared unmovable by deleting the move constructor and assignment operator. When a type is not moveable, types which contain that type are also not. Thereby the property and requirements flow natively through the type hierarchy and get checked by the compiler, instead of forwarding the property inside some calls (not all, e.g. not drop()). I think this is a lot easier to understand. But maybe not applicable to Rust, since Futures must currently often be moved before something starts polling them? But on the other hand that might be fixable with a new definition with that trait, or separating movement and polling phase.

Regarding Alex Unpin comment: I'm slowly grasping what Unpin means, but I agree that it's hard to understand. Maybe another name helps, but I can't find a concise one right now. Something along ThingsInsideDontBreakIfObjectGetsMoved, DoesntRequireStableAddress, PinDoesntMatter, etc.

What I haven't yet fully understood is, why getting &mut self from Pin<&mut self> can't be safe for all types. If Pin would just mean the address of the object itself is stable, then this should hold true for most typical Rust types. It seems there is another concern integrated into Pin, which is that also self-referential types inside it don't break. For those types manipulating them after having a Pin is always unsafe. But I think in most cases those will be compiler-generated, and unsafety or even raw pointers there should be fine. For a manually future combinators that need to forward pinned calls to subfields there would obviously need to be unsafe calls to create pins to the fields before polling those. But in that case I see the unsafety more related to the place where it actually matters (is the pin still valid?) than to accessing other fields for which it doesn't matter at all.

Another thought that I had is whether it's possible to get a simpler system if some limitations are applied. E.g. if things that require pinning could only be used inside async methods and not with future combinators. Maybe that could simplify things to one of Pin or Unpin. Considering that for lots of code async/await methods with select/join support might be favorable to combinators, this might not be the hugest loss. But I haven't really put enough thoughts into whether this really helps.

On the positive side: Being able to write async/await code with "stack" borrows is pretty cool and much needed! And ability to use pinning for other use-cases, like intrusive collections, will help performance as well as targets like embedded systems or kernels. So I'm really looking forward to a solution on this.

[glaebhoerl]

Minor nit w.r.t. the stabilization report:

fn get_unchecked_mut(self) -> &'a mut T

I'm guessing this is actually an unsafe fn?

[tikue]

@Matthias247 you can't safely get &mut T from Pin<P<T>> if T isn't Unpin because then you could mem::swap to move T out of the Pin, which would defeat the purpose of pinning things.

One thing I have trouble explaining to myself is what makes Future fundamentally different from other traits such that Pin needs to be part of its API. I mean, I know intuitively it's because async/await requires pinning, but does that say something specifically about Futures that's different from, say, Iterators?

Could poll take &mut self and only implement Future for Pin<P> types or types that are Unpin?

[Nemo157]

How can I access mutable fields in my method that takes a Pin as a parameter.

This actually makes me wonder whether there's a couple of missing methods on Pin,

impl<'a, T: ?Sized> Pin<&'a T> {
    fn map<U: Unpin, F: FnOnce(&T) -> &U>(self, f: F) -> Pin<&'a U>
}

impl<'a, T: ?Sized> Pin<&'a mut T> {
    fn map<U: Unpin, F: FnOnce(&mut T) -> &mut U>(self, f: F) -> Pin<&'a mut U>
}

These could be used in the pin_utils::unsafe_unpinned! macro.

I'm trying to work out _why_ this macro claims to be unsafe. If we are !Unpin and have a Unpin field, why would it be unsafe to project to this field?

The only situation where I can see it being unsound is implementing a custom !Unpin type, taking a raw pointer to an Unpin field of self (and relying on it having a stable address/pointing to the same instance), then acquiring an &mut to the same field and passing it to an external function. This seems to fall under the same reasoning of why implementing Unpin is safe though, by taking a raw pointer to a field of a !Unpin you are opting out of being able to call some of the safe APIs.

To make the previous situation safer there could be a wrapper built on Pinned for marking which normally Unpin fields of a struct are actually !Unpin instead of just adding a Pinned to the struct as a whole:

pub struct MustPin<T: Unpin>(T, Pinned);

impl<T: Unpin> MustPin<T> {
    pub const fn new(t: T) -> Self { ... }
    pub fn get(self: Pin<&Self>) -> *const T { ... }
    pub fn get_mut(self: Pin<&mut Self>) -> *mut T { ... }
}

---

This all seems backwards compatible with the current API, it could remove some of the unsafety from futures-rs combinators (e.g. it would give safe access to extra fields like this one), but isn't necessary for the current usecases. We could probably experiment with some APIs like this for implementing custom !Unpin types (like intrusive collections) and add them later on.

[withoutboats]

@Nemo157 Those map functions are unsafe because I could mem::swap on the &mut T the function passes me before returning an &mut U. (well the mutable one is, the immutable one might not be unsafe)

EDIT: Also the pin-utils macro is different, unsafe_unpinned doesnt have to do with the target type being Unpin, its just an "unpinned projection" - a projection to a &mut Field. Its safe to use as long as you never pin project to that field, even if the field is !Unpin.

[withoutboats]

One thing I have trouble explaining to myself is what makes Future fundamentally different from other traits such that Pin needs to be part of its API. I mean, I know intuitively it's because async/await requires pinning, but does that say something specifically about Futures that's different from, say, Iterators?

There's no theoretical difference, but there are pragmatic ones. For one, Iterator is stable. But as a result, unless we figure out something very clever, you'll never be able to run a for loop over a self-referential generator without a double indirection, even though that should be completely safe without it (because the for loop will consume and never move the generator).

Another important pragmatic difference is that the code patterns between Iterator and Future are quite different. You can't go 10 minutes without wanting to borrow across an await point in a future, which is why on futures 0.1 you see these Arcs appearing all over the place so you can use the same variable in two different and_then calls. But we've gotten quite far without being able to express self-referential iterators at all, it's just not that important of a use case.

[Nemo157]

Those map functions are unsafe because I could mem::swap on the &mut T the function passes me before returning an &mut U

Ah, damn, forgot about that part of it :frowning:

[RalfJung]

fn as_mut(&mut self) -> Pin<&mut P::Target> fn into_ref(self) -> Pin<&'a T>`

Should these be called as_pinned_mut and into_pinned_ref, to avoid confusing them with methods that actually return references?

Notable implementations of Unpin in std:

We also have impl<P> Unpin for Pin<P> {}. Should this maybe become impl<P: Unpin> Unpin for Pin<P> {}? For the types we use this has no effect and it seems a bit safer? After all, Pin itself does not add a ptr indirection, so the argument about pinning not being structural does not apply.
Never mind, you have that on your list. ;)

Drop guarantee

I think we have to codify the Drop guarantee before stabilization, or else it will be too late:

It is illegal to invalidate the storage of a pinned object (the target of a Pin reference) without calling drop on the object.

"Invalidate" here can mean "deallocation", but also "repurposing": When changing x from Ok(foo) to Err(bar), the storage of foo gets invalidated.

This has the consequence, for example, that it becomes illegal to deallocate a Pin<Box<T>>, Pin<Rc<T>> or Pin<Arc<T>> without first calling drop::<T>.

Repurposing Deref, DerefMut

I still feel slightly uneasy about how this repurposes the Deref trait to mean "this is a smart pointer". We use Deref for other things as well, e.g. for "inheritance". That might be an anti-pattern, but it is still in fairly common use and, frankly, it is useful. :D

I think this is not a soundness issue though.

Understanding Unpin

Shameless plug: I wrote two blog posts on this, that might help or not depending on how much you enjoy reading (semi-)formal syntax. ;) The APIs have changed since then, but the basic idea has not.

• A Formal Look at Pinning
• Safe Intrusive Collections with Pinning

Safe pin projections

@Matthias247 I think one problem you are running into is that building abstractions that involve pinning currently almost always requires unsafe code. Using those abstractions is fine, but e.g. defining a future combinator in safe code won't work. The reason for this is that "pin projections" have constraints that we could only safely check with compiler changes. For example, you ask

Why does my drop() method now again take &mut self, instead of a Pin.

Well, drop() is old -- it exists since Rust 1.0 -- so we cannot change it. We'd love to just make it take Pin<&mut Self>, and then Unpin types could get their &mut like they do now, but that's a non-backwards-compatible change.

To make writing future combinators safe, we'd need safe pin projections, and for that we have to change the compiler, this cannot be done in a library. We could almost do it in a derive proc_macro, except that we'd need a way to assert "this type does not have any implementation of Drop or Unpin".

I think it'd be worth the effort to figure out how to get such a safe API for pin projections. However, that does not have to block stabilization: The API we stabilize here should be forward compatible with safe pin projections. (Unpin might have to become a lang item to implement the above assertion, but that doesn't seem too bad.)

Pinning is not !Move

I often thought about C++, where most of these issues are avoided: Types can be declared unmovable by deleting the move constructor and assignment operator. When a type is not moveable, types which contain that type are also not.

There were several attempts at defining unmovable types for Rust. It gets very complicated very quickly.

One important thing to understand is that pinning does not provide unmovable types! All types remain movable in Rust. If you have a T, you can move it anywhere you want. Instead of unmovable types, we make use of Rust's ability to define new encapsulated APIs, as we define a pointer type from which you cannot move out. All the magic is in the pointer type (Pin<&mut T>), not in the pointee (T). There is no way for a type to say "I cannot get moved ever". However, there is a way for a type to say "If I got pinned, don't move me again". So a MyFuture that I own will always be movable, but Pin<&mut MyFuture> is a pointer to an instance of MyFuture that cannot get moved any more.

This is a subtle point, and we probably should spend some time fleshing out the docs here to help avoid this very common misunderstanding.

[SimonSapin]

But we've gotten quite far without being able to express self-referential iterators at all, it's just not that important of a use case.

This is because, so far, all iterators are defined using a type and an impl Iterator for … block. When state needs to be kept between two iterations, there is no choice but to stash it in fields of the iterator type and work with &mut self.

However, even if this is not the primary motivation for including generators in the language, it would be very nice to eventually be able to use generator yield syntax to define something that can be used with a for loop. As soon as that exists, borrowing across yield because just as important for generator-iterators as it is for generator-futures.

[Nemo157]

(Unpin might have to become a lang item to implement the above assertion, but that doesn't seem too bad.)

Unpin and Pin already have to be lang items to support safe immovable generators.

[alexcrichton]

Ok thanks for the explanations all! I agree with @Gankro that a nomicon-style chapter about Pin and such would be massively useful here. I suspect that there's a lot of development history for why various safe methods exist and why unsafe methods are unsafe and such.

In an effort to help myself understand this though I wanted to try again to write down why each function is safe or why it's unsafe. With the understanding from above I've got all the following, but there's a few questions here and there. If others can help me fill this in that'd be great! (or help point out where my thinking is wrong)

• fn new(P) -> Pin<P> where P: Deref, P::Target: Unpin

  • This is a safe method to construct Pin<P>, and while the existence of
    Pin<P> normally means that P::Target will never be moved again in the
    program, P::Target implements Unpin which is read "the guarantees of
    Pin no longer hold". As a result, the safety here is because there's no
    guarantees to uphold, so everything can proceed as usual.

• unsafe fn new_unchecked(P) -> Pin<P> where P: Deref

  • Unlike the previous, this function is unsafe because P doesn't
    necessarily implement Unpin, so Pin<P> must uphold the guarantee that
    P::Target will never move ever again after Pin<P> is created.

  A trivial way to violate this guarantee, if this function is safe, looks
  like:

  fn foo<T>(mut a: T, b: T) {
      Pin::new_unchecked(&mut a); // should mean `a` can never move again
      let a2 = mem::replace(&mut a, b);
      // the address of `a` changed to `a2`'s stack slot
  }
  

  Consequently it's up to the user to guarantee that Pin<P> indeed means
  P::Target is never moved again after construction, so it's unsafe!

• fn as_ref(&Pin<P>) -> Pin<&P::Target> where P: Deref

  • Given Pin<P> we have a guarantee that P::Target will never move. That's
    part of the contract of Pin. As a result it trivially means that
    &P::Target, another "smart pointer" to P::Target will provide the same
    guarantee, so &Pin<P> can be safely translate to Pin<&P::Target>.

  This is a generic method to go from Pin<SmartPointer<T>> to Pin<&T>

• fn as_mut(&mut Pin<P>) -> Pin<&mut P::Target> where P: DerefMut

  • I believe the safety here is all roughly the same as the as_ref above.
    We're not handing out mutable access, just a Pin, so nothing can be easily
    violated yet.

  Question: what about a "malicious" DerefMut impl? This is a safey way
  to call a user-provided DerefMut which crates &mut P::Target natively,
  presumably allowing it to modify it as well. How's this safe?

• fn set(&mut Pin<P>, P::Target); where P: DerefMut

  • Question: Given Pin<P> (and the fact that we don't know anything about
    Unpin), shouldn't this guarantee that P::Target never moves? If we can
    reinitialize it with another P::Target, though, shouldn't this be unsafe?
    Or is this somehow related to destructors and all that?

• unsafe fn map_unchecked<U, FnOnce(&T) -> &U>(Pin<&'a T>, f: F) -> Pin<&'a U>

  • This is an unsafe function so the main question here is "why isn't this
    safe"? An example of violating guarantees if this were safe looks like:

  ...

  Question:: what's the counter-example here? If this were safe, what's
  the example that shows violating the guarantees of Pin?

• fn get_ref(Pin<&'a T>) -> &'a T

  • The guarantee of Pin<&T> means that T will never move. Returning &T
    doesn't allow mutation of T, so it should be safe to do while upholding
    this guarantee.

  One "maybe gotcha" here is interior mutability, what if T were
  RefCell<MyType>? This, however, doesn't violate the guarantees of
  Pin<&T> because the guarantee only applies to T as a whole, not the
  interior field MyType. While interior mutability can move around
  internals, it still fundamentally can't move the entire structure behind a
  & reference.

• fn into_ref(Pin<&'a mut T>) -> Pin<&'a T>

  • Pin<&mut T> means that T will never move. As a result it means Pin<&T>
    provides the same guarantee. Shouldn't be much issue with this conversion,
    it's mostly switching types.

• unsafe fn get_unchecked_mut(Pin<&'a mut T>) -> &'a mut T

  • Pin<&mut T> means that T must never move, so this is trivially unsafe
    because you can use mem::replace on the result to move T (safely). The
    unsafe here "although I am giving you &mut T, you're not allowed to ever
    move T".

• unsafe fn map_unchecked_mut<U, F: FnOnce(&mut T) -> &mut U>(Pin<&'a mut T>, f: F) -> Pin<&'a mut U>

  • I think the unsafe here is basically at least the same as above, we're
    handing out &mut T safely (can't require an unsafe closure) which could
    easily be used with mem::replace. There's probably other unsafety here too
    with the projection, but it seems reasonable that it's at least unsafe
    because of that.

• fn get_mut(Pin<&'a mut T>) -> &'a mut T where T: Unpin

  • By implementing Unpin a type says "Pin<&mut T> has no guarantees, it's
    just a newtype wrapper of &mut T". As a result, with no guarantees to
    uphold, we can return &mut T safely

• impl<P: Deref> Deref for Pin<P> { type Target = P::Target }

  • This can be safely implemented with as_ref followed by get_ref, so the
    safety of this impl follows from the above.

• impl<P: DerefMut> DerefMut for Pin<P> where T::Target: Unpin { }

  • This can be safely implemented with as_mut followed by get_mut, so the
    safety of this impl follows from the above.

• impl<T: ?Sized> Unpin for Box<T> (and other pointer-related implementations)

  • If no other action were taken, Box<T> would implement the Unpin trait
    only if T implemented Unpin. This implementation here codifies that even
    if T explicitly doesn't implement Unpin, Box<T> implements Unpin.

  Question: What's an example for what this is fundamentally empowering?
  
  For example, what safe could would otherwise require unsafe if this impl
  
  did not exist.

[Matthias247]

@Matthias247 you can't safely get &mut T from Pin> if T isn't Unpin because then you could mem::swap to move T out of the Pin, which would defeat the purpose of pinning things.

Thanks! Yes, since swap is no unsafe method this is obviously problematic. But could that be fixed by adding an Unpin bound to swap()? Since all code up to now should be Unpin or is unsafe anyway, this shouldn't break anything.

I guess one of the things that still confuses me most is that Pin<T> codifies multiple guarantees: That the address of the T is stable, as well as some guarantees about it's internal state (it's kind of frozen/immutable in some situations, but also not really).

It seemed to me like moving the unsafe code/projections only to the places where further Pin based calls are needed (e.g. in polls on fields) might be preferable to dealing with those unsafe projections in all cases. However I now also realized that there is another issue with that: Code that has access to the mutable reference can move the field freely around, and when drop() is then safely called on this field, it might break due to it's address being stored somewhere else. For that to the fixed the drop(Pin<T>) overload that was talked about would be required.

@RalfJung Thanks for the explanations! I agree on the fact that I certainly tried to do something unsafe in general, and therefore it should be fine to require the extra understanding from me. I'm more concerned about people who want to write more generally safe future combinators, but now also might get confronted with all of those terms. If they can write combinators without having to understand Unpin and pin projections at all, and then only get combinators which work in a reduced fashion (only on Unpin futures), that seems preferable. Since I haven't tried it I can't say if it's currently the case. I think one still needs to add at least Unpin bounds manually.

I also understand the fact that non-moveable types are different from the pin type. However I am currently more focused on use-cases than the difference. And for the use-case of intrusive collections non-moveable types work very well without introducing too much complexity. For futures obviously some research would be required, since they way from a moveable to a non-moveable type is missing. If that way would be not more ergonomic than the Pin APIs, there would also be no win.

[tikue]

Adding T: Unpin to mem::swap would mean it couldn't be used on certain types even when they're not inside a Pin.

[RalfJung]

But could that be fixed by adding an Unpin bound to swap()? Since all code up to now should be Unpin or is unsafe anyway, this shouldn't break anything.

That would break all generic code: If you write a function in today's stable rust for some T without constraints, you can call swap on it. This has to keep working.

non-moveable types work very well without introducing too much complexity.

Nobody has demonstrated a way to add non-movable types to Rust in a backwards-compatible way without complexity significantly exceeding what is being proposed for stabilization here. You should find some of these old proposals and discussions in the RFC repo. See https://github.com/rust-lang/rfcs/pull/1858 for one example.

The RFC at https://github.com/rust-lang/rfcs/pull/2349 as well as boat's blog series starting here should help to give you some background and some impression of the other designs that were considered. (Also notice the dates, this design has been in the works for almost 10 months!)

[Gankra]

Also mem::swap is a red herring, because it isn't at all an interesting function. It's literally just

let temp = *a; 
*a = *b; 
*b = temp;

[tikue]

@Gankro doesn't it use unsafe code? Afaik it's not possible to write that literally.

Edit: I guess another way to think about this is that adding T: Unpin to mem::swap is actually changing the definition of safety at the language level. It would break all the mycrate::swap fns in the wild.

[Matthias247]

Adding T: Unpin to mem::swap would mean it couldn't be used on certain types even when they're not inside a Pin.

If Unpin is automatically derived (in the same fashion as Sync/Send are) then I thought this shouldn't be an issue.

That would break all generic code: If you write a function in today's stable rust for some T without constraints, you can call swap on it. This has to keep working.

But this one obviously is. Didn't thought about the fact that trait bounds need to be explicitly propagated in Rust.

Nobody has demonstrated a way to add non-movable types to Rust in a backwards-compatible way without complexity significantly exceeding what is being proposed for stabilization here. You should find some of these old proposals and discussions in the RFC repo. See rust-lang/rfcs#1858 for one example.

Thanks, I will read up a bit more on the previous work on this if I find some time. Obviously lots of thoughts and effort already have went into this, and I certainly don't want to block things. I just wanted to provide my concerns and questions when trying to work with this.

[tikue]

@Matthias247

If Unpin is automatically derived (in the same fashion as Sync/Send are) then I thought this shouldn't be an issue.

I'm not sure if I was clear. To clarify, it's perfectly safe to move around a !Unpin type and thus perfectly safe to mem::swap them around. It's only unsafe to move a type T: !Unpin after it's pinned, i.e. inside of a Pin<P<T>> .

For example, in async/await code, you can move around futures returned from an async function as much as you want. You only stop being able to move them once you pin_mut! them or put them in a Pin<Box<..>>> or whatnot.

[Centril]

I have some assorted questions about this:

1. Given the importance of Pin<T> for async and for generators and potential for unsoundness when interacting with other parts of Rust (e.g. swap & replace), has any formal verification (e.g. by @jhjourdan or @RalfJung) been done of the variant of the pinning API that is proposed here?

2. Does this API give any new guarantees about the abstract machine / operational semantics of Rust? I.e., if we forget about the use case as a backing mechanism for async & await / generators, could we put this inside a crate in the ecosystem and it would just work given the current guarantees we give?

3. What sort of APIs or type system additions become impossible as a consequence of stabilizing the pinning API? (this question is an extension of 2.)

4. What avenues does the to-be-stabilized API provide in terms of evolving it a language provided &pin T type to improve upon field projection and such (which doesn't seem great with the proposed API).

I have some assorted notes:

1. The documentation in the standard library seems quite sparse. :(

2. I agree with others that the pinning construct is quite mentally taxing / complex.

3. The Unmovable example in the documentation seems overly complicated and involves unsafe; this seems sub-optimal. With gradual initialization as elaborated on in a draft RFC in the language (i.e. improving upon NLL) could instead give us:

struct Unmovable<'a> {
    data: String,
    slice: &'a str,
}

let um: Unmovable<'_>;
um.data = "hello".to_string();
um.slice = &um.data; // OK! we borrow self-referentially.

drop(um); // ERROR! `um.slice` is borrowing `um.data` so you cannot move `um`.

// You won't be able to take a &mut reference to `um` so no `swap` problems.

This involves zero unsafe and it's quite easy for the user to deal with this.

[pcpthm]

Additionally, the std APIs provide no safe way to pin objects to the stack.
This is because there is no way to implement that safely using a function API.

What about API like this?

pub fn using_pin<T, R, F>(value: T, f: F) -> R
where F: for<'a> FnOnce(Pin<&'a mut T>) -> R {
    pin_mut!(value);    // Actual implementation inlines this but the point is this API is safe as long as pin_mut! is safe.
    f(value)
}

[davidtwco]

I’ve not been following along with the development of the pinning APIs too closely, so this could have been mentioned or explained elsewhere and I’ve been unable to find it, apologies if that is the case:

The Pinned type is a ZST which does not implement Unpin; it allows you to
supress the auto-implementation of Unpin on stable, where !Unpin impls
would not be stable yet.

Is there an explanation anywhere about why !Unpin impls couldn’t be made stable?

[nrc]

It is only safe if Foo (the containing type) is not repr(packed),
because this causes fields to be moved around to realign them.

Does packed mean that fields can be moved dynamically? That is a bit scary. Are we sure that llvm will never generate code to move fields in other circumstances? Likewise is it possible that pinned values on the stack might be moved by llvm?

[nrc]

Despite the subtlety, this seems like a really nice API. Good work!

[withoutboats]

@Centril Ralf has written about pinning on his blog.

The pin APIs involved no language changes at all, and are entirely implemented in the standard library using pre-existing language features. It has no impact on Rust the language and doesn't foreclose on any other language feature.

Pin is really just a clever implementation of one of Rust's most valuable and classic features: the ability to introduce invariants to an API by marking part of the API unsafe. Pin wraps a pointer to make one operation (DerefMut) unsafe, requiring that people who do it uphold certain invariants (that they don't move out of the reference), and allowing other code to assume that this will never occur. A similar, much older example of this same trick is String, which makes it unsafe to put non-UTF8 bytes into a string, allowing other code to assume that all the data in the String is UTF8.

[withoutboats]

Is there an explanation anywhere about why !Unpin impls couldn’t be made stable?

Negative impls are currently unstable, totally unrelated to these APIs.

[davidtwco]

Negative impls are currently unstable.

🤦‍♂️ That makes sense, should have thought about that for a little bit longer. Thanks.

[RalfJung]

@alexcrichton That's some great analysis of this API, we should try to preserve it some place better than some comment that will get lost!

Some comments:

as_mut: Question: what about a "malicious" DerefMut impl? This is a safey way
to call a user-provided DerefMut which crates &mut P::Target natively,
presumably allowing it to modify it as well. How's this safe?

Basically, when you call new_unchecked, you make a promise about the Deref and DerefMut implementations for this type.

set: Given Pin

(and the fact that we don't know anything about
Unpin), shouldn't this guarantee that P::Target never moves? If we can
reinitialize it with another P::Target, though, shouldn't this be unsafe?
Or is this somehow related to destructors and all that?

This drops the old content of the pointer, and puts new content in there. "Pinned" doesn't mean "never dropped", it means "never moved until dropped". So Dropping it or overwriting it while calling drop is fine. Calling drop is crucial, that's the drop guarantee I mentioned above.

Where do you see something moving here?

map_unchecked: Question:: what's the counter-example here? If this were safe, what's the example that shows violating the guarantees of Pin?

An example would be starting with a Pin<&&T> and using this method to get a Pin<&T>. Pinning does not "propagate through" references.

get_ref: One "maybe gotcha" here is interior mutability, what if T were
RefCell?

Indeed, this is a gotcha, but as you observed not a soundness problem. What would be unsound is having a method that goes from Pin<RefCell<T>> to Pin<&[mut] T>. Basically, what happens is that RefCell does not propagate pinning, we could impl<T> Unpin for RefCell<T>.

[RalfJung]

has any formal verification (e.g. by @jhjourdan or @RalfJung) been done of the variant of the pinning API that is proposed here?

No, not from our side. The blog posts I mentioned above contain some thoughts for how I'd start formalizing this, but we haven't actually gone through and done it. If you give me a time machine or an interested PhD student we'll do it. ;)

if we forget about the use case as a backing mechanism for async & await / generators, could we put this inside a crate in the ecosystem and it would just work given the current guarantees we give?

That is the intention.

What sort of APIs or type system additions become impossible as a consequence of stabilizing the pinning API? (this question is an extension of 2.)

Uh, I have no idea how to answer that question with any confidence. The space of possible additions is just too large and also too high-dimensional that I'd dare making any kind of universal statement about it.

What avenues does the to-be-stabilized API provide in terms of evolving it a language provided &pin T type to improve upon field projection and such (which doesn't seem great with the proposed API).

I am not sure any more about &pin T, it doesn't match very well the new generic Pin<T>. In terms of handling projections, we'd need some hack to say "Unpin and Drop have not been implemented for this type", then we could do this safely with a macro. For additional ergonomics, we'd likely want a generic "field projection" capability in the language, one that would also cover going from &Cell<(A, B)> to &Cell<A>.

Is there an explanation anywhere about why !Unpin impls couldn’t be made stable?

AFAIK negative impls have some long-standing limitations, and if you add generic bounds to them they will often not work the way you think they should. (Generic bounds sometimes just get ignored, or so.)

Maybe Chalk fixes all of this, maybe just some of it, but either way we'd probably not want to block this on Chalk.

Does packed mean that fields can be moved dynamically? That is a bit scary.

It is the only sound way to call drop on a packed field, given that drop expects an aligned reference.

Are we sure that llvm will never generate code to move fields in other circumstances? Likewise is it possible that pinned values on the stack might be moved by llvm?

LLVM copying bytes around should be no problem, as it cannot change program behavior. This is about a higher-level concept of "moving" data, in a way that is observable in Rust (e.g. because pointers to the data no longer point to it). LLVM cannot just move data elsewhere that we may have pointers to. Likewise, LLVM cannot just move values on the stack that have their address taken.

[alexcrichton]

Ok thanks @RalfJung, makes sense! Some follow-up questions...

When you say "never moved until dropped", this means that Drop may be called where &mut self has moved from the address of Pin<&mut Self>? Destructors can't rely on interior pointers being accurate, right?

My worry when looking at set was how panics would be handled, but I think that this is a non-issue after digging into the codegen a bit more.

[Nemo157]

When you say "never moved until dropped", this means that Drop may be called where &mut self has moved from the address of Pin<&mut Self>? Destructors can't rely on interior pointers being accurate, right?

@alexcrichton my understanding was that means never moved until after Drop::drop returns. Otherwise some usecases (intrusive collections and stack allocated DMA buffers at least) become impossible.

[withoutboats]

Destructors can rely on something never having been moved if they can prove it was previously in a Pin. For example if a state machine can only enter a state through an API which requires it to be pinned, the destructor can assume that it was pinned before it was dropped if it is in that state.

Code cannot assume that nonlocal destructors do not move members, but obviously you can assume your own destructors don't move things around because you're the one writing them.

[RalfJung]

When you say "never moved until dropped", this means that Drop may be called where &mut self has moved from the address of Pin<&mut Self>? Destructors can't rely on interior pointers being accurate, right?

I mean that the data will never move (in the sense of not going anywhere else, including not being deallocated) until drop was called. And yes, drop can rely on running on the pinned location, even though its type cannot express that. drop should take Pin<&mut self> (for all types), but alas, too late for that.

After drop was called, the data is just meaningless bytes, you may do anything with them: Put new stuff in there, deallocate, whatever.

This allows, for example, an intrusive linked list where the destructor of an element deregisters it by adjusting the neighboring pointers. We know the memory will not go away without that destructor being called. (The element can still be leaked, and then it will just stay in that linked list forever. But in that case it will remain valid memory so there is no safety issue.)

[jonhoo]

I've been reading through all I can find on Pin, Unpin, and the discussion leading up to it all, and while I _think_ I now understand what's going on, there is also a lot of subtlety around what the different types mean, as well as the contract that implementors and users must follow. Sadly, I see relatively little discussion of that in the docs for std::pin or on Pin and Unpin specifically. In particular, I'd love to see some of the content from these comments:

• https://github.com/rust-lang/rust/issues/55766#issuecomment-436782229
• https://github.com/rust-lang/rust/issues/55766#issuecomment-437428085
• https://github.com/rust-lang/rust/issues/55766#issuecomment-437846602
• https://github.com/rust-lang/rust/issues/55766#issuecomment-437987966

incorporated into the docs. Specifically:

• That Pin only protects "one level deep".
• That Pin::new_unchecked places restrictions on the impl for Deref and DerefMut.
• The interaction between Pin and Drop.
• How !Unpin is only not allowed to move once placed in a Pin.
• The rationale for why Pin<Box<T>>: Unpin where T: !Unpin. This ties back to the "one level deep" restriction above, but I think this concrete example with a proper explanation would be helpful to readers.

I found @alexcrichton's counter-examples pretty helpful too. Giving the reader an idea of what can go wrong for the different unsafe methods beyond just prose I think would help a lot (at least it certainly did for me). In general, due to the subtlety of this API, I would like to see the Safety sections of the various unsafe methods expanded, and possibly also referenced from the std::pin module-level docs.

[Kimundi]

I haven't used this API much yet myself, so I'll trust the technical judgement that its in a good state now. However, I think the names Pin, Pinned and Unpin are too similar and inexpressive for what are very different types/traits and a relatively complex API, which makes understanding it harder (as evidenced by a few comments on this thread).

They seem to follow the naming conventions for marker traits, so I can't really complain, but I wonder if we could make a tradeoff between verbosity and self-explaining names:

• Pinned - kind of confusing because its the same word as Pin. Given that its used like PhantomData to augment a struct with meta information that would be missing otherwise, how about PinnedData, PhantomPinned, PhantomSelfRef or even DisableUnpin? Something that indicates usage pattern and/or the effect it is going to have.
• Unpin - Like in https://github.com/rust-lang/rust/issues/55766#issuecomment-437266922, I'm confused by the name Unpin, because "un-pin" can be understood in multiple ambiguous ways. Something like IgnorePin, PinNeutral maybe?

Generally I'm failing and finding good alternative names myself though...

[mark-i-m]

PhantomPin and PinNeutral strike me as particularly nice names.

[tikue]

Just to provide a counterpoint, I've found Unpin intuitive (once I understood it). Pinned is harder to keep straight, granted I haven't used it in my own code. What about changing Pinned to NotUnpin?

[jonhoo]

I agree that finding better names may be a worthwhile bikeshedding exercise for the purposes of making pinning easier to talk about. I propose:

• Pin -> Pinned: When you're given a Pin, that's really a promise that what you're given _has been pinned_, and will _remain_ pinned forever. I don't feel too strongly about this though, as you _could_ also talk about being given a "pin of a value". Pinned<Box<T>> just reads nicer to me, especially in that only the Box is pinned, not the stuff it contains.
• Unpin -> Repin: for other marker traits, we usually talk about what you can do with something that has that marker trait. Which is presumably why Unpin was chosen in the first place. However, I think what we really want the reader to take away here is that something that is Unpin can be pinned, and then re-pinned somewhere else, without consequence or consideration. I also like @mark-i-m's suggestion of PinNeutral, even though it's somewhat more verbose.
• Pinned -> PermanentPin: I don't think Pinned is a good name, because something that contains a Pinned isn't really pinned.. It just isn't Unpin. PhantomPin has a similar issue in that it refers to Pin, when Pin isn't really the thing you want to get at. NotUnpin has a double negative, which makes it hard to reason about. @Kimundi's sugestion of PhantomSelfRef gets pretty close, though I still think it's a little "complex", and it ties the property "cannot be moved once pinned" with one instance where that is the case (when you have a self reference). My suggestion could also be phrased as PermanentlyPinned; I don't know which form is less bad.

[tikue]

I think that Pinned should end up being NotX where X is whatever Unpin ends up being named. The only job of Pinned is to make it so the enclosing type does not implement Unpin. (Edit: and if we're changing Unpin to something else, presumably the double negative isn't an issue)

Repin doesn't make sense to me, because "This type can be pinned somewhere else" is just a side effect of "This type can be moved out of a pin".

[jonhoo]

@tikue In some sense I feel the same, but in reverse. I think Unpin should be phrased in the negative "this is _not_ constrained by Pin", whereas Pinned should be phrased in the positive "this _is_ constrained by Pin". Mostly just to avoid the double negative. But that's a detail; I do like the idea of there being a duality. Maybe: s/Unpin/TemporaryPin, s/Pinned/PermanentPin?

EDIT: Yeah, I see your point about Repin being a side-effect of Unpin. I wanted to communicate the fact that the Pin is "unimportant" for a type that is Unpin, which I don't think Unpin does super well. Hence the suggestion above of TemporaryPin.

[tikue]

@jonhoo I think the main reason I prefer the opposite is because Pinned prevents a trait from being implemented, so that is, in the plainest sense to me, the true negative.

Edit: what about:

Unpin -> Escape
Pinned -> NoEscape

[jonhoo]

Interesting.. I'm trying to see how it'd fit into the documentation. Something like:

In general, when you are given a Pin<P>, that comes along with a guarantee that P's target will not move until it is dropped. The exception to this is if P's target is Escape. Types marked as Escape promise that they remain valid even if they are moved (for example, they contain no internal self-references), and they are therefore allowed to "escape" a Pin. Escape is an auto-trait, so all types that consist entirely of types that are Escape are themselves also Escape. Implementors can opt out of this on nightly using impl !Escape for T {}, or by including the NoEscape marker from std::phantom.

That seems pretty decent, though the connection to the word "escape" seems a little tenuous. Separately, writing the above also made me realize that Pin<P> doesn't _really_ guarantee that P's target won't be moved (precisely because of Unpin). Instead, it guarantees that _either_ it doesn't matter if P's target moves _or_ P's target won't be moved. Don't know how to use that to inform a better choice of names though... But it is probably something that should make it into the docs one way or another.

[yasammez]

Personally, I too have a big dislike for Unpin as a name, maybe because it is most often seen as impl !Unpin which reads "not-un-pin" and requires several brain-cycles (I'm an older model) to conclude that it means "okay this one will be pinned forever once it is pinned the first time", so I can't even optimize away the double negative.
In general, humans tend to have a harder time thinking in negatives instead of positives (without a direct source, but check out the work of Richard Hudson if in doubt).
Btw Repin sounds really nice to me.

[tikue]

Pin is hard to explain because it doesn't always make the pinned value immovable. Pinned is confusing because it doesn't actually pin anything. It just prevents escaping a Pin.

Pin<P<T>> could be explained as a pinned value: pinned values can't move unless the value is a type that can escape a pin.

Some quick googling seems to show that in wrestling, when one is pinned, breaking out of a pin is called escaping.

[zroug]

I also like the term escape instead of Unpin but I would go for EscapePin.

[Matthias247]

Lots of good thoughts here!

One general thing: For me as a non-native speaker Pin and Unpin are mostly verbs/actions. While Pin makes sense, since the object is pinned to one memory location at a time, I can't see the same for Unpin. Once I receive a reference of Pin<&mut T>, T will always be pinned in the sense that it's memory location is stable, whether it's Unpin or not. It's not possible to really unpin an object as an action. The difference is that Unpin types do not require the guarantees of pinning to be uphold in further interactions. They are not self-referential, and their memory address is e.g. not sent to another object and stored there.

I would agree with @tikue that it would be nice to have a striking work for what Unpin actually means, but it's hard to quantify. It's not purely that those types are moveable, and it's also not purely their lack of self-referentiality? Maybe it's something about "no pointers in the whole memory space get invalidated when the object is moved". Then something like StableOnMoveAfterPin, or just StableMove might be an option, but those also sounds not really great.

Repin for me has the same complications as Unpin, which is that it implies that one thing first gets unpinned - which in my understanding does not happen.

Since the trait mostly defines what happens after one sees a Pin of the type, I find things like PinNeutral, or PinInvariant not too bad.

Regarding Pin vs Pinned, I think I would prefer Pinned, since that's the state of the pointer at the point of time when one sees it.

[jonhoo]

@Matthias247 I don't think that you are guaranteed that P::Target's address is stable if P: Unpin? I could be wrong about this though?

[tikue]

@Matthias247

Once I receive a reference of Pin<&mut T>, T will always be pinned in the sense that it's memory location is stable, whether it's Unpin or not.

Can you clarify what you mean by this? Given a T: Unpin you can write the following:

let pin_t: Pin<&mut T> = ...
let mut other_t: T = ...
mem::replace(Pin::get_mut(pin_t), &mut other_t);
// Now the value originally behind pin_t is in other_t

[Matthias247]

@jonhoo Actually a good question. My reasoning was that the futures get boxed, and then their poll() method will be called on the same memory address. But that obviously only applies to the top level task/future, and the intermediate layers might may move futures around when they are Unpin. So it seems like you are right.

[glaebhoerl]

W.r.t. the latest bikeshedding:

• Unpin: what about MoveFromPin? If I'm not still missing some subtleties, I think this directly states what capability the trait actually enables: if the type is within a Pin, you can still move it.

  Crucially it kind of says the same thing as Unpin, but framed as a positive assertion, so instead of the double-negative !Unpin we now have !MoveFromPin. I think I find that easier to interpret, at least... it's uh, types you can't move out of a pin.

  (There's some room for variation on the basic idea: MoveOutOfPin, MoveFromPinned, MoveWhenPinned, and so on.)

• Pinned: this can then become NoMoveFromPin, and its effect is to make a type !MoveFromPin. I think that seems straightforward enough.

• Pin itself: this one is unconnected to the other two and also not as significant, but I think there might be room for some slight improvement here as well.

  The issue is that Pin<&mut T> (for instance) doesn't mean that the &mut is pinned, it means that the T is (a confusion I think I've seen at least one recent comment evidence). Since the Pin part acts as a kind of modifier on the &mut, I think there's a case that it would be better to call it Pinning.

  There is some indirect precedent for this: if we want to modify the overflow semantics of an integer type to wrap around instead of panic, we say Wrapping<i32> rather than Wrap<i32>.

All of these are longer than the originals, but given how delicate some of the reasoning around these is, it might be a worthwhile investment for greater clarity.

[Nemo157]

Re: Repin, I would expect this to be something like

unsafe trait Repin {
    unsafe fn repin(from: *mut Self, to: *mut Self);
}

which could be used to support !Unpin types inside a Vec-like collection that occasionally moves its contents (this is not a proposal for adding such a trait now or ever, just my first impression from the trait name).

[Thomasdezeeuw]

Also bikshedding the names:

• Pin<P> -> Pinned<P>: the value the the pointer P points to is pinned in memory for it's lifetime (until dropped).
• Unpin -> Moveable: the value doesn't need to be pinned and can be freely moved around.
• Pinned (struct) -> Unmoveable: the needs to be Pinned and can't be moved around.

[withoutboats]

I don't think Pin or Unpin should change, the alternatives all add verbosity without clarity in my opinion, or are even quite misleading. Also we had this conversation already, reached a decision to use Pin and Unpin, and none of the arguments brought up in this thread are new.

However, Pinned was added since the previous discussion, and I think it makes sense to make it PhantomPinned to be clear its a phantom marker type like PhantomData.

Personally, I too have a big dislike for Unpin as a name, maybe because it is most often seen as impl !Unpin which reads "not-un-pin" and requires several brain-cycles (I'm an older model) to conclude that it means "okay this one will be pinned forever once it is pinned the first time", so I can't even optimize away the double negative.

This is completely the opposite of my experience, manually implementing !Unpin means you're implementing a self-referential structure by hand using unsafe code, an extremely niche use case. In contrast, anything which keeps a potentially unpin struct behind a pointer has a positive impl of Unpin. All of the impls of Unpin in std are positive polarity, for example.

[jonhoo]

@withoutboats could you provide a link to the previous discussion around these names where the arguments brought up here have already been argued through?

[withoutboats]

Here is one thread, though it was certainly also discussed on the RFC thread and tracking issue https://internals.rust-lang.org/t/naming-pin-anchor-move/6864

(the types in this thread called anchor and pin are now called Pin<Box<T>> and Pin<&'a mut T>)

[skippy10110]

Is Unpin supposed to be read as short for Unpinnable? Unpinnable as in you can't actually keep the value pinned, even though it's inside a Pin. (Is that even a correct understanding on my part?)

I've looked through some of the docs and comment threads and didn't see any reference to Unpinnable specifically.

[withoutboats]

Unpin is not supposed to be short for anything. One thing that I think is not obvious to a lot of users, but is true, is that the standard library style guide is to prefer verbs as trait names, not adjectives - hence Send, not Sendable. This hasn't been applied with perfect consistency, but it is the norm. Unpin is as in "to unpin," as in it is possible to unpin this type from the Pin you have pinned it with.

Names like Move (not "moveable," remember) are less clear than Unpin because they imply that it has to do with being able to move it at all, rather than connecting the behavior to the pin type. You can move !Unpin types, because you can move any Sized value in Rust.

Names that are whole phrases as I've seen suggested would be very unidiomatic for std.

[Nemo157]

It may not be short for it, but that's exactly how I read it. Since traits are verbs, you have to manually transform it to an adjective if you want to use it to describe a type rather than an operation on a type; std::iter::Iterator is implemented by something that is iterable, std::io::Seek is implemented by something that is seekable, std::pin::Unpin is implemented by something that is unpinnable.

[tikue]

@withoutboats any specific problem with some of the other names that don't have the verb issue, e.g. Escape or EscapePin? Understood that this discussion has happened before, but presumably a lot more eyes are on this now, so I'm not sure it's a completely redundant rehash...

[withoutboats]

One thing I think is true is that its unfortunate that Pin and Unpin could be read as a pair (some types are "pin" and some are "unpin"), when Pin is supposed to be a noun, not a verb. The fact that Pin isn't a trait hopefully clarifies things. The argument in favor of Pinning makes sense, but here we run up against the name length problem. Especially since method receivers will have to repeat self twice, we end up with a lot of characters: self: Pinning<&mut Self>. Not convinced that Pinning<P> is a whole four characters worth of clarity over Pin<P>.

@tikue the "escape" terminology is much more overloaded than pinning I think, conflicting with concepts like escape analysis.

[tikue]

Also we had this conversation already, reached a decision to use Pin and Unpin, and none of the arguments brought up in this thread are new.

This rubs me the wrong way -- is the community's experience report undesired? I personally didn't see a clear issue with Unpin until some of the other comments in this thread, as well as the juxtaposition with Pinned double negative.

[tikue]

Rust doesn't perform escape analysis, so I'm not sure I see that as a real issue.

[rfcbot]

:bell: This is now entering its final comment period, as per the review above. :bell:

[jonhoo]

I'd still like to see documentation improvements as outlined in https://github.com/rust-lang/rust/issues/55766#issuecomment-438316891 before this lands :)

[cramertj]

I've opened https://github.com/rust-lang/rust/pull/55992 to add the documentation suggested above and rename Pinned to PhantomPinned.

[tikue]

I think Pinned (and PhantomPinned) encourages conceptualizing a "pinned" value as one that cannot move out of a Pin, which means that many values in a Pin (those whose types impl Unpin) are not "pinned"!

That seems confusing. I find it easier to conceptualize all values in a Pin as pinned while they're in the Pin, and whether or not being pinned is permanent is what the thing formerly named Pinned controls. A name separate from Pin* would prevent conflation of two distinct concepts.

[cramertj]

PhantomNotUnpin :P

[RalfJung]

Personally, I too have a big dislike for Unpin as a name, maybe because it is most often seen as impl !Unpin which reads "not-un-pin" and requires several brain-cycles

Thanks! I have also been bothered by Unpin for quite some time bot not been able to pin-point (heh) why. Now I think I understand: It's the double-negation.

This is completely the opposite of my experience, manually implementing !Unpin means you're implementing a self-referential structure by hand using unsafe code, an extremely niche use case. In contrast, anything which keeps a potentially unpin struct behind a pointer has a positive impl of Unpin. All of the impls of Unpin in std are positive polarity, for example.

It's not just about implementation though, also about discussion. impl !Sync is fairly rare (not alone because it is unstable), but talking about Sync and !Sync types is quite common. Similarly, !Unpin came up quite a lot in discussions of this feature, at least the ones I've had.

I, too, would prefer something that positively expresses a property (MoveFromPin or so). I am not entirely convinced by ergonomics, since unlike Pin one shouldn't have to write this trait bound quite so often.

Rust doesn't perform escape analysis, so I'm not sure I see that as a real issue.

LLVM does, so escape analysis is still quite relevant to Rust.

[josephg]

The way to solve that is to pick words which invert the meaning of Pin / Unpin. Eg, rename Unpin to Relocate. Then !Unpin becomes !Relocate. That makes far more intuitive sense to me - I read it as "Oh, objects of this type can't be relocated". Another contender is Movable.

I'm not sure what the opposite word is that could replace Pin, or if we even need to. But I could certainly imagine the docs saying something like this:

A Pinned object can be modified directly via DerefMut if and only if the object can be relocated in memory. Relocate is an automatic trait - it is added by default. But if there will be direct pointers to the memory holding your values, opt out of Relocate by adding impl !Relocate on your type.

impl<T: Relocate> DerefMut for Pin<T> { ... }

This makes far more intuitive sense to me than Unpin.

[RalfJung]

I've opened #55992 to add the documentation suggested above

This only adds a part of what was suggested in https://github.com/rust-lang/rust/issues/55766#issuecomment-438316891, though.

[Matthias247]

I like the MoveFromPin suggestion. Relocate is also good, but maybe isn't associated with Pin enough. One could again understand it as a non-moveable type (which it isn't). RelocateFromPin would again be good.

Escapeing is a thing that is also e.g. associated in Swift with closures, and whether they are called inside or outside of the current call chain. That sounds misleading.

I don't see any issues with longer names as long as it helps clarity.

[alexcrichton]

FWIW I'd like to throw in a vote as well to rename Unpin to something "positive" like Relocate or MoveFromPin (or even more verbose but perhaps slightly more accurate MayMoveFromPin).

I agree that the double negative of !Unpin or just Unpin has been confusing to me historically, and something framed in a positive "this can move despite being inside Pin" I think would help alleviate some confusion!

[cramertj]

FWIW I initially thought the same thing about Unpin, but when I actually went to use it IMO it made sense-- it's not really a double negation, since the operation you're looking for is the ability to Unpin (take something in and out of Pin freely) not the ability to keep things in a pin. It's the same as MoveFromPin, just worded differently. I'd prefer a name that doesn't make people think it's "not Pin" or something, but IMO MoveFromPin and others are far too wordy. UndoPin? (FreePin for the haskell crowd?)

[runiq]

I still think !Unpin reads weird – I've trained my inner voice to treat it more like "Don't unpin this!" instead of the usual "Does not implement Unpin", but it's taken some effort.

What about !Pluck?

[cramertj]

@runiq

instead of the usual "Does not implement Unpin"

I mentioned this in my earlier comment, but I think this is actually a fine way to say it: Unpin is the operation of taking it in and out of Pin<C<_>>. Things that don't implement Unpin don't provide that ability.

[jonhoo]

@cramertj I quite like UndoPin

[alexcrichton]

@cramertj I do agree that I'm not a huge fan of alternatives proposed so far, but I would personally favor the wordy MoveFromPin over Unpin. It's a good point that it's not a double-negative, but on reading it (as someone who hasn't worked with it a ton yet) it keeps tripping me up that it is a double negative. I keep trying to read the "Un" prefix as negative...

I'm curious, but @cramertj or others do you feel like there's a good handle on how much the Unpin bound ergonomically comes up? Is it super rare? Is it super common? Common enough to be a pain if it's a pain to type?

For a short-and-sweet name I personally like the Relocate idea, but for longer-and-wordier-but-ok-because-you-don't-type-it-as-much I like MoveFromPin. I personally feel that regardless of the ergonomics of typing the name both are better than Unpin

[withoutboats]

I'm curious, but @cramertj or others do you feel like there's a good handle on how much the Unpin bound ergonomically comes up? Is it super rare? Is it super common? Common enough to be a pain if it's a pain to type?

In my experience, there are two cases where Unpin actually appears in user code:

1. Unpin bounds come up because you actually need to move a future around as you poll it (e.g. stuff like some select APIs).
2. More commonly, if your future is generic only over something other than a future, you usually want to add an unconditional implementation of Unpin, because it doesnt matter if those other types are Unpin since you never pin project to them.

An example of the second case is if you have some kind of buffer type youre generic over (e.g. T: AsRef<[u8]>). You dont need to pin it to get the slice out of it, so you dont care if it implements Unpin or not, so you just want to say your type unconditionally implements Unpin so you can implement Future ignoring pinning.

[cramertj]

Unpin is pretty common to see as a bound-- select!, StreamExt::next and other combinators all require the types that they operate on to be Unpin.

[jonhoo]

@withoutboats I'm curious about your point 2 there; do we think that impl Unpin is something that people are going to have to remember to implement often? Similar to how library authors today often forget to #[derive(Debug)] or impl std::error::Error for their custom types, which makes it harder to use those libraries?

[josephg]

Unpin is an auto trait. The only time you'll have a type which doesn't implement unpin is when that type explicitly opts out of it. (Or contains a field that opts out of unpin).

[jonhoo]

I understand that that is the case. And I assumed that that would by far be the most common case, which is why I was surprised that @withoutboats even mentioned the second point. It suggests to me that it may be more common than I originally thought (though perhaps only when implementing your own futures), so I'm curious about the frequency of those kinds of use-cases :)

[Pauan]

@alexcrichton I'm curious, but @cramertj or others do you feel like there's a good handle on how much the Unpin bound ergonomically comes up? Is it super rare? Is it super common? Common enough to be a pain if it's a pain to type?

I had written a long post on my experience of porting my code to Futures 0.3 (which uses Pin). You don't need to read it, the summary is:

Most of the time you don't need to worry about Unpin at all, because Unpin is auto-implemented for almost all types.

So the only time you need to worry about Unpin is:

1. You have a type which is generic over other types (e.g. struct Foo<A>).

2. And you want to implement a pinning API (e.g. Future/Stream/Signal) for that type.

In that case you need to use this:

impl<A> Unpin for Foo<A> where A: Unpin {}

Or this:

impl<A> Unpin for Foo<A> {}

impl<A> Future for Foo<A> where A: Unpin { ... }

That's usually the only situation where Unpin is needed. As you can see, that usually means needing to use Unpin ~2 times per type.

In the case of non-combinators, or in the case of types which don't implement Future/Stream/Signal, you don't need to use Unpin at all.

So I would say that Unpin comes up very rarely, and it only really comes up in the situation of creating Future/Stream/Signal combinators.

So I am strongly in favor of a name like MoveFromPin. Pinning is a niche feature that most people never need to deal with, so we shouldn't be over-optimizing for name length.

I think the cognitive benefits (avoiding double negation) are much more important than saving a few characters in rare situations.

Especially because pinning is already hard enough to understand! So let's not make it unnecessarily harder.

@jonhoo do we think that impl Unpin is something that people are going to have to remember to implement often? Similar to how library authors today often forget to #[derive(Debug)] or impl std::error::Error for their custom types, which makes it harder to use those libraries?

I don't think it's possible to forget to impl Unpin, because if the author does forget, they'll get a compiler error, which prevents their crate from being published in the first place. So it isn't like #[derive(Debug)] at all.

The situation @withoutboats is talking about is only for people implementing Future/Stream/Signal combinators, it doesn't affect anybody else (in particular, it doesn't affect downstream users of the library).

[glaebhoerl]

(I think double-negation is a part of it, and maybe simply more-negation-than-strictly-necessary also is, but I feel it's not the whole story (trying to introspect here)... "Unpin" is a bit, metaphorical? Indirect? It makes sense when explained, and since "pinning" itself is already a metaphor it's not clear why my brain would have a problem with a taking-further of that metaphor, but, nonetheless, my brain finds the meaning of "unpin" for some reason obscure and hard to latch onto firmly.)

[RalfJung]

I guess you are right @cramertj, it is not actually double negation in the usual sense, but like @alexcrichton and @glaebhoerl I keep getting tripped by it. "Un-" as a prefix has a very negation-y feeling to it ("unsafe", "unimplemented" and so on is how I usually encounter that prefix), and it is negating pinning here, if only as a verb.

[withoutboats]

@withoutboats I'm curious about your point 2 there; do we think that impl Unpin is something that people are going to have to remember to implement often? Similar to how library authors today often forget to #[derive(Debug)] or impl std::error::Error for their custom types, which makes it harder to use those libraries?

Absolutely not! If a user is manually implementing a Future which is generic over a non-future, they probably want to be able to mutate state in their future implementation without writing unsafe code - that is, to treat Pin<&mut Self> as &mut self. They will get errors indicating that MyFuture<T: AsRef<[u8]>> does not implement Unpin. The best solution to this problem is then to implement Unpin. But the only impact this has is on the user trying to implement Future, and its impossible for them to forget because their code will not compile.

The situation @withoutboats is talking about is only for people implementing Future/Stream/Signal combinators, it doesn't affect anybody else (in particular, it doesn't affect downstream users of the library).

I'm specifically talking about non-combinator generic manual futures, which should just have blanket impls of Unpin even if their generics are not Unpin.

[withoutboats]

I made a flowchart for the question "What do I do about pinning when I am implementing a manual future/stream?"

[yasammez]

To bikeshed a bit more, today at lunch I came up with LeavePin. It carries the same tone as escape without the implied misleading semantics.

[vi]

Are there any interesting interactions between impl specialistion and pins, as is with lifetimes?

The substring "specializ" occurs in tracking issue discussion a bit, but it's not conclusive. Pin RFC or specialisation RFC should explicitly mention this interaction, either as "It's verified to be OK" or "further research is needed to judge it safe".

[withoutboats]

@vi not only is there no soundness interaction, its not possible for there to be a soundness interaction. The pin APIs are strictly defined in the standard library and involve no new language features, a user can define them just as easily in a third party libraries. If a language feature is unsound in the face of this library code existing, it is unsound period, because any user could write this library code today and it will compile fine.

[Centril]

If a language feature is unsound in the face of this library code existing, it is unsound period, because any user could write this library code today and it will compile fine.

Not that it should have any bearing on pin... but I don't think it's nearly as simple as that.

If a library feature, when using unsafe, is making use language constructs whose behavior are as of yet unspecified (e.g. &packed.field as *const _, or making various assumptions about ABI) then if additional language changes invalidate the assumptions of those libraries then I think that it is the libraries which are unsound and not the language changes. On the other hand, if language changes make defined behavior unsound then it's the fault of the language changes. Compiling fine is thus not a sufficient condition for the soundness of a library in the face of unsafe and language changes.

[ejmahler]

+1 to MoveFromPin or similar

If you ask the question "When should I unimplement Unpin for my own type?", the answer is much more clear if you instead ask "When should I unimplement MoveFromPin for my own type?"

Same with "Should I add Unpin as a trait bound here?" vs "should I add MoveFromPin as a trait bound here?"

[glandium]

Pinning is not !Move

Sorry if this was mentioned somewhere, but I only skimmed the vast amount of discussion there has been around Pin here, in the implementation issue, and in the RFC issue.

Will rust ever have !Move? I can certainly see use cases for such a thing (heck, I came looking about Pin because I was looking for a way not to shoot myself in the foot with types that can't be moved). If the answer is yes, how would that interact with Pin? Would the existence of Pin make it harder than it already is to add !Move?

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

[tikue]

There should be an unresolved concern around the naming of Unpin.

[jonhoo]

As @RalfJung pointed out, #55992 also only adds a small amount of the extra documentation asked for in https://github.com/rust-lang/rust/issues/55766#issuecomment-438316891 and elsewhere. Don't know that that's grounds for not merging yet though.

[wmanley]

Why does my drop() method now again take &mut self, instead of a Pin.

Well, drop() is old -- it exists since Rust 1.0 -- so we cannot change it. We'd love to just make it take Pin<&mut Self>, and then Unpin types could get their &mut like they do now, but that's a non-backwards-compatible change.

I was wondering if it would be possible to implement this change in a backward compatible way. AIUI until we add Unpin (and people can specify !Unpin) all types implement Unpin. So we could add a trait:

trait DropPinned {
    fn drop(Pin<&mut> self);
}

and then impl this trait for all Unpin types, which until people can opt out is all types. Something like:

impl<T> PinDrop for T where T:Unpin + Drop {
    fn drop(Pin<&mut T> self) {
        Drop::drop(self.get_mut());
    }
}

Then we'd have the compiler insert calls to DropPinned::drop instead of Drop::drop. Essentially trait DropPinned becomes the lang-item rather than Drop. AFAICS this would be backward compatible if and only if this mechanism was introduced at the same time as Unpin.

[cramertj]

There should be an unresolved concern around the naming of Unpin.

@tikue No libs team member filed a concern with rfcbot before or during the FCP, and I don't believe any of the arguments about Unpin were new or novel to this thread, so typically our process would consider the current naming to be finalized. Obviously if someone has concerns they should speak up, but the point of having team checkboxes followed by FCP is to ensure that the everyone is comfortable stabilizing the API as-proposed.

[jonhoo]

@cramertj I'm a little confused. Several people have spoken up. When I asked for references about where else the arguments about the naming of Unpin had been raised and resolved, I was pointed to https://internals.rust-lang.org/t/naming-pin-anchor-move/6864, which as far as I can see also has people complaining about the naming of Unpin, and no real counter-argument. The original RFC in https://github.com/rust-lang/rfcs/pull/2349 also does not have much rationale as to why proposed alternatives to Unpin are worse. Even in this thread, it seems like the only counter-arguments that really come up are "it's shorter" and "it's technically correct". Are you able to point to concrete discussion where alternative names that are easier to understand (such as MoveFromPin) are discussed and rejected?

[tikue]

I have explained in previous comments why I believe novel perspectives have been brought up in this thread. I've been following the pin API discussions fairly closely since inception and don't remember ever seeing the double negative issue brought up before this thread.

[cramertj]

@tikue I've brought up and seen the double-negative issue brought up multiple times, and the precise naming of Unpin has been raised as an issue many times over and has consistently been resolved in favor of Unpin. As I said before, if someone on the libs team wants to register a (late) objection then I'm fine responding to that, but they all signed off on the stabilization proposal above which clearly does not include the naming of Unpin as an unresolved question: we've discussed the alternatives, and this FCP was the process for deciding that we were prepared to stabilize on the decisions that had been made, including the name Unpin.

[jonhoo]

@cramertj Could you please provide a link to where that discussion has happened? I'm not doubting you, would just like to see the arguments in favor of Unpin, because I do not believe they have been given here. As mentioned, the references I've been given so far do not provide any resolution on the naming of Unpin.

[tikue]

@cramertj +1 to @jonhoo's ask. If there are discussions that the libs team had among themselves that are not registered in official channels, I think the main thrust of those discussions ought to be reiterated here. I think there is even an official rule that RFC decisions can only be made based on publicly-known arguments?

[cramertj]

I think there is even an official rule that RFC decisions can only be made based on publicly-known arguments?

Yup, that's true-- and for the record, I'm not on the libs team and so haven't been present for any libs-team only discussions. Looking through the RFC thread and the Pin tracking issue, there were numerous times where the name of Unpin was brought up. I don't see anyone specifically say the words "double negative", but I certainly remember "!Unpin is a double negative" being brought up before, in addition to the general API rule of naming traits for what you can do with them, rather than what you can't (as I pointed out above, I think Unpin actually follows both these rules, though realizing that requires hearing "Unpin" as a verb rather than hearing it as an adjective "not-pin", which isn't intuitive to folks).

[RalfJung]

@wmanley That does not work, e.g. impl<T> Drop for Vec<T> would break because we do not have Vec<T>: Unpin. Also, proposals along this line have been made before, even in this thread. Please read the discussions before replying. I know that's asking quite a lot, but it is the only way to avoid the same issues being explained again and again.

[steveklabnik]

I think there is even an official rule that RFC decisions can only be made based on publicly-known arguments?

This is informally known as the "no new rationale" rule.

[DutchGhost]

Not sure where to post this, but could someone take a look at https://github.com/rust-lang/rust/issues/56256 ?

[Nemo157]

Related to #56256, impl<T> From<Box<T>> for Pin<Box<T>> is not listed in the OP as being stabilized, but it will implicitly become stable once Pin does. Are there any other trait implementations that are non-trivial and should be looked at for stabilization? (scanning the docs all the others appear to be trivial delegating implementations for the wrapped pointer to me).

[Kimundi]

We talked about this Issue in the libs team today. The last weeks of discussion have shown that there are still a few things that should get addressed first, especially regarding the naming of Unpin.

Therefore we will not move forward with stabilizing this for now (FCP notwithstanding).

It would be much appreciated if someone could gather up the ideas in this thread and prepare a standalone proposal for improving the naming situation.

[cramertj]

@Kimundi

It would be much appreciated if someone could gather up the ideas in this thread and prepare a standalone proposal for improving the naming situation.

Does this mean that the libs team is unwilling to stabilize the current APIs as-is? I personally don't think that anyone has come up with a better name than the current set of names as-implemented, and so I'm not able to make such a proposal, but I do care a lot about seeing these APIs stabilized, so if someone from the libs team has a name they'd prefer then :shipit:

[cramertj]

Bikeshedded this with a bunch of coworkers and @anp suggested DePin, which I actually like quite a bit, since it removes the "not pin" connotation of Unpin and emphasizes that it's talking about a type which can be de-Pin'd.

[aturon]

@Kimundi can you or someone from the libs team please register an explicit "fcp concern" to take this out of FCP and lay out more clearly what is meant by "a few things taht should get addressed"?

[alexcrichton]

@rfcbot concern naming-of-Unpin

I'm not sure if this works really once FCP is entered, but I'd like to place a formal blocking concern on the naming of the Unpin trait. The Unpin trait looks to be pretty crucial to the API, and the "double negative" (as discussed above) throws me for a spin every time I read it.

There's been a whole bunch of comments about various names, but unfortunately I'm not overly thrilled with any yet. My "favorite" is still along the lines of MoveFromPin or Relocate (my god there are so many comments here I have no idea how to review this).

I'm personally ok with the naming of Pin itself as well as Pinned for a ZST that doesn't implement the Unpin trait.

[jonhoo]

I completely agree with @alexcrichton that the naming of Unpin is the big point of contention here. I don't think there are any technical concerns as far as I can see about the proposed feature itself (though there _are_ a lot of comments, so could have missed something).

I do still think Pinned is a weird name for the ZST, because something that contains a Pinned isn't really pinned.. It just isn't Unpin. PhantomPinned (as it's been renamed to in #55992) has the same issue in that it refers to Pin, when the ZST is _really_ about Unpin.

I also still think the docs need more work given the subtlety of this feature, but that probably doesn't count as a blocker.

Also, @Kimundi, I'm happy to see that the libs team is willing to give this a little more time to settle. It may seem a lot like unnecessary bikeshedding, but I (and others too it seems) think it is pretty important to improve the teachability of this feature :)

[tikue]

Agreed with @jonhoo about Pinned still feeling weird, and PhantomPinned not being any better (it's not Pinned in any way, not even in a phantom way). I think if we find a good name for Unpin, then Pinned will naturally lend itself to being renamed Not{NewNameForUnpin}.

[cramertj]

I really don't think we need to spend any more time discussing PhantomPinned-- this type will almost never appear to end users. PhantomNotUnpin/PhantomNotDePin/PhantomNotMoveFromPin / etc. aren't going to make it any more or less common or make the API more or less obvious to users who are already comfortable enough with the API to have come up with a legitimate use for PhantomPinned.

[stjepang]

Just a quick idea: trait Move and ZST Anchor.

Every type is Moveable, unless it contains an Anchor that makes it stick to a Pin.
I like how Anchor: !Move intuitively makes a lot of sense.

[tikue]

I'm not suggesting we spend time specifically on PhantomPinned, but I do think it'd be low effort to keep an open mind, because it's possible that whatever is landed on for Unpin will work just as well for PhantomPinned.

[cramertj]

We've covered Move and explained why it's inappropriate numerous times. All types are movable until they are pinned. Similarly, Anchor has previously been suggested but it isn't clear at all from the name that it is used to opt out of Unpining/Moveing.

[jonhoo]

@stjepang I think Move has been discarded a while ago because something being !Unpin does not actually prevent it from being moved. It's only if a type is under a Pin, and it isn't Unpin, that you are "contractually obliged" not to move it.

[stjepang]

In the original comment @withoutboats said:

The Pin wrapper modifies the pointer to "pin" the memory it refers to in place

I think it's odd that types implement Unpin because values are not "pinned" - memory is. However, it makes sense to talk about values that are "safe to move". What if we rename Unpin to MoveSafe?

Consider the UnwindSafe trait. It's just a "hinting" marker trait and is safe to implement. If you let a !UnwindSafe value cross a catch_unwind boundary (with AssertUnwindSafe), you'll "break" it by invalidating its invariants.

Similarly, if you have a !Unpin/!MoveSafe value, it can still be moved (of course), but you'll "break" it by invalidating its self-references. The concept seems similar.

The trait Unpin really just means MoveSafe. It seems to me it's not about values that can be moved out of memory behind Pin. Rather, it's about values that won't "break" when you move them.

[cramertj]

MoveSafe has the same problem as Move-- all values any type can be safely moved. It only becomes impossible to move a value after it has been pinned.

[stjepang]

MoveSafe has the same problem as Move-- all values any type can be safely moved.

Right, but it's a different meaning of "safe", just like in UnwindSafe. Anyways, I'd be fine with Relocate or anything of that sort.

To summarize, I feel that the trait name should not be DePin, Unpin, or anything with "pin" in its name. To me, that's the main source of confusion. The trait is not really an "escape hatch" from the shackles of Pin - the trait says the value won't be invalidated when moved.

I just see Pin and Unpin as totally separate things. :)

[cramertj]

I feel the exact opposite ;). The trait is only meaningful with respect to Pin, which is the only type we have which meaningfully expresses constraints on the movability of the underlying value. Without Pin, Unpin is entirely meaningless.

[DutchGhost]

I like to think about pinning as putting something on a pinboard. If you remove the pin of an object that is pinned to the board, you can move the object. Removing the pin is Unpinning.
I like the name Unpin.

I can also see how !Unpin is a double negation, and can cause confusion. However, I wonder how often you need to write !Unpin.

One other name I could come up with for Unpin is Detach. Back to the pinboard metaphor, you wouldn't _Unpin_, but _Detach_ the Pin from its object.

[RalfJung]

I think I really like DePin! So far it's my favorite -- it's concise, it's clearly a verb and not an adjective, and !DePin seems pretty clear as well ("cannot be de-pinned").

I think it's odd that types implement Unpin because values are not "pinned" - memory is.

The value is pinned to memory. But the value is crucially important here as well. Pinning memory, for me, is just about making sure that it remains dereferencable or so, but it is not violated by mem::swap. Pinning a value to memory is about not moving that pinned value anywhere else, which is exactly what Pin is about.

[josephg]

I can also see how !Unpin is a double negation, and can cause confusion. However, I wonder how often you need to write !Unpin.

I hear you when you say you don't find the name confusing. I, and many many others in this thread have found ourselves confused by the naming.

I don't have the code at hand, but the first time I tried to use futures I wanted a function to return an impl Future<Output=T>. I can't remember what happened, but I immediately got a gnarly compiler error complaining about T and Unpin. The question I needed an answer for was "Is it safe to constrain T to only be Unpin". And that led me to stare into the abyss for about 2 harrowing hours.

"Oh, ok so if thats what Pin means. So Unpin means .. Box? Why is that a trait?"

"Wait, impl !Unpin? Why is that not impl Pin?"

"Right, Pin and Unpin ... aren't opposites. They're totally different things. Wait, then what does Unpin mean again? Why is it called that?"

"What on earth does !Unpin mean? Not ... the other, not pin thing? Hrrrgh"

Still the only way this makes sense in my head is by replacing 'unpin' with 'relocatable'. "Type is not relocatable in memory" makes complete sense. And yet, even knowing what Pin does, "type is not unpin" leaves me confused.

Renaming Unpin to Relocatable (or Relocate) gets my vote 🗳. But I'd find just about any of the other suggestions better than Unpin.

[withoutboats]

The trait is only meaningful with respect to Pin, which is the only type we have which meaningfully expresses constraints on the movability of the underlying value. Without Pin, Unpin is entirely meaningless.

To put a point on this - the guarantees around pin are entirely around certain behaviors, not types. For example, a generator function which yields () could trivially implement FnOnce by resuming repeatedly. While this type might not implement Unpin - because its yield states could be self-referential, its FnOnce interface (which moves itself) is completely safe because it hasnt yet been pinned when you FnOnce it, which is necessary to bring it into a self-referential state. Unpin is specifically about the kinds of behaviors that are asserted to be safe once the type has been pinned (namely, moving it), not about some intrinsic property of the type.

[glaebhoerl]

Ironically enough I just came here to comment that while the Unpin naming definitely has been debated in the past, the debate over it I remember witnessing was when we chose to replace Move with Unpin, which, I was going to say, is an unambiguous improvement. And often one only realizes later on that, while the current design is a definite improvement over what came before, it still has some room left for yet further improvement. Which is the direction I was coming at this from.

[RalfJung]

Unpin is specifically about the kinds of behaviors that are asserted to be safe once the type has been pinned (namely, moving it), not about some intrinsic property of the type.

The behavior when pinned is an intrinsic property of the type, just like the behavior/invariant when it is shared. I went into this in a lot more detail in this blog post.

[withoutboats]

@RalfJung we're speaking past each other. There's a confusion I see a lot that self-referential types "cannot be moved" - this is not accurate, they have certain states they can enter into during which they cannot be moved, but while they are in other states, it is perfectly safe to move them (and our APIs rely on the ability to move them, for example to apply combinators to them or to move them into a Pin<Box<>>). I'm trying to clarify that its not the case that these types "cannot be moved."

[RalfJung]

Ah, yes, then I agree. A !DePin type is not always pinned, and when it is not, it can be moved like any other type.

[alexcrichton]

@cramertj @withoutboats one thing I haven't quite been able to surise yet, are y'all against renaming Unpin? It doesn't sound like y'all feel you agree it needs to be renamed, but I'm not sure if you're against it.

[Centril]

I personally don't think the main problem here lies with the name Unpin and that "if we could just rename it everything would be intuitive". While renaming may help a little bit (and Relocate / DePin seem nice here...), I think the main complexity comes from the concept of pinning itself; it's certainly not one of the easiest concepts to understand or explain; not even nearly.

Therefore, I think the documentation of the core::pin module needs to be beefed up _significantly_ and a lot more examples need to be included. Good examples would both show canonical use cases as well as uses of the unsafe functions and implementations which are unsound.

[cramertj]

@alexcrichton I'm not against renaming, no. I think Unpin is already fine, but I'd be okay with DePin, which is why I suggested it. RemoveFromPin is the most obviously "correct", but verbose to the point of syntactic salt, so I think I'd oppose that name specifically. I am, however, opposed to putting off stabilization indefinitely until we find a name that everyone agrees is the best-- I think the API has some inherent complexities that won't be made dramatically better or worse because of the name of the Unpin trait. I'd really like to push towards stabilization soon in order to prevent more churn to the code, docs, and discussions around futures_api (and so we can start getting the pieces in place to stabilize futures_api itself). Perhaps we should schedule a dedicated name-settling VC meeting so that everyone who has an opinion can pitch their solution and we can have a more high-bandwidth opportunity to settle this?

[liigo]

My vote is std::pin::Reloc (Relocate)

[Darsstar]

I have two very hypothetical situations (well, really just one, but two different executions) of which I wouldn't mind having explicitly stated wether it is allowed or not.

Unpin indicates that it is trivial to transition from all of T's (where T: Unpin) states while in the pinned type-state (I hope I am correctly remembering that term from one of @RalfJung's blog posts) to the not-pinned type-state. Which is why Pin can hand out &mut T.

So let's assume I want to create a type Woof for which it is also always possible to transition from all of Woof's states while in the pinned type-state to the not-pinned type-state, but it isn't trivial to do so and can therefor not implement Unpin, would it be allowed for Woof to have a function like fn unpin(self: Pin<Box<Woof>>) -> Box<Woof>?

Same for a type Meow for which it is only sometimes possible to transition from pinned to not-pinned and a function like fn unpin(self: Pin<Box<Meow>>) -> Result<Box<Meow>, Pin<Box<Meow>>>.

[Keruspe]

My 2 cts for the bikeshedding:

If I understand correctly, what Unpin realy means is "Pin has no effect on me".

What about something like BypassPin or IgnorePin?

[RalfJung]

So let's assume I want to create a type Woof for which it is also always possible to transition from all of Woof's states while in the pinned type-state to the not-pinned type-state, but it isn't trivial to do so and can therefor not implement Unpin, would it be allowed for Woof to have a function like fn unpin(self: Pin>) -> Box?

Yes, that should be possible. For example, if Woof is an element of an intrusive linked list, it could provide a function to remove the element from the list, and remove the Pin at the same time (arguing that for non-enqueued Woofs, the two typestates are equivalent, so we can de-pin it).

[Nemo157]

Relocate has the same issue as RePin to me, it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely. It’s a less strong connotation than RePin has, but still seems a little confusing.

[RalfJung]

it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely.

While that's not exactly what this trait is about, it is not entirely wrong either: For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely. Actually, given that anyone can create a Pin<Box<T>>, I don't even see why you are making a fundamental distinction here.

[Nemo157]

For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely.

Yes, which is why having a trait adding that operation to a type could be useful (this could not be a marker trait like Unpin as it may need to do things like update internal references). I’m not proposing we add something like this now, just that it’s something I could see being provided in the future (either in std or third party) that could end up with a confusing overlap of names.

I don’t know of any strong use cases for it now, but I have thought about using something like it with pinned collections.

[alexcrichton]

Ok so here's some thinking. I was initially wondering if we could have a name like PinInert or PinNoGuarantees as that's also what the description is, but thinking about that what I really want is a trait describing an action as opposed to a property, or at least it makes it much easier to think about in my head.

One problem with Unpin (I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do". The trait as-is conveys action "the act of unpinning" but I can't seem to quite understand that when I read it. It feels weird to have Pin<T: Unpin> because if it's "unpin" why is it in a Pin?

I wonder if a name like CanUnpin could work? A lot of this isn't about a hard guarantee one way or another (implementing Unpin doesn't mean that you will remove it from a pin, it just means you can remove it from a pin). How does that sound? Is CanUnpin readable enough to others? Short enough?

(having the prefix of Can conveys the verb to me much more easily as well, and it says you can remove it from a pin but you're not necessarily always going to).

---

As another unrelated tangent, one thing I forgot to bring up earlier (sorry about that!) is that I don't think that all of the methods above should necessarily be inherent methods. We've already had loads and loads of bugs around inference and clashing methods, and adding very common names like as_ref and as_mut on types which also implement Deref seems like a problem waiting to happen.

Do these operations happen commonly enough to justify the risky location to put them? (inherently) Or are they used rarely enough that the safe route of associated functions can be stomached?

[anp]

Since I apparently have skin in this game now: CanUnpin seems very close in spirit to me to Unpinnable or something similar, and my impression has always been that the community frowns upon those kinds of modifiers in trait names, since most traits describe the action that the type can take. The impl itself is an implied "can" or "-able" in that sense. Whatever is decided (and the name doesn't matter to the extent stated here IM-not-so-HO -- very few users will likely have to type this), I would encourage everyone to feel some urgency about resolving the questions here. I'm excited for this stabilization, and I know that many many people are!

[stjepang]

@alexcrichton

One problem with Unpin (I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do".

What if you think of T: Unpin as "T is immune to Pin"? Then if you have Pin<T: Unpin> it just means we've got a value inside a Pin which is immune to pinning so pinning is here effectively moot.

Or in other words: Unpin neutralizes Pin. To be honest, after so much discussion I've kind of internalized this and now it makes sense. 😆

A lot of this isn't about a hard guarantee one way or another (implementing Unpin doesn't mean that you will remove it from a pin, it just means you _can_ remove it from a pin).

A counterpoint to this is the Send trait. It doesn't mean that you will send the value, it just means you can send it.

[alexcrichton]

@anp I agree that CanUnpin isn't a great name, I'm attempting to do my damndest to figure out a better name! It seems few still feel this needs to be renamed though as all suggestions seem to get shot down for basically the same reasons I feel like Unpin needs renaming in the first place.

Also as a reminder any stabilization rides the trains like all other changes, and with a release next week this definitely won't make it into that release and will next be a candidate for the next release. That means we have 7 weeks, nearly two months, to land all this to ensure it gets in ASAP. While I agree urgency is necessary it's just "let's not forget about this urgency", not "let's resolve this before Monday" urgency.

@stjepang I, too, have grown used to Unpin now after thinking about it so much! All alternative names at this point seem quite lackluster, so I'm getting to the point where I would settle for better documentation. I would ideally like to find someone who doesn't know much about the Pin apis to read said documentation afterwards, though, to double check it's sufficient for learning.

[alexcrichton]

I'd also like to go ahead and formally block stabilization on improved documentation, as it feels especially critical for this issue.

@rfcbot concern improving-the-documentation

Concretely what I feel needs better documentation is:

• More prose on each method about why it is safe and/or why it is unsafe. For example the contract of P's DerefMut implementation "behaving reasonably" isn't documented on Pin::new_unchecked.
• Each unsafe function should have a code example of why it is unsafe, or at least a clear explanation of a sequence of steps that can go wrong.
• I feel the module docs can go into more detail beyond simply what a Pin is and what it means. I think they'd benefit from information like "how this isn't general immovable types but a form of that" or "how is Pin used in practice, for example with futures".
• I'd like to see an example with generics and Unpin in either the documentation or code examples. For example it sounds like many combinators in futures have to handle this, and similarly some combinators specifically require this. Some example use cases of how one works with generics and how they play out might help understanding Unpin quite a bit.

I'm also curious if others have concrete actionable items to add to the documentation too!

[alexcrichton]

Next I'd also like to formally block on the self-question with as_ref, but I suspect this will be quick to resolve. (again sorry for not remembering to bring this up sooner)

@rfcbot concern self-methods

This proposes as_ref, as_mut, get_ref, get_mut, into_ref, and set to all be methods on the Pin type. The proposal mentions that we don't do this for smart pointers because of method clashes, but cites that experience shows today's implemented Pin API isn't consistent and it ends up often just getting in the way.

These method names, however, are very short and sweet names and are quite common to find throughout the ecosystem. The libs team has run into a never ending series of bugs in the past where we add trait implementations and such and it causes clashes with existing trait methods in various crates. These methods seems especially high risk because of their names. Additionally, it's unclear whether we could ever add more methods to Pin in the future as even if the names now end up working out any future name added has a high risk of collision.

I'd like to just be sure that we rethink this divergence from convention, I'm personally particularly worried about the consequences and I think it'd be great if we could find a middle-ground which would avoid these hazards.

[alexcrichton]

And while I'm at it, one last thing I've noticed, and again I think this'll be pretty quick to resolve

@rfcbot concern box-pinnned-vs-box-pin

Has the name Box::pin been considered for Box::pinned? With the existence of Pinned and/or PhantomPinned it seems like it'd be neat if the name could match the return type!

[glaebhoerl]

@alexcrichton Has anything in particular persuaded you against MoveFromPin as an option? I see you were favorable towards it earlier (and multiple people seemed to like it as well). For the record, the direct objections I could remember, or quickly find by Ctrl+F-ing, are that "names that are whole phrases ... would be very unidiomatic" (@withoutboats) and that it's "far too wordy" (@cramertj).

I have to admit that motivations in the vein of "I have grown used to ... now after thinking about it so much" make me rather uneasy. Humans can get used to and post-hoc-rationalize essentially anything. No matter what the name is, it would be a major surprise if we didn't eventually end up getting used to it. (This is the very reason why we choose obviously-suboptimal names like existential type as temporaries in other cases, to try to avoid them becoming permanent.)

Accidentally privileging the perspectives of experienced users (which we all are!) over others who will be coming at things with fresher brains is the kind of mistake I think we need to be ever-vigilant against making, as hard as it is.

[Kimundi]

While it certainly isn't the style of the std lib, I also feel like CanUnpin somehow makes it way more clear how that particular piece fits together with the other ones.

[valff]

It is impossible to invent a name which describes the meaning of Unpin to a beginner without reading documentation. Send and Sync are also hard to understand for a beginner, but they look nice and concise, as well as Unpin. If you don't know what these names mean, you must read their docs.

[Pauan]

@valff Indeed that is correct, however there are quite a few people who have read the docs and understand how pinning works, yet the Unpin name still causes them significant mental difficulty, whereas the other names cause less.

[alexcrichton]

@glaebhoerl oh sorry to clarify I'm personally more of a fan of MoveFromPin or CanUnpin than the current Unpin. Just trying to help move to a conclusion!

[HeroicKatora]

I tried understanding the current proposal, maybe this sort of progress report can shine some additional light on naming as well.

• Extending the guarantees of Pin to a stable address until Drop puts additional requirements on the creator of a Pin. I would have found it useful if those necessary preconditions for invoking unsafe fn new_unchecked(pointer: P) -> Pin<P> were mentioned directly. Understanding how a Pin can be created helps hugely in understanding their concept, imo.
• The summary mentioned in the tracking issue, prefaced with we agreed that we are ready to stabilize them with no major API changes., contains a lot of old api. At the same time, it also contains a lot of insight about Drop and pin projection that is relevant. It also contains an explicit formulation of the additional guarantee referred to in bullet point one that is not found in this report. This mix of outdated and fresh information was a bit confusing, consider moving all of the information into this issue or indicating outdate paragraphs.
• Since the additional guarantee is referred to as a slight extension, I expected there to be some kind of way to opt out of it. Since pinning once carries a type state on that pointer until it is dropped, some way to go back from this type state to a 'normal' state. In fact, that was what I thought Unpin to do at first but I think Unpin is actually a bit stronger. It says the the pinned state can be freely turned into an unpinned state, at will. It's fine that the current interface is minimal in this regard but Unpin could be confused with an operation on the type that removes some inner invariant that requires the pin state (like a soft version of Drop).

Clarification: I personally favour MoveFromPin over the other names but it is arguably artificial and clunky.

The major actionable concrete actionable item that comes to mind, besides the documentation improvements already requested by @alexcrichton , would be explaining the reasoning behind the field rule for pin projection in map_unchecked and map_unchecked_mut. Something along the lines of:

A pin promises to not move the referred data until it is dropped. Since the fields of a struct are dropped after their containing struct, their pin state extends beyond the Drop implementation of the structs itself. Projecting to a field promises not to move from it within the structs destructor.

[HeroicKatora]

What about the name ElidePin for the auto derived marker trait?

It would capture that the type can always be treated as if or behaves as if it were unpinned. And !ElidePin seems to be quite clear as well, although that might be subjective.

The standard marker doesn't have a perfect name for understanding pinning as well. Pinned seems to evoke the notion that the containing struct itself is pinned but pinning applies to pointers and only after an explicit action. This is not quite as critical but not insignificant either, especially since it could be the type that is first encountered in an implementation for a self-referential type.

[cramertj]

My general opposition to MoveFromUnpin/RemoveFromUnpin is just that they're far too wordy and would regress the ergonomics of manual Future implementations. CanUnpin/DePin both seem fine to me-- I wish it was more clear that Unpin follows the pattern of Read/Write/etc. but it seems that's not intuitive to folks so I'll +1 anything that makes this clearer without looking like syntactic salt.

[jeekobu]

I agree NotWhateverUnpinBecomes is probably the best name for Pinned. That said, Adhere means both to heed and to stick to. :slightly_smiling_face:

[wmanley]

CanUnpin/DePin both seem fine to me-- I wish it was more clear that Unpin follows the pattern of Read/Write/etc

I think one of the things that makes Unpin hard, unlike Read is that it is a marker trait. Read is easy to understand because there is a method Read::read - everything you need to know is right there in the trait. If x is Read I understand that I can call x.read() - similarly write for Write, etc. It's harder to explain that X implementing Unpin means that Pin<Ptr<X>> implements DerefMut - which means that you can treat it as if it were just X.

[cramertj]

Read is easy to understand because there is a method Read::read

If only we could add always-applicable methods in auto trait definitions + GATs, we could have Unpin::unpin-- fn unpin<P: DerefFamily>(self: Pin<P::Deref<Self>>) -> P::Deref<Self>). .....on second thought, I don't think that's going to make anyone less confused ;)

(on a more serious note, I'd support Pin::unpin for going from Pin<P<T>> to P<T>)

[HeroicKatora]

I'd support Pin::unpin for going from Pin> to P

This confuses two terms in my head, just like the current name itself. unpin sounds very much like reversing the guarantees of the type state, for a comparison that would be as if there was a method fn unborrow(&'_ T) or fn unmove(??). Since pin provides guarantees until some memory representing a type is Drop::droped, we don't really reverse the state, the type merely guarantees that all other representations uphold equivalent guarantees and we can thus ignore this. This is also the main difference I see between marker traits and something like io::Read. One enables operations for the compiler or language, while the other enables operations for the programmer.

Furthermore, it is a major point that this cannot currently be accurately represented just using correct typing. To call the operation unpin makes it sound like there was a converse function pin. It being a function also slightly wrongly hints that such an unpin operation is somehow bound to computational work, for example into_pointer() would be bettter in this regard by also following a more established naming convention.

Lastly, I think there's potential for types specifically having an unpin function with computational work. A type with very special inner invariants might be able to 'fix' its inner state in such a way that it can offer an interface fn unpin(self: Pin<&'a mut T>) -> &'a mut, where it willingly forfeits all guarantees of Pin for some lifetime 'a. In that case, both of the above points no longer apply. Such a function could be envisioned as if it had equivalent effect to dropping and reconstructing in the same memory location (thus actually removing the type state). And it may involve computation, e.g. by moving some self references into dynamic allocations.

It would be unfortunate if a confusing name would also make it harder for library designers and implementors to choose non-confusing names, by conflating these two ideas.

[Matthias247]

My general opposition to MoveFromUnpin/RemoveFromUnpin is just that they're far too wordy and would regress the ergonomics of manual Future implementations.

I don't think that's true, especially for MoveFromPin, which seems reasonable to type for me - and with any kind of smart or dumb autocompletion the problem is mostly nonexistent anyway.

One important part of ergonomics should be also readability and understandability of code. Rust already earned a bit of criticism in the past for abbreviating things aggressively (fn, mut, etc), which makes some code harder to read. For things that for a concept that is even more complicated to grasp and which fulfills a niche purpose for most users, using a more verbose and descriptive name should be totally acceptable.

[alexcrichton]

@rfcbot resolve naming-of-Unpin

Ok I've been stewing for awhile now and have also talked with @withoutboats in person about this. I've come around that I'm now personally comfortable at least with the name Unpin as the marker trait. As a result, I'm going to remove a blocking objection (although if others on the libs team feel differently, please let us know!)

The main reason that I've come around to Unpin is basically what @cramertj already said above, it's the most idiomatic name for this trait that we've been able to come up with. All other trait names I've seen proposed don't meet the idiomatic bar that Unpin lives up to (or at least in my opinion).

While I still believe there are better names for "I just saw the name of this trait and I want to know what it does", I don't think that's the right problem to solve here. It's clear to me at this point that we can't, at this time at least, figure out a different name for this trait which is also idiomatic, but rather we're switching around on trait names that solve a different problem. Understanding Unpin is just like Send and Sync, the documentation needs to be read to understand fully what's going on.

---

As another clarifying point, I listed a few "blocking objections" but they're moreso TODO items than blocking objections per se. I just don't have a great way of otherwise navigating such a long thread! In that vein I think it's fine for a stabilization PR to be posted at any time now with FCP lapsed for a week or so now. The final points about self/pin/pinned can be briefly discussed there (if even necessary, they could be left as the proposal above).

Documentation I think especially is not a prerequisite for stabilization in this case. We've got a full cycle (6 weeks) to add docs before these types hit stable and we've got even longer to shore up the documentation here before the full async/await story hits stable. That's loads of time to improve what's there now, and what's there now is already quite useful!

[jaredr]

What does "idiomatic" even mean here? What's unidiomatic about Reloc[ate], Escape, Evade, Pluck, or Roam? They're all verbs, and none of them can be mistaken for having the double-negative problem.

[Pauan]

@alexcrichton Is there a reason why Unpin is considered more idiomatic than Depin or DePin?

I think Depin is a very solid alternative, and it doesn't cause me the same mental difficulties as Unpin.

[jimmycuadra]

One simple reason might be that depin isn't a word and unpin is. Personally, I don't have any trouble understanding Unpin. I think it fits with the naming of the other marker traits.

[Pauan]

@jimmycuadra There's a lot of names in Rust which aren't "real" words, including in the stdlib.

I would be surprised if that was considered a significant reason to choose one name over another.

[withoutboats]

@Pauan It is a significant reason. As a native English speaker, Depin sounds to me like we forgot that the word for unpinning exists and tried to make one up. It seems glaringly wrong to me gramatically: the English word for "depinning" is "unpinning."

A good analogy would be if we had APIs that said "delock" instead of "unlock".

[alexcrichton]

@jaredr by "idiomatic" I mean following the established conventions of the standard library already like the (previously mentioned) Read and Write traits. We have a convention of not-wordy-names, verbs, short where possible, and appropriate for the situation.

Names like you've suggested are all possible, but Unpin (or so I feel at least) is the most appropriate for this particular action ("you can unpin this type" guarantee). The others, while loose synonyms of Unpin, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin as Unpin does.

@Pauan I agree with @withoutboats that Depin doesn't sound like it jives as well as Unpin.

[Centril]

@aturon noted in https://github.com/rust-lang/rfcs/pull/2592#issuecomment-438873636 that:

The use of Pin is no more of an implementation detail than the choice of &self versus &mut self; in the long run, it's likely that Pin will itself be a separate mode of reference with language support. In all these cases, what's being conveyed in the signature is the permissions being granted to the callee, with Pin forbidding moving out of the reference.

Ostensibly this refers to a native &pin type or something... but idk how this squares with Pin<&mut T> and such. In my conversations with @withoutboats and in particular @cramertj they were not at all sure about the idea of a separate mode of reference with language support and how we could get there from Pin<T>.

Before stabilizing pin, it would be prudent to reconcile these views to ensure that we are on the same page wrt. this critical piece of infrastructure; so I'd mainly like for Aaron to expand on this and for boats and Taylor to then weigh in.

[crlf0710]

The others, while loose synonyms of Unpin, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin as Unpin does.

@alexcrichton this is actually a good thing, i think? Like Send to move/copy (=) operation, Sync to borrow (&) operation. Maybe such connection is actually causing more confusion?

[alexcrichton]

@crlf0710 it may! I'm not sure I would agree with such a connection though myself. Send and Sync are more about what they're enabling ("send"ing types to other threads and "sync"hronizing access across threads), and when naming them we didn't try to avoid naming them close to one operation or another

[crlf0710]

@alexcrichton exactly! so maybe this trait should also be about what it is enabling. ("moving(A verb here) out of a Pin). I'm not native english speaker, but i still think "unpining" out of a pin is a little...weird?

[withoutboats]

@crlf0710 But what the trait enables is not <moving> out of a Pin, its <moving out of a Pin>. The problem with move and synonyms for move is that they imply the trait controls the ability of the type to move at all, which it does not do. The connection to Pin is vital to understanding what the trait actually does.

So the most idiomatic name for this trait would be a verb which means "to move out of a pin," for which the word "Unpin" is the most obvious to me by a lot. Here's Wiktionary's definition of "unpin":

1. To unfasten by removing a pin.
2. (transitive, computing, graphical user interface) To detach (an icon, application, etc.) from the place where it was previously pinned.

[crlf0710]

@withoutboats thanks for explaining! Actually i think i can accept this Unpin name or any name rust team finally decides on, i don't want to block the stabilization of Pin types at all, and i absolutely understands the concerns about "moving" etc.

It just feels there's a very slightest "repetition" somewhere. and i have to persuade my self: it doesn't mean "unpinnable", but the opposite. I guess after some time i'll get used to it if that's the final decision.

At the same time please allow me to make my last suggestion: actually I think the Detach verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.

[qnighy]

I was thinking about safe disjoint pinned projections, and came up with the idea that may enable it. This is a macro to simulate matching against pinned mutable lvalues. With this macro we can write

pin_proj! { let MyStruct { field1, field2 } = a_pinned_mutable_reference; }

to decompose a Pin<&mut MyStruct> into pinned mutable references to the fields.

To make this safe and usable, we'll need two other things:

• The Kite type to mark "always-Unpin" fields
• Make Unpin unsafe

The latter is needed for the safety of the projection. Without this, we can define "Kite with pinned projections" safely, which is actually wrong.

With this in mind, I propose to make Unpin unsafe before stabilization to leave a room for other useful operations to be safe.

[withoutboats]

@qnighy Its possible for a type to violate pinning guarantees in its Drop implementation, which makes Drop equivalently powerful to Unpin. Making Unpin unsafe does not make any additional code safe, because Drop is also safe, and that can't be changed. We've talked about this a lot on the tracking issue, and its been mentioned on this thread as well.

Fundamentally, pin projections cannot be made safe without the ability to assert certain negative bounds that can't be expressed in Rust today.

@crlf0710

At the same time please allow me to make my last suggestion: actually I think the Detach verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.

Detach seems a lot better than names like Move and Relocate, but it seems to conflict with other possible uses of the detach metaphor (similar to how Escape could conflict with "escape analysis" etc). There's only so many metaphors we have about how data can relate to other data in computer science, which makes me think of another new advantage of Unpin: by cleaving tightly to the "pin" metaphor, it doesn't occupy space for future metaphoric language we may need to use for a different purpose, the way names like Detach or Escape or Relocate could.

[Centril]

@withoutboats I have no particular preference for any name or much investment in this bikeshed... but I'm curious; what other purpose would Detach fit (if you speculate...)?

[withoutboats]

@Centril I don't have a clear use case in mind, but I could imagine some use case related to destructuring for example.

[Centril]

@withoutboats Yeah that makes sense; cheers!

[HeroicKatora]

@withoutboats I'm not sure if the Wiktionary entry is the best motivation to justify the naming of Unpin to enable move from a pin. The physical representation metaphors suffer from the fact that moving in Rust is not taking away an object in the world. To be more precise, 'upinning' a pinned pointer does not give me control over the referred to memory, it's allocation and validity as an object representation of T are still required and guaranteed by the pointer semantics. Hence unpinning does not give a fully controlled representation of T, as unboxing would. And, a dictionary entry defining unpinning in terms of moving is in fact part of the confusion more than the justification of such a name.

In the proposed API, none of the methods has such an effect (and it's not in the scope of pin either). I don't see a safe way for example to transition Pin<Box<T>> to Box<T> (and by extension to T) and nor am I sure if Unpin should have such strong possibilities. I'm not exactly sure where all the difference would be. But as far as I understood, Pin applies to some memory location while Unpin guarantees that nothing in Ts representation relies on pin guarantees. However, would that be the same as being able to take out of and forget the memory entirely? I think not. I'll think of a more concrete example.

Edit: More concretely, even if T is Unpin, can I rely on some instance of T::drop(&mut) being called on the memory that was pinned, before that memory is deallocated? As far as I can tell from the formalism, the answer should be yes, but the name Unpin communicates to me the opposite.

Edit 2: Rc allows one to observe Pin<&T> but for T: Unpin still not have drop called on the original memory location. Keep a reference alive outside the pin, then after the pin was dropped you can move out with Rc::try_unwrap. https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=dec6f6c6d2c0903d87a4a9cefe50a0ca That effectively answers the question through existing mechanisms but does it work as intended?

[wmanley]

Perhaps IgnorePin? If T is IgnorePin you can treat Pin<Box<T>> as &mut T essentially ignoring the presence of Pin (AIUI also ignoring the Box). To ignore is a verb, I guess that IgnorePin isn't but I'm not sure what it is. IgnorePin is descriptive of what it allows, it's not descriptive of the constraints placed upon the type, but neither is Unpin.

[HeroicKatora]

@wmanley I had a very similar idea, ElidePin, in some comment above although back then I had not been able to express concretely why that one felt more precise. But I concur that 'one verb' is the style guide for Rust marker traits. Even though it also allows a more natural negation in the form of !ElidePin/!IgnorePin, it is not optimal.

[HeroicKatora]

@withoutboats Follow up question: Since pin seems specified in terms of the underlying memory, how does Pin interact with ZST? Due to Pinned being a ZST, even a ZST may be either Unpin or not. I would intuitively say that its memory representation is never formally invalidated, thus T::drop(&mut self) need never be called, quite similar to how one could build a pin out of &mut 'static _ memory for example from a leaked Box. At the same time, that may all be wrong and I see how another interpretation would be possible. I feel like these settings deserve attention in the documentation.

[tikue]

Is it safe to create a Pin<P<T>> with a T: !Unpin and then immediately unpin the value, if it's guaranteed that nothing observed the pinning? Is it only undefined behavior if the pinned value is moved after it has had a poll-like method called on it?

[cramertj]

@HeroicKatora Unfortunately, it's possible today to implement Drop for a type that could be !Unpin due to generics, e.g.:

struct S<T>(T); // `!Unpin` if `T: !Unpin`
impl<T> Drop for S<T> { ... }

[HeroicKatora]

@cramertj Thank you, just realized that as well.

[HeroicKatora]

@cramertj So we default to T: ?Unpin on generic bounds? Is there another reason behind not defaulting to T: Unpin for existing types like Sized? It would give a few instances where that would be annoyingly strict but can that additional auto bound cause regressions?

[cramertj]

@HeroicKatora That would require sprinkling ?Unpin bounds on every type in the standard library and in a bunch of code that doesn't care about Unpin at all.

[withoutboats]

It's also safe to add a PhantomPinned field, as another way of creating a Drop + !Unpin type.

Is there another reason behind not defaulting to T: Unpin for existing types?

Unpin is just an auto trait like Send and Sync, this proposal involves no new language features. So it has the same semantics as auto traits are already defined to have, which is that they are not applied to generics by default (unlike Sized, which is a built in trait to the compiler, not an auto trait).

Is it only undefined behavior if the pinned value is moved after it has had a poll-like method called on it?

We should be clear here that undefined behavior in this context is not really the traditional sort of UB. Rather, its that code, observing a type in a Pin, can make assumptions about the ownership semantics of that value (namely, that if it does not implement Unpin it will not be moved again or invalidated until after its destructor has run). It's not UB in the sense that it is assumed by the compiler never to happen (the compiler doesnt know what a Pin is).

So yes, in the sense that you can verify that no code is relying on the guarantee you have provided. But also what you have done is certainly pointless, since no code could observe that the type was pinned.

[HeroicKatora]

@cramertj I see, that would be a bit unfortunate. I'm a bit uncomfortable with Pin interaction with Drop in the specification but not in the language. Quite conflicting between interface correctness, usability and releasing it in the near future. Can I get clarification on Edit 2 above:

Rc allows one to observe Pin<&T> but for T: Unpin still not have drop called on the original memory location. Keep a reference alive outside the pin, then after the pin was dropped you can move out with Rc::try_unwrap. https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=dec6f6c6d2c0903d87a4a9cefe50a0ca

[withoutboats]

@HeroicKatora the problem with your code is that Mem<'a>: Unpin, but the line of unsafe code you use relies on assumptions that only apply to types that don't implement Unpin. I've edited your gist to include proof that Mem<'a>: Unpin: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=201d1ae382f590be8c5cac13af279aff

You rely on the Unpin bound when you call Pin::new, which can only be called on pointers whose target implements Unpin.

The loophole you thought you found has been considered, which is why there is no way to go from an Rc<T: ?Unpin> to a Pin<Rc<T>>. You have to construct the Rc in the pin with Rc::pinned.

[HeroicKatora]

@withoutboats Just wanted to get that confirmed, that T: Unpin indeed also opts-out of the drop call before invalidation. That begs the question, should there not also be

fn into_inner(Pin<P>) -> P where P: Deref, P::Target: Unpin;

since it does not protect any part of the interface, and unwrapping a Pin<Box<T>> into Box<T> is potentially useful (for T: Unpin of course).

[withoutboats]

@HeroicKatora you're correct that that function would be safe. I don't mind adding it but I'd like to avoid expanding the API we're stabilizing in this moment, since this thread is already hundreds of comments long. We can always add it later as the need arises just as we do with all std APIs.

[HeroicKatora]

I would just say that both naming and functionality of Unpin makes a lot more perceived sense with that converse method. And knowing that the Pin guarantees are indeed restricted to T: !Unpin. That would also be directly demonstrated by the existence of such a method, instead of constructing escape hatches to show the possibility :smile: . And its documentation would be a perfect place to explain (or link once more to) the restrictions. (One might even consider naming it unpin instead of into_inner).

Edit: It would mostly just generalize what's already there.

impl<P> Pin<P> where P: Deref, P::Target: Unpin

• fn unpin(self) -> P

has the instantiation for P=&'a mut T, that is equivalent to the proposed

impl<'a, T: ?Sized> Pin<&'a mut T> where T: Unpin

• fn get_mut(self) -> &'a mut T

[Centril]

Now that the stabilization has gone through, the point of the PFCP seems moot, so therefore:

@rfcbot cancel

[RalfJung]

Is there any tracking to make sure the comments developed starting at https://github.com/rust-lang/rust/issues/55766#issuecomment-437374982 get turned into docs? Seems we are already too late for the release as beta got branched off... even though that was explicitly called out here to happen before stabilization :/

[RalfJung]

Is there any reason that this is still open? @Centril you closed and reopened this, was that deliberate?

[Centril]

@RalfJung I tried to cancel the FCP but it didn't listen; @alexcrichton can you cancel the FCP?

[alexcrichton]

This is stable, so closing.