Implement parsing of pinned borrows

[frank-king]

This PR implements part of #130494.

EDIT: It introduces &pin mut $place and &pin const $place as sugars for std::pin::pin!($place) and its shared reference equivalent, except that $place will not be moved when borrowing. The borrow check will be in charge of enforcing places cannot be moved or mutably borrowed since being pinned till dropped.

Implementation steps:

• parse the &pin mut $place and &pin const $place syntaxes
• borrowck of &pin mut|const
• support autoref of &pin mut|const when needed

[rustbot]

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred to the CTFE machinery

cc @rust-lang/wg-const-eval

[rustbot]

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

[petrochenkov]

r? compiler

[traviscross]

It introduces &pin const place and &pin mut place as sugars for Pin::new(&place) and unsafe { Pin::new_unchecked(&mut place) }.

This is not the semantic we want or a good placeholder for it. What we want is that &pin mut $place and &pin const $place do safe pinning and safe pin projection. Probably the right placeholder is that &pin mut $place does pin!($place) and &pin const $place does the shared reference equivalent of that. Then, the next extension is to make this also work:

fn f<T: Unpin>(x: &mut T) {
    let _: Pin<&mut T> = &pin mut *x; // i.e., `Pin::new(&mut *x)`.
}

That is, if the type in the place is Unpin, we can skip the precautionary move.

After that, we start talking about how to extend this to safe pin projection to struct fields.

cc @tmandry @eholk

[traviscross]

r? traviscross

[frank-king]

It introduces &pin const place and &pin mut place as sugars for Pin::new(&place) and unsafe { Pin::new_unchecked(&mut place) }.

This is not the semantic we want or a good placeholder for it. What we want is that &pin mut $place and &pin const $place do safe pinning and safe pin projection. Probably the right placeholder is that &pin mut $place does pin!($place) and &pin const $place does the shared reference equivalent of that.

I agree with you, and my previous statement was not precise. What I mean is similar to yours. I expect &pin mut $place to apply only to pinned places, &mut $place and move $place (move comes from the MIR terminology I think) to apply only to non-pinned places, and they do not intersect with each other.

The pinned places might be defined as:

A place is a pinned place if one of the following conditions holds:

• It is dereferenced from a Pin<&mut T> (when T: Unpin, it can also be considered as a non-pinned place).
• It is a binding variable of type T declared by pin mut (I use the feature introduced by Parse pinned local variable declarations #135631). When T: Unpin, it can also be considered as a non-pinned place.
• It is a temporary (temporaries can be considered as either pinned places or non-pinned places).
• It is a field access of a pinned place, and the field must have type T that doesn't impl Unpin.
• It is an indexing (const or non-const) or subslicing of a pinned place.

Here are a few examples to explain:

#[derive(Default)]
struct Foo<T> {
    x: T,
}

fn f<T: Default, U: Default + Unpin>() {
   // Case 1: not `Unpin`, and not pinned
   let mut a = Foo::<T>::default(); // `a` and `a.x` are non-pinned places but not pinned places
   let _: &mut Foo = &mut a; // ok
   let _: &mut T = &mut a.x; // ok
   let _: Pin<&mut Foo> = &pin mut a; // ERROR
   let _: Pin<&mut T> = &pin mut a.x; // ERROR

   // Case 2: not `Unpin`, and pinned
   let pin mut b = Foo::<T>::default(); // `b` and `b.x` are pinned places but not non-pinned places
   let _: &mut Foo = &mut b; // ERROR
   let _: &mut Foo = &mut b.x; // ERROR
   let _: &pin mut Foo = &pin mut b; // ok
   let _: &pin mut Foo = &pin mut b.x; // ok

   // Case 3: `Unpin`, and not pinned
   let mut c = Foo::<U>::default(); // `c` and `c.x` are both pinned and non-pinned places
   let _: &mut Foo = &mut c; // ok
   let _: &mut T = &mut c.x; // ok
   let _: Pin<&mut Foo> = &pin mut c; // ok
   let _: Pin<&mut T> = &pin mut c.x; // ok

   // Case 4: `Unpin`, and pinned
   let pin mut d = Foo::<U>::default(); // WARN (useless `pin`) // `d` and `d.x` are both pinned and non-pinned places
   let _: &mut Foo = &mut d; // ok
   let _: &mut T = &mut d.x; // ok
   let _: Pin<&mut Foo> = &pin mut d; // ok (useless `pin`)
   let _: Pin<&mut T> = &pin mut d.x; // ok (useless `pin`)
}

[traviscross]

Thanks, that's helpful. Going through the examples, starting on Case 1, we're actually instead interested in this working:

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
    let _: &Foo<T> = &x; //~ OK
    let _: &mut Foo<T> = &mut x; //~ ERROR place has been pinned
    let _move: Foo<T> = x; //~ ERROR place has been pinned
}

The &pin mut x is allowed even though the type is not Unpin because the borrow checker will track this and then prevent a later mutable reference to or move of the place. Of course, if the type of the place is Unpin, then these can be allowed.

In this way, &pin mut $place then entirely subsumes the pin! macro.

This is also why we're not sold on doing let pin mut, because we suspect we may not need it with these mechanics.

Regarding the safe pin projections to fields, let's do that in a separate PR so we can set those questions aside for the moment. That a place is pinned doesn't necessarily imply that we can safely pin project to a field without taking a number of other steps.

[frank-king]

Thanks for your feedback too! Regarding your code snippet, I have some questions.

First, this is the semantics of pin!($place), where $place is moved after been pinned (playground).

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    let _: Pin<&mut Foo<T>> = pin!(x); //~ OK
    let _: &Foo<T> = &x; //~ ERROR place has been moved
    let _: &mut Foo<T> = &mut x; //~ ERROR place has been moved
    let _move: Foo<T> = x; //~ ERROR place has been moved
}

Then in your version, &pin mut $place acts like pin!($place) but more similar to a normal borrow without moving the place. However, $place is still pinned even if the pinned borrow has expired.

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
    let _: &Foo<T> = &x; //~ OK
    let _: &mut Foo<T> = &mut x; //~ ERROR place has been pinned
    let _move: Foo<T> = x; //~ ERROR place has been pinned
}

What I expect is, the borrow checker should allow the original place to be mutably borrowed or moved after all pinned borrows expired (if we don't introduce let pin mut to make places pinned explicitly), that is:

fn ignore<T>(_: T) {}
struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    {
        let _pinned: Pin<&mut Foo<T>> = &pin mut x; //~ OK
        let _: &Foo<T> = &x; //~ OK
        // Mutable borrows or moves are not allowed during the pinned borrow is alive
        let _: &mut Foo<T> = &mut x; //~ ERROR place has been pinned
        let _move: Foo<T> = x; //~ ERROR place has been pinned

        // Make sure `_pinned` is used
        ignore(_pinned);
    }
    let _: &Foo<T> = &x; //~ OK
    // Mutable borrows or moves are allowed again after the pinned borrow expires
    let _: &mut Foo<T> = &mut x; //~ OK
    let _move: Foo<T> = x; //~ OK
}

[traviscross]

What I expect is, the borrow checker should allow the original place to be mutably borrowed or moved after all pinned borrows expired...

This can't be allowed. The contract with Pin is that once something has been pinned, its memory cannot be invalidated or repurposed until after drop has run, and that implies we cannot allow mutable references to exist to it or for it to be moved, even after that pinned reference is out of scope.

What the borrow checker must do here, then, is to flag the &pin mut place in a way that once the pinned mutable reference is dead it's OK to allow new pinned references or shared ones, but it's not OK to allow a later unpinned mutable reference or a move. So we'd expect, as above:

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    let _: &mut Foo<T> = &mut x; //~ OK
    //~^ Before we pin the thing, it's OK for mutable references to
    //~| exist to it.
    //~|
    //~| Note that the above mutable reference is dead at this point.
    let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
    //~^ Now we create a pinned mutable reference.  This is allowed
    //~| since the earlier mutable reference is dead.  Once this
    //~| pinned reference is created, we can no longer allow the
    //~| pointed to memory to be invalidated or repurposed until
    //~| after `drop` has been run, even after this pinned reference
    //~| is dead.
    //~|
    //~| Note that the pinned reference is dead at this point.
    let _: &Foo<T> = &x; //~ OK
    //~^ We can allow shared references because they don't allow us
    //~| to invalidate or repurpose the memory.
    let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
    //~^ We can allow a new pinned mutable reference for the same
    //~| reason.
    let _: Pin<&Foo<T>> = &pin const x; //~ OK
    //~^ Or a new pinned shared reference, for that matter.
    let _: &mut Foo<T> = &mut x; //~ ERROR place has been pinned
    //~^ We can't allow a mutable reference to exist, though, because
    //~| that would violate the contract.
    let _move: Foo<T> = x; //~ ERROR place has been pinned
    //~^ Neither can we allow the once-pinned thing to be moved.
}

What the pin! macro does is enforce this contract in a crude way by forcing a move. With this work, that trick wouldn't (we hope) be necessary, as the borrow checker would directly enforce the pinning rules.

[frank-king]

This can't be allowed. The contract with Pin is that once something has been pinned, its memory cannot be invalidated or repurposed until after drop has run, and that implies we cannot allow mutable references to exist to it or for it to be moved, even after that pinned reference is out of scope.

Oh, yes, I forgot that Pin also interact with drops. Then I'd slightly prefer explicitly mark places pinned via let pin mut or else it can be anti-intuitive that &pin mut|const looks like borrows but the pin effect is not tied to the borrow's lifetime. Though it will lose some flexibility that a place must be pinned at creation. (BTW, I personally don't really like the taste that places can be freely moved before being pinned. Anyway, we can still set it aside and revisit it later when doing something like constructing values in place.)

[traviscross]

Yes, the intuition point is something we'll I'm sure consider later. We want to try it first in the way mentioned.

[tmandry]

What @traviscross said above sounds right to me. I would also mention that we want dispatch on self: Pin<&mut Self> methods to work with pin autoref, and to have all the same invariants enforced. Since borrow check is implemented on MIR I think it should be agnostic to this, but it's important for the end user story.

[bors]

☔ The latest upstream changes (presumably #138114) made this pull request unmergeable. Please resolve the merge conflicts.

[rustbot]

Some changes occurred to constck

cc @fee1-dead

[frank-king]

@rustbot ready
(I assume that this PR only implements parsing of &pin mut|const syntax, the borrowck and autoref stuffs will be implemented in the next PRs)

[traviscross]

This is wildly unsound, which gives me pause even about doing it on nightly, but the feature flag is marked as incomplete, and we're planning to follow-up with the borrow checker rules, so I suppose this is OK.

[traviscross]

@rustbot author

The author addressed.

[small-white0-0]

What I expect is, the borrow checker should allow the original place to be mutably borrowed or moved after all pinned borrows expired...

This can't be allowed. The contract with Pin is that once something has been pinned, its memory cannot be invalidated or repurposed until after drop has run, and that implies we cannot allow mutable references to exist to it or for it to be moved, even after that pinned reference is out of scope.

According to this requirement, using &pin mut $place for a non-pinned place might be quite confusing.

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    {
        let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
        let _: &Foo<T> = &x; //~ OK
    }
    // The following code error is confusing.
    // Because the pinned reference has disappeared, 
    // but mutable references still cannot be used.
    let _: &mut Foo<T> = &mut x; //~ ERROR place has been pinned
    let _move: Foo<T> = x; //~ ERROR place has been pinned
}

However, the following approach seems much clearer and more consistent with the current habits of mutable and immutable references.

struct Foo<T> { x: T }
fn f<T>(mut x: Foo<T>) {
    let pin mut x = x;
    {
        let _: Pin<&mut Foo<T>> = &pin mut x; //~ OK
        let _: &Foo<T> = &x; //~ OK
    }
    let _: &mut Foo<T> = &mut x; //~ ERROR place is pinned, like `&mut x` fail because `let x = 1` is immutable.
    let _move: Foo<T> = x; //~ ERROR place is pinned
}

This is just a beginner's perspective. The latter approach makes the code easier to understand for me, even without knowing some of the deeper underlying reasons.

If there are any mistakes, I apologize in advance.

Regards,

[frank-king]

@rustbot ready