Name-based + new position-based destructuring

[serras]

This is an issue to discuss name-based destructuring and new syntax for position-based destructuring. The current full text of the proposal can be found here.

We propose a new form of destructuring based on names instead of positions. In addition, we propose new syntax for position-based destructuring.

(val age, val name = fullName) = person

⚠️ Please do not begin discussions about syntax before thoroughly reading the rationale (part 1 on Syntax, part 2 on Grammar). The current choice strikes a balance between familiarity, but also the need for the new syntax to be completely backwards-compatible with the rest of the Kotlin syntax and being able to parse it without lookeahead.

[serras]

One of the open design questions is what properties should be available through this syntax. There are three possible levels:

🚀 Only primary constructor properties 1️⃣,
🎉 Only member properties 2️⃣,
❤️ Any property, including extensions 3️⃣.

To direct discussion, I would love to gather feedback on this specific point in this thread. Please use emojis 🚀/🎉/❤️ (edit: GitHub does not allow generic emoji responses) if you just want to state "I prefer this option" without any additional reasoning.

[babisRoutis]

We can already define componentN as extension function, enabling positional "destructuring" even for classes outside our code-base (even java).
For consistency, I would expect the same to work for the name-based case, as well.

Option 3 for me

[klitoskyriacou]

The multiple-inheritance issue mentioned above is moot, as Impl won't compile unless you override chars. I vote for Option 3.

[zhelenskiy]

Yes, there is no difference between it and just regular property use.

[ZacSweers]

@klitoskyriacou fair point! I didn't realize the compiler prohibits this 👍

[pdvrieze]

Option 1 is a challenge for maintainability/the ability to change implementations. It effectively exposes an implementation detail where there is no need to do so from a source/binary compatibility reason. As such I would advocate for either option 2 or 3.

[zhelenskiy]

About further extensions:

Assignment. This proposal only defines destructuring to declare new variables. It could be possible to extend the syntax to also assign new values to existing variables.

var name = "unknown"
(name, val age) = person

However, such syntax raises some questions, like whether it is possible to mix declaration and assignment. Since destructuring has only been available for declaration in Kotlin up to this point, we think respecting that decision is the best way to go.

I think allowing assignments without mixing with declarations is already useful and covers a common case:

[a, b] = b to a

Mixing can be added if the use-cases are found.

[zhelenskiy]

Do you find Python

a, b = b, a

tricky?

IMHO, it is more obvious than some C-like

swap(&a, &b)

---

I guess, swap is quite useful, but I don't have any exact numbers.

Besides swap, the usecase is to reassign a property in a loop:

[var a, var b] = initialC
for (x in y) {
    if (someCondition(x)) {
        [a, b] = newC(x)
    }
}

[serras]

Not super-tricky, and once they see it, almost everybody recognizes the pattern. But I see it definitely as stretching some syntax to cover an additional operation. About the case for reassigning a property in a loop, Kotlin tries to push people towards using things like fold and reduce.

Again, not saying we should not have assignment. Just that these cases are IMO not common enough.

[zhelenskiy]

fold and reduce are not always the best options as Kotlin is not functional-only language. For example, it is often a bad practice to use maps, filters and other operations instead of loops if the collection creation is not the only or the main output of iteration. Here it works as well.

[CLOVIS-AI]

Do you find Python

a, b = b, a

tricky?

Personally, I do. I always parse this in my head as (a), (b = b), (a) because it looks like the syntax for named arguments in a function call. I dislike that = has priority over , in assignments, but the opposite is true in argument lists.

[zhelenskiy]

I meant the syntax at all. Of course, I prefer the Cs version

(a, b) = (b, a)

[zhelenskiy]

I see a problem in the current design. From the declaration below it is not fully comprehensible, which of the names is a property of a class and which is a variable.

(val name1 = name2, ...) = ...

One may say that the one near val is always a new declaration, but in parameters the comprehension is different:

f(a = b)

The first one is proposed by the declaration owner and the SECOND one is OUR argument. In the proposed feature the FIRST one is OUR name.

Do you consider something like:

(val name1 = .name2, ...) = ...

(val name1 = declaration.name2, ...) = ...

(val name1 = decl.name2, ...) = ...

The following was also suggested in the comments:

(val name1 = _.name2, ...) = ...

Another problem is that if you have a variable with the declaration name above, it is not obvios that you reference not it:

val name2 = "#"
(val name1 = name2, ...) = ... // name2 here is not from above

So, the .name2 and _.name2 syntaxes solve the both problems.

[serras]

In particular with .name, we've explored this syntax in relation to the context-sensitive resolution proposal, and it brings a lot of complexity to the grammar. Thus we may need to choose some other syntax (like (val name1 = _.name2)).

[zhelenskiy]

_. is totally fine, what is not fine is the current suggestion. Especially because of the second reason.

[zhelenskiy]

By now, I like the _ the most because it is literally a placeholder for a generally unused variable. It solves all the mentioned problems and doesn't introduce the new ones.

[lengors]

From the declaration below it is not fully comprehensible

I disagree with this part. You always assign the value from the right identifier to the left identifier, so in both of your examples (named-based destructuring and parameter assignment) while the declarer swaps positions, the assignment is done in the same direction so you can infer what is what based on context.

So, in parameter assignment, to me it's intuitive that in a = b, the a is declared in the function/method, while b is the variable with the content we are assigning it (the parameter) to when invoking said function.

Likewise, in destructuring, to me it's intuitive that in (val a = b, ...) = ..., the a is the variable we are assigning a value to, and b the declared property in the object we are assigning the destructuring to.

However, I do somewhat agree with this:

Another problem is that if you have a variable with the declaration name above, it is not obvious that you reference not it

Even though it can be inferred from context, I wouldn't mind having an opt-in syntax to make it clearer (or even forced said opt-in syntax in case of name collision).

[zhelenskiy]

@lengors

Regarding the first part, I partially agree, partially disagree with it. Everything depends on the mental model your brain chooses.

Regarding the second part, I would prefer unambiguous syntax to the warnings in the actual case of ambiguity.

I'm pretty sure that the following is unambiguous to you and still concise:

val a = 3
val (b = _.a, ...) = foo()

[vlsi]

data class UserData(val name: String, val lastName: String) is a bad example since lastName can't be a required property for a user.

See:

• https://shinesolutions.com/2018/01/08/falsehoods-programmers-believe-about-names-with-examples/
• https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/

It would be nice to stop using UserData(val lastName: String) examples.

[serras]

Thanks for bringing this up, I was not aware of these problems. This is corrected in a further commit.

[rnett]

Am I correct that with the long-term syntax (post-migration), you would not be required to specify any vals for name-based for-loop or lambda parameter destructuring? The only examples are for local vars so I wanted to double check, since IMO using val there is rather awkward (it's fine as an intermediate step, IMO).

[serras]

That is correct. I'll clarify this in the KEEP.

[rnett]

For the migration plan, it would be very nice to have the ability to start with Phase 3 for new projects and projects that have been manually migrated. E.g. with an experimental compiler arg.

[gildor]

I agree, I would like to have this mode from the beginning to migrate the project immediately and do not wait for migration.
Could it be a "progressive mode" option?

[serras]

Yes, that's the idea. I'll clarify this in the KEEP.

[kyay10]

I want to push back a tiny bit on the [] syntax. Collection literals are specifically designed for collections, and intentionally prevent Pair, Triple etc from using their syntax. Further, the ideas for syntax for Pair, Point don't involve [] at all. I understand that [] communicates that it is positional, but it's still inconsistent a bit

[kyay10]

However though, if tuples would use the [] syntax, then this is reasonable. It just feels premature when tuple syntax is not decided on yet to make this the tuple destructuring syntax (positional destructuring only makes sense for tuple-like things anyways)

[serras]

To be honest, we're not even sure tuple syntax is even needed at all. Many languages are shying away from them and introducing some sort of simplified data class syntax, because in many cases the naming is quite important.

In any case, of all the available options, I think [] is the most sensible one for positional matching. Furthermore, Kotlin has the ability to match fewer than the amount of elements in a tuple, which to me speaks also a lot about "sequencing".

val [x, y] = Triple(1, 2, 3)

[klitoskyriacou]

A question about what happens as the number of deconstructed properties goes down to 1, especially after phase 3: would you be able to write val (age) = person?

On the one hand, it seems silly to allow this when val age = person.age is more readable.

On the other hand, disallowing it would be an arbitrary restriction:

• val (name, age, hobbies) = person is allowed
• val (age, hobbies) = person is allowed
• val (age) = person is not allowed?

[bitspittle]

Just curious, would you then argue for val () = person to work? If so, fair enough! If not, it means we're ok drawing the line somewhere and that line being 2 or more seems reasonable to me?

[klitoskyriacou]

@bitspittle good question. The current behaviour allows val (a) = Pair('x', 'y'), meaning val a = Pair('x', 'y').first(). So I guess I've answered my own question: yes, you'll be able to write val (age) = person. In the case of val (a) = Pair('x', 'y'), this will have to be written as val (first) = Pair('x', 'y') or val (a = first) = Pair('x', 'y').

[bitspittle]

Pretty interesting, thanks!

[serras]

The current grammar requires at least one element inside the parentheses. But as currently, you can have just one element inside it:

(val a = first) = Pair('x', 'y')

[Hullaballoonatic]

I think that’s completely fine. It’s pretty common in js to do that too, so already intuitive to many coders

[zarechenskiy]

Quick update for transparency: we’ve been working on the implementation of name-based destructuring in both the compiler and the IDE. The frontend part is nearly complete (KT-79274), but we still need some time to polish the IDE experience. The feature is not yet publicly available, not even an experimental version.

[sir-wabbit]

I feel like the current KEEP is not taking into consideration future extensions and feature interactions. I read the rationale and it all makes sense in isolation but how exactly would nested destructuring work? How would this integrate with when expressions? The rationale omits both (see the EDIT below).

Stuff like (val (val a = b) = c) = d would quickly get unreadable IMHO (the order of names and patterns is right to left and there are no constructor names). Would something like (val Foo(val b = c) = d) = foo ever be supported?

And would something similar to this ever work?

when (f()) {
    is Data.Foo(val x) -> funcRequiresInt(x)
    is Data.Baz(val z0 @ true = bazField1,  val z1 = bazField2) -> funcRequiresInt(z1)
}

Would it not make sense to plan to unify the syntax for both irrefutable (destructuring assignment) and refutable patterns (pattern matching)?

EDIT:

I realized that the document mentions pattern matching at the very end, but that still doesn't address my concern for pattern matching beyond single level. is Data.Foo(val x) is fine, but what if I want to match on the value of x? Or what if I want to destructure deeper?

EDIT 2:

Purely hypothetically:

{ bazField1 = var z0; bazField2 = { var z1 } } = f() // assuming this is irrefutable

Baz { bazField1 = var z0; bazField2 = Wozzle { var z1 } } = f() // assuming this is irrefutable

when (f()) {
  is Baz { bazField1 = true; bazField2 = Wozzle { var z1 } } -> ...
  // or
  is Baz(bazField1 = true, bazField2 = Wozzle(var z1) } -> ...
}

this supports var/val, has readable order (names before pattern), unifies syntax, allows for nested destructuring.

Comparatively, the current proposal:

(var z0 = bazField1, (var z1) = bazField2) = f() // ???

// Hypothetically
Baz(var z0 = bazField1, Wozzle(var z1) = bazField2) = f()

when (f()) {
  is Baz(true = bazField1, Wozzle(var z1) = bazField2 } -> ... // ???
}

EDIT 3:

I suppose the conclusion one can draw from this is that nested destructuring or even matching on constant values (without a guard) is not a priority and will not be a priority in the future? 🥲

[serras]

General note: when writing KEEPs we try to walk a thin line between ensuring that we can extend the design in the future, but not stating already how the future should look like. In this case, we actively didn't want to mention anything about pattern matching because we actually don't know.

The current proposal with val "inside" comes in fact from the desire of not blocking nested destructuring or pattern matching. First of all, in kotlin you need a "marker" to distringuish whether you want to bind a new variable to the value in that position, or compare against an already existing value, hence the need for val. But if you have another pattern in the left-hand side, the need for val disappears! As a result, you would be able to write something along the lines of:

fun foo(x: Pair<Pair<Int, String>, Boolean>) {
  ((val n = first, val s = second) = first, val b = second) = x
}

During the design phase we have also discuss the potential of mixing is checks and destructuring in a single line. But we want to first ensure that the name-based destructuring model is OK, and then we can think about making everything shorter. At this point, the difference between these two is quite small

// possible with NBD
when (x) {
  is Foo -> {
    (val bar, val moo = bax) = x
    ...
  }
}

// all in one line
when (x) {
  is Foo (val bar, val moo = bax) -> {
    ...
  }
}

[serras]

even matching on constant values

What exactly do you want to do here? Right now Kotlin allows matching against a completely constant value (since they are checked for equality), so I guess you mean about just partially checking some contents.

My point of view here (which I guess you may not agree on) is that guards provide a more generic solution, since they also allow the whole range of predicates. That is, we can write the following two branches with the same syntax.

when (val x = f()) {
  is Baz if x.bazField1 -> ...
  is Woozle if predicate(x.woozleField1, x.woozleField2) -> ...
}

The place in which guards start to become cumbersome is when you work with deeply-nested data. Right now it seems that Kotlin code does not use that pattern that much, but of course this may change in the future (maybe when people use a more immutable approach to data?)