feat: parse DataType List, ListView, LargeList, LargeListView, FixedSizeList

[dqkqd]

Which issue does this PR close?

• Part of Improve parse_data_type for List, ListView, LargeList, LargeListView, FixedSizeList, Union, Map, RunEndCoded. #8648.
  This PR only implements for list types to make review easier.

Rationale for this change

The format for DataType::List includes:

• List(Int64): list not nullable.
• List(nullable Int64): list nullable.
• List(nullable Int64, field: 'foo'): list nullable with field.
  List(nullable Int64, metadata: {"foo1": "value1"}): list with metadata.

(... The list goes on for ListView, LargeList, LargeListView, FixedSizeList)

parse_data_type cannot (or incorrectly) work on those data types listed above.

What changes are included in this PR?

• Add Token::... to support new Display format for list types introduced in Improve Display for DataType #8351
  (e.g. FixedSizeList(5 x nullable Int64, field: 'foo').
• Add fn nullable to check whether nested data type is nullable.
• Add parse_single_quoted_string and parse_list_field_name to handle field: 'foo'.

Are these changes tested?

Yes. Added round trip tests.

Are there any user-facing changes?

Yes. This is related to #8351

[dqkqd]

I cannot find a better name for x.
Using Time seems confuse with time related data types.

[alamb]

Thank you @dqkqd - this looks good to me

[dqkqd]

Thank you @alamb for reviewing.

[alamb]

Thanks again @dqkqd 🙏

[alamb]

While testing the next 57 release, I believe this PR has introduced several behavior changes / regresssions:

• Regression: FixedSlizeList data type parsing fails on 57.1.0 #8880
• Regression: Parsing List(Int64) results in nullable list in 57.0.0 and a non-nullable list in 57.1.0 #8883

I have a fix for #8880 but ran out of time to look into #8883 more carefully . Does anyone have time to help check it out to unblock the release?

[etseidl]

Does anyone have time to help check it out to unblock the release?

@alamb I gave it a stab in #8886

[alamb]

Update here is that in order to maintain backwards compatibility we changed the behavior in

• Fix regression caused by changes in Display for DataType - display (List(non-null Int64) instead of List(nullable Int64) #8890

So that non-null is printed when the column is not nullable, rather than printing nullable. This aligns more closely with SQL and it is backwards compatible with prior releases where List(Int64) meant a nullable list of Int64 elements

For example

• Instead of List(nullable Int64) --> List(Int64)
• Instead of List(Int64) --> List(non-null Int64)