Skip to content

More types in try_* #2179

@davidlghellin

Description

@davidlghellin

What is the problem the feature request solves?

Some data types seem to be missing in the current implementation of try_*.
I might be a bit late to the party 😅 but I didn’t find an open issue tracking this.
Comes from here #2073

Describe the potential solution

Support for more data types, similar to how it was done in Sail.
Here are some example tests from Spark showing expected behavior:

>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2015-09-30', 1) as d1,
...   try_add(DATE '2000-01-01', 366) as d2,
...   try_add(DATE '2021-01-01', 1) as d3,
...   try_add(NULL, 100) as d4
... """)
>>> df.show()
+----------+----------+----------+----+
|        d1|        d2|        d3|  d4|
+----------+----------+----------+----+
|2015-10-01|2001-01-01|2021-01-02|NULL|
+----------+----------+----------+----+

>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2015-01-31', INTERVAL 1 MONTH) as d1,
...   try_add(DATE '2020-02-29', INTERVAL 12 MONTH) as d2,
...   try_add(NULL, INTERVAL 3 MONTH) as d3
... """)
>>> df.show()
+----------+----------+----+
|        d1|        d2|  d3|
+----------+----------+----+
|2015-02-28|2021-02-28|NULL|
+----------+----------+----+

>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2000-07-31', INTERVAL -1 MONTH) as d1,
...   try_add(DATE '2021-01-31', INTERVAL -1 MONTH) as d2
... """)
>>> df.show()
+----------+----------+
|        d1|        d2|
+----------+----------+
|2000-06-30|2020-12-31|
+----------+----------+

>>> df = spark.sql("""
... SELECT
...   try_add(INTERVAL '1' YEAR, INTERVAL '2' YEAR) as result
... """)

>>> spark.conf.set("spark.sql.session.timeZone", "UTC")
>>> df = spark.sql("""
... SELECT
...   try_add(TIMESTAMP '2021-01-01 00:00:00', INTERVAL 1 DAY) as result
... """)
>>> df.show(truncate=False)
+-------------------+
|result             |
+-------------------+
|2021-01-02 00:00:00|
+-------------------+

Additional context

Implementation in Sail for reference:
lakehq/sail#635

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions