Skip to content

Conversation

chenjian2664
Copy link
Contributor

Previously, a null value was appended when the VARIANT block had zero positions. This caused a mismatch between key and value block sizes, leading to errors when building map types.

Description

When reading the null map value and the value type is a variant type, we will encounter:

Caused by: java.lang.IllegalArgumentException: keyBlock and valueBlock has different size: 0 1
	at io.trino.spi.block.MapBlock.validateConstructorArguments(MapBlock.java:143)
	at io.trino.spi.block.MapBlock.fromKeyValueBlock(MapBlock.java:82)
	at io.trino.spi.block.MapBlock.fromKeyValueBlock(MapBlock.java:71)
	at io.trino.spi.type.MapType.createBlockFromKeyValue(MapType.java:424)
	at io.trino.parquet.reader.ParquetReader.readMap(ParquetReader.java:571)
	at io.trino.parquet.reader.ParquetReader.readColumnChunk(ParquetReader.java:755)
	at io.trino.parquet.reader.ParquetReader.readBlock(ParquetReader.java:741)
	at io.trino.parquet.reader.ParquetReader$ParquetSourcePage.getBlock(ParquetReader.java:353)

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Delta Lake
* Fix reading `null` map with `VARIANT` as value type. ({issue}`issuenumber`)

Previously, a `null` value was appended when the `VARIANT` block had
zero positions. This caused a mismatch between key and value block
sizes, leading to errors when building map types.
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a bug where reading null map values with VARIANT as the value type would fail due to a mismatch between key and value block sizes. The fix removes problematic null value appending in the VARIANT reader when there are zero positions.

  • Removes null appending logic in VARIANT reader that caused block size mismatches
  • Updates test data to include map columns with VARIANT values for better coverage
  • Expands test cases to verify the fix handles null maps with VARIANT values correctly

Reviewed Changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 1 comment.

File Description
ParquetReader.java Removes problematic null appending logic in readVariant method
TestDeltaLakeBasic.java Adds comprehensive test coverage for null maps with VARIANT values
test_variant_null/_delta_log/*.json Updates test metadata to include map columns with VARIANT values
test_variant_null/README.md Documents updated test data structure with map column

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@wendigo wendigo merged commit 072ed20 into trinodb:master Sep 25, 2025
63 of 65 checks passed
@github-actions github-actions bot added this to the 478 milestone Sep 25, 2025
@chenjian2664 chenjian2664 deleted the fix-variant-in-map branch September 25, 2025 12:16
Comment on lines +1646 to +1647
"(4, NULL, NULL)," +
"(5, JSON '{\"a\":5}', NULL)");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indentation at L1647 is wrong.

@ebyhr
Copy link
Member

ebyhr commented Sep 26, 2025

Fix reading null map with VARIANT as value type. ({issue}issuenumber)

@chenjian2664 When fixing a query failure, please start the sentence with "Fix failure …". There are some common patterns for release note entries. Also, replace issuenumber with the PR number when there is no corresponding issue link.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector
Development

Successfully merging this pull request may close these issues.

3 participants