fix(array): honor struct offsets in constructors and validity masks#922
Conversation
zeroshade
left a comment
There was a problem hiding this comment.
Good catch — NewStructArrayWithNulls used child.Len() instead of length-offset (leading to OOB child slicing), and the validity masking cleared logical bit positions instead of the child's physical offset. Both are fixed correctly (offset validation in both constructors, length-offset, physical-offset-aware masking with SetNullN(UnknownNullCount)). One test nit inline; possible follow-up: validate that a non-nil parent null bitmap is long enough for offset+len.
| } | ||
|
|
||
| length := cols[0].Len() | ||
| if err := validateStructArrayOffset(offset, length); err != nil { |
There was a problem hiding this comment.
The existing offset-validation test only exercises NewStructArrayWithNulls; since this fix also guards NewStructArrayWithFieldsAndNulls, please add a mirrored invalid-offset case for it.
zeroshade
left a comment
There was a problem hiding this comment.
Writing the mask at the child's physical offset is the right fix. One offset-sensitive edge case remains in the caller — inline.
| @@ -169,17 +188,19 @@ func (a *Struct) String() string { | |||
| func (a *Struct) newStructFieldWithParentValidityMask(fieldIndex int) arrow.Array { | |||
There was a problem hiding this comment.
This masking fix is correct, but the caller in String() still guards it behind !bytes.Equal(structBitmap, v.NullBitmapBytes()) (around L171). When a sliced child's bitmap bytes happen to equal the parent's but their physical offsets differ, masking is skipped and Struct.String() can print a value where the parent struct is null. Please fold offset compatibility into that shortcut (e.g. also apply the mask when a.Offset() != v.Data().Offset()), and add a regression test where the bitmap bytes are identical but the offsets differ.
zeroshade
left a comment
There was a problem hiding this comment.
Thanks — all prior review feedback is addressed. LGTM.
Struct convenience construction could retain the full child length after applying a non-zero offset, causing child slicing to run past the end. Parent validity masking also addressed logical row positions instead of the child bitmap's physical offset.
This change validates struct offsets, uses length-offset for the constructed array, applies the existing null-bitmap validation to the names-based constructor, and masks child validity at physical bitmap positions. It also sizes replacement bitmaps for leading child offsets.
Tests: