diff --git a/docs/source/format/Intro.rst b/docs/source/format/Intro.rst index ce35402bff51..e98bd40e0107 100644 --- a/docs/source/format/Intro.rst +++ b/docs/source/format/Intro.rst @@ -296,6 +296,13 @@ key is the field name and the child array its values. The field (key) is saved in the schema and the values of a specific field (key) are saved in the child array. +Since child arrays are independent, Arrow does not enforce physical +consistency between the struct's validity bitmap and those of it's children. +Logically, a struct row is only valid if both the parent and the child +bitmaps have a value of 1 for that slot (a logical AND operation). +This allows for "hidden" data to exist in child arrays at null struct +positions (see ``alice`` below). + .. figure:: ./images/struct-diagram.svg :alt: Diagram is showing the difference between the struct data type presented in a Table and the data actually stored in computer diff --git a/docs/source/format/images/struct-diagram.svg b/docs/source/format/images/struct-diagram.svg index 81ff29689a85..17053414e057 100644 --- a/docs/source/format/images/struct-diagram.svg +++ b/docs/source/format/images/struct-diagram.svg @@ -1,6 +1,6 @@ - + - + - structPhysical layout - Struct Layoutcolumn 4{"name": "joe", "id": 1}{"name": null, "id": 2}null{"name": "jane", "id": null}{"name": "mark", "id": 4}column 4Values bufferValidity bitmap bufferOffset bufferValidity bitmap buffer0001110100011011children arraysValues bufferjoemarkjaneVariable-size Binary child arrayFixed-size Primitive child arrayValidity bitmap buffer000010110 3 3 3 7 111 2 _ 4 _ \ No newline at end of file + structPhysical layout - Struct Layoutcolumn 4{"name": "joe", "id": 1}{"name": null, "id": 2}null{"name": "jane", "id": null}{"name": "mark", "id": 4}column 4Values bufferValidity bitmap bufferOffset bufferValidity bitmap buffer0001110100011011children arraysValues bufferjoealicemarkjaneVariable-size Binary child arrayFixed-size Primitive child arrayValidity bitmap buffer000010110 3 3 8 12 161 2 _ 4 _