Skip to content

Commit

Permalink
apacheGH-37145: [Python] support boolean columns with bitsize 1 in fr…
Browse files Browse the repository at this point in the history
…om_dataframe (apache#37975)

### Rationale for this change

Bit-packed booleans are currently not supported in the `from_dataframe` of the Dataframe Interchange Protocol.

Note: We currently represent booleans in the pyarrow implementation as `uint8` which will also need to be changed in a follow-up PR (see data-apis/dataframe-api#227). 

### What changes are included in this PR?

This PR adds the support for bit-packed booleans when consuming a dataframe interchange object.

### Are these changes tested?

Only locally, currently!
* Closes: apache#37145

Lead-authored-by: AlenkaF <[email protected]>
Co-authored-by: Alenka Frim <[email protected]>
Signed-off-by: AlenkaF <[email protected]>
  • Loading branch information
AlenkaF authored and dgreiss committed Feb 17, 2024
1 parent 9fdb695 commit 37fb2be
Showing 1 changed file with 9 additions and 4 deletions.
13 changes: 9 additions & 4 deletions python/pyarrow/interchange/from_dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@
DtypeKind.FLOAT: {16: pa.float16(),
32: pa.float32(),
64: pa.float64()},
DtypeKind.BOOL: {8: pa.uint8()},
DtypeKind.BOOL: {1: pa.bool_(),
8: pa.uint8()},
DtypeKind.STRING: {8: pa.string()},
}

Expand Down Expand Up @@ -232,19 +233,23 @@ def bool_column_to_array(
-------
pa.Array
"""
if not allow_copy:
buffers = col.get_buffers()
size = buffers["data"][1][1]

# If booleans are byte-packed a copy to bit-packed will be made
if size == 8 and not allow_copy:
raise RuntimeError(
"Boolean column will be casted from uint8 and a copy "
"is required which is forbidden by allow_copy=False"
)

buffers = col.get_buffers()
data_type = col.dtype
data = buffers_to_array(buffers, data_type,
col.size(),
col.describe_null,
col.offset)
data = pc.cast(data, pa.bool_())
if size == 8:
data = pc.cast(data, pa.bool_())

return data

Expand Down

0 comments on commit 37fb2be

Please sign in to comment.