GH-37145: [Python] support boolean columns with bitsize 1 in from_dataframe

[AlenkaF]

Rationale for this change

Bit-packed booleans are currently not supported in the from_dataframe of the Dataframe Interchange Protocol.

Note: We currently represent booleans in the pyarrow implementation as uint8 which will also need to be changed in a follow-up PR (see data-apis/dataframe-api#227).

What changes are included in this PR?

This PR adds the support for bit-packed booleans when consuming a dataframe interchange object.

Are these changes tested?

Only locally, currently!

• Closes: [Python] support boolean columns with bitsize 1 in from_dataframe #37145

[jorisvandenbossche]

Implementation looks good, just needs some tests then (I assume the problem here is that we ourselves don't generate this (yet), so we can't actually test it easily?)

[AlenkaF]

Implementation looks good, just needs some tests then (I assume the problem here is that we ourselves don't generate this (yet), so we can't actually test it easily?)

Yeah 😬 I think the only other library that will use bit-packed booleans is Polars.

I guess this PR should stay draft until we generate this ourselves?

[jorisvandenbossche]

Maybe we can also merge this, if you can test it locally with polars, and then open an issue to remind ourselves to add tests for it later (or when we export such data ourselves).

Merging this sooner rather than later would be good, as otherwise it breaks polars->pyarrow conversion

[AlenkaF]

OK, I will test one more time and then ping you for merge (or I can do it, if that is OK).

I plan to create an issue for us to export bit-packed boolean data so we can discuss what would be the correct approach. I will add the note about the tests there.

[AlenkaF]

Tested the code from the original report:

We can marge.

[jorisvandenbossche]

Thanks for testing!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 161510e.

There were 3 benchmark results indicating a performance regression:

• Commit Run on ursa-i9-9960x at 2023-10-05 23:00:33Z
  • file-read (Python) with compression=uncompressed, dataset=nyctaxi_2010-01, file_type=feather, output_type=dataframe
  • file-read (Python) with compression=uncompressed, dataset=nyctaxi_2010-01, file_type=feather, output_type=table
• and 1 more (see the report linked below)

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.