support "bool[pyarrow]" columns #229
Conversation
Unfortunately the dtype magic cannot compare all boolean types with one comparison.
|
@yehoshuadimarsky The main purpose of this PR is just one change in main.py. However, I had to change more to fix CI. I hope this is fine. I chose to fix Self-Signed-Certificate issues by using encrypt=no whenever connecting to mssql container during tests. Testing encrypted communication might also be valuable, but this is harder to configure correctly. |
| copy(col) | ||
| ) | ||
| for col, dtype in df.dtypes[ | ||
| (df.dtypes == "bool[pyarrow]") | (df.dtypes == "bool") | (df.dtypes == "boolean") |
There was a problem hiding this comment.
Is this the most ergonomic way of comparing dtypes in pandas? 🤔 haven't used pandas in a while 👀
There was a problem hiding this comment.
The "==" operator for pandas dtypes does quite some magic. That is why it cannot be replaced with "in". I was hoping that it actually knows a generic "bool" which matches to all types of booleans. However, the nullable boolean is actually called "boolean" and not "bool[python]".
borchero
left a comment
borchero
left a comment
There was a problem hiding this comment.
This seems reasonable to me, thanks :) is there a low-effort way to write a test for this?
| # attention: the `(lambda col: lambda...)(copy(col))` part looks odd but is | ||
| # needed to ensure loop iterations create lambdas working on different columns. |
There was a problem hiding this comment.
I think we can remove this comment
| # attention: the `(lambda col: lambda...)(copy(col))` part looks odd but is | |
| # needed to ensure loop iterations create lambdas working on different columns. |
2 participants
This PR is supposed to fix #228