Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG (string dtype): logical operation with bool and string failing #60234

Open
Tracked by #54792
jorisvandenbossche opened this issue Nov 7, 2024 · 4 comments · May be fixed by #60529
Open
Tracked by #54792

BUG (string dtype): logical operation with bool and string failing #60234

jorisvandenbossche opened this issue Nov 7, 2024 · 4 comments · May be fixed by #60529
Assignees
@ldlin1
Labels
Bug Numeric Operations Arithmetic, Comparison, and Logical operations Strings String extension data type and string data
Milestone
2.3

Comments

@jorisvandenbossche
Copy link
Member

We do allow using logical operators like | to be used with non-boolean data (at which point the non-bool series would be cast to bool, I assume). For example:

>>> ser1 = pd.Series([False, False])
>>> ser2 = pd.Series([0.0, 0.1])
>>> ser1 | ser2
0    False
1     True
dtype: bool

This also worked with strings in object dtype:

>>> ser2 = pd.Series(["", "b"], dtype=object)
>>>  ser1 | ser2
0    False
1     True
dtype: bool

but currently fails with the pyarrow-backed string dtype:

>>> pd.options.future.infer_string = True
>>> ser2 = pd.Series(["", "b"])
>>> ser1 | ser2
...

File ~/scipy/repos/pandas/pandas/core/arrays/arrow/array.py:833, in ArrowExtensionArray._logical_method(self, other, op)
    831     return self._evaluate_op_method(other, op, ARROW_BIT_WISE_FUNCS)
    832 else:
--> 833     return self._evaluate_op_method(other, op, ARROW_LOGICAL_FUNCS)

File ~/scipy/repos/pandas/pandas/core/arrays/arrow/array.py:824, in ArrowExtensionArray._evaluate_op_method(self, other, op, arrow_funcs)
    822     result = pc_func(self._pa_array, other)
    823 except pa.ArrowNotImplementedError as err:
--> 824     raise TypeError(self._op_method_error_message(other_original, op)) from err
    825 return type(self)(result)

TypeError: operation 'ror_' not supported for dtype 'str' with dtype 'bool'
@jorisvandenbossche jorisvandenbossche added Bug Numeric Operations Arithmetic, Comparison, and Logical operations Strings String extension data type and string data labels Nov 7, 2024
@jorisvandenbossche jorisvandenbossche added this to the 2.3 milestone Nov 7, 2024
@jorisvandenbossche jorisvandenbossche mentioned this issue Nov 7, 2024
Open
41 tasks
@simonjayhawkins
Copy link
Member

but currently fails with the pyarrow-backed string dtype:

also fails with the numpy backed string dtype:

>>> ser2 = pd.Series(["", "b"], dtype="string[python]")
>>> ser2
0     
1    b
dtype: string
>>> 
>>> ser2.dtype.storage
'python'
>>> 
>>> ser1 | ser2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/simon/pandas/pandas/core/ops/common.py", line 76, in new_method
    return method(self, other)
  File "/home/simon/pandas/pandas/core/arraylike.py", line 79, in __or__
    return self._logical_method(other, operator.or_)
  File "/home/simon/pandas/pandas/core/series.py", line 5881, in _logical_method
    res_values = ops.logical_op(lvalues, rvalues, op)
  File "/home/simon/pandas/pandas/core/ops/array_ops.py", line 439, in logical_op
    res_values = op(lvalues, rvalues)
  File "/home/simon/pandas/pandas/core/arrays/numpy_.py", line 193, in __array_ufunc__
    result = getattr(ufunc, method)(*inputs, **kwargs)
TypeError: unsupported operand type(s) for |: 'bool' and 'str'
>>>

@ldlin1
Copy link

ldlin1 commented Nov 22, 2024

take

@ldlin1
Copy link

ldlin1 commented Nov 23, 2024

Hi @jorisvandenbossche, the bot doesn't seem to be working for me, would it possible for you to manually assign this issue to me?

@jorisvandenbossche
Copy link
Member Author

It seems to have worked now

@ldlin1 ldlin1 linked a pull request Dec 9, 2024 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ldlin1 ldlin1

Labels
Bug Numeric Operations Arithmetic, Comparison, and Logical operations Strings String extension data type and string data
Projects
None yet
Milestone
2.3
Development

Successfully merging a pull request may close this issue.

BUG: Fix pyarrow and numpy logical bug concerning bool and string ldlin1/pandas
3 participants
@jorisvandenbossche @simonjayhawkins @ldlin1