Checks with element_wise = True while using PolarsCheckBackend are called twice. #1707
Open
3 tasks done
Labels
bug
Something isn't working
Describe the bug
This may be an issue either within python polars or potentially rust polars, but figured I'd start here. I also posted on Stack Overflow here
I'm using pandera 0.19.3 with the polars 0.20.31 backend. While running my schema validation with cProfile, I noticed that all my custom validation checks are being called twice. The check functions when using pandas backend are only called once. I do not have any schema/container wide checks, just column/component checks.
As stated in the Stack Overflow post, I traced this down to a polars.expr.expr.py file at line 4837 calling self._pyexpr.map_batches which wraps a call to
col('uid').map_list()
(I believe a rust function), and this is where the check function is getting called twice, eventually in the polars.series.series.py at line 5518.So I am not positive if this is an issue with Pandera or with polars. But figured I'd start here.
Code Sample, a copy-pastable example
Expected behavior
Doing this with polars results in :
whereas switching it to use pandas and pandera.pandas results in:
I would expect only a single call to the check function in the above example code.
Desktop (please complete the following information):
The text was updated successfully, but these errors were encountered: