Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic compatibility issue #1677

Open
2 tasks done
riziles opened this issue Jun 10, 2024 · 1 comment · May be fixed by #1704
Open
2 tasks done

Pydantic compatibility issue #1677

riziles opened this issue Jun 10, 2024 · 1 comment · May be fixed by #1704
Labels
bug Something isn't working

Comments

@riziles
Copy link

riziles commented Jun 10, 2024

I believe that the latest versions of Pydantic and Pandera are not fully compatible.

This relates to #1395 which was closed, but I think should still be open

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of pandera.

This code throws an error:

import pandas as pd
import pandera as pa
from pandera.typing import DataFrame, Series
import pydantic

class SimpleSchema(pa.DataFrameModel):
    str_col: Series[str] = pa.Field(unique=True)

class PydanticModel(pydantic.BaseModel):
    x: int
    df: DataFrame[SimpleSchema]

print(PydanticModel.model_json_schema())

error message:

Exception has occurred: PydanticInvalidForJsonSchema
Cannot generate a JsonSchema for core_schema.PlainValidatorFunctionSchema ({'type': 'no-info', 'function': functools.partial(<bound method DataFrame.pydantic_validate of <class 'pandera.typing.pandas.DataFrame'>>, schema_model=SimpleSchema)})

For further information visit https://errors.pydantic.dev/2.7/u/invalid-for-json-schema
  File "C:\LocalTemp\Repos\RA\RiskCalcs\scratch.py", line 18, in <module>
    print(PydanticModel.model_json_schema())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic.errors.PydanticInvalidForJsonSchema: Cannot generate a JsonSchema for core_schema.PlainValidatorFunctionSchema ({'type': 'no-info', 'function': functools.partial(<bound method DataFrame.pydantic_validate of <class 'pandera.typing.pandas.DataFrame'>>, schema_model=SimpleSchema)})

For further information visit https://errors.pydantic.dev/2.7/u/invalid-for-json-schema

I have tried various config options to get around this error to no avail.

  • OS: Windows
  • Pydantic version: 2.7.3
  • Pandera version: 0.19.3
@riziles riziles added the bug Something isn't working label Jun 10, 2024
@riziles
Copy link
Author

riziles commented Jun 10, 2024

Here is my real hacky workaround (no idea if it is right):

import pandas as pd
import pandera as pa
from pandera.typing import DataFrame as _DataFrame, Series

from pydantic_core import core_schema, CoreSchema
from pydantic import GetCoreSchemaHandler, BaseModel
from typing import TypeVar, Generic, Any

T = TypeVar("T")  

class DataFrame(_DataFrame, Generic[T]):

    @classmethod
    def __get_pydantic_core_schema__(
        cls, source_type: Any, handler: GetCoreSchemaHandler
    ) -> CoreSchema:

        schema = source_type().__orig_class__.__args__[0].to_schema()

        type_map = {
            "str": core_schema.str_schema(),
            "int64": core_schema.int_schema(),
            "float64": core_schema.float_schema(),
            "bool": core_schema.bool_schema(),
            "datetime64[ns]": core_schema.datetime_schema()
        }

        return core_schema.list_schema(
            core_schema.typed_dict_schema(
                {
                    i:core_schema.typed_dict_field(type_map[str(j.dtype)]) for i,j in schema.columns.items()
                },
            )
        )


class SimpleSchema(pa.DataFrameModel):
    str_col: Series[str]

class PydanticModel(BaseModel):
    x: int
    df: DataFrame[SimpleSchema]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant