fix: OnDemandFeatureView type inference for array types

[alexmirrington]

What and Why

OnDemandFeatureView.feature_transformation.infer_features should be able to infer features from python types for all supported feast data types, for all transformation backends. This PR passes values to python_type_to_feast_value_type so that list types can be inferred correctly.

Tests

• Unit tests passing in CI for all Python versions on my fork.
• Tested against a local project by patching PandasTransformation.infer_features, e.g.

from feast.transformation.pandas_transformation import PandasTransformation

def __patched_infer_features(
    self, random_input: dict[str, list[Any]]
) -> list[Field]:
    df = pd.DataFrame.from_dict(random_input)
    output_df: pd.DataFrame = self.transform(df)
    return [
        Field(
            name=f,
            dtype=from_value_type(
                python_type_to_feast_value_type(
                    f, value=output_df[f].tolist()[0], type_name=str(dt)
                )
            ),
        )
        for f, dt in zip(output_df.columns, output_df.dtypes)
    ]

PandasTransformation.infer_features = __patched_infer_features

Note to reviewers
If you can point me to where I could add some feature repository tests to catch any regressions in the future that would be great 🙏

Fixes

Fixes #4308

[tokoko]

I think the best way for this to be tested would be in respective unit tests: test_on_demand_pandas_transformation.py, test_on_demand_python_transformation.py and test_substrait_transformation.py

[alexmirrington]

@tokoko Added some tests for lists of bools, strings, floats and ints for both python and pandas ODFVs, as well as a request source in there. LMK what you think!

[HaoXuAI]

what is this type bool_?

[alexmirrington]

It's a numpy bool type, but harder to identify since we use type(value).__name__, e.g.

>>> import pandas as pd
>>> data = pd.DataFrame({"a": [True, False]})
>>> type(data["a"].iloc[0])
<class 'numpy.bool_'>
>>> type(data["a"].iloc[0]).__name__
'bool_'

We need this mapping for boolean features in pandas transformations.

[HaoXuAI]

gotcha. add a simple doc would be better.

[HaoXuAI]

Could the list be empty and index out of range?

[alexmirrington]

It's certainly possible if the transformation written by the user doesn't return a list of length 1 - but I think under the assumption that the transformation is written correctly, then we wouldn't expect index errors. Perhaps we could add some validation checks in here with some nice error messages for users, instead of just bubbling up IndexErrors during UDF development?

[HaoXuAI]

validation sounds good to me

[alexmirrington]

@HaoXuAI FYI seeing a few rate limits for bigtable and redshift in integration tests, doesn't seem related to my changes but maybe you know who best could take a look?

[tokoko]

Do we need these type asserts? The fact that you provided a schema above should be enough, as feast itself checks that inferred and provided schemas match? wdyt?

[alexmirrington]

Happy to remove them if you think they are cluttering tests

[tokoko]

No, I suppose checking return values can also matter in some cases

[tokoko]

@alexmirrington Can you make a dummy commit to force a rerun?
@HaoXuAI Some tests still need First-time contributor approval

[tokoko]

LGTM