Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: new method Series.struct.assign to allow adding sub-fields to a struct #59014

Open
1 of 3 tasks
tswast opened this issue Jun 14, 2024 · 0 comments
Open
1 of 3 tasks
Labels
Arrow pyarrow functionality Enhancement Needs Discussion Requires discussion from core team before further action

Comments

@tswast
Copy link
Contributor

tswast commented Jun 14, 2024

Feature Type

  • Adding new functionality to pandas

  • Changing existing functionality in pandas

  • Removing existing functionality in pandas

Problem Description

When working with a Series with dtype pd.ArrowDtype(pa.struct(...)), it would be nice to have a way to add new sub-fields to the struct.

Feature Description

A new method assign on the StructAccessor with similar semantics as DataFrame.assign but instead of adding a new column to the DataFrame, it adds a new subfield to the struct and returns the struct Series.

This could use the pyarrow.compute.make_struct function to either add or replace the subfields in the existing struct array.

Alternative Solutions

__getitem__ and __setitem__ on StructAccessor could be a natural way to do this as well, but I'm not sure about adding inplace operations on an accessor object.

Additional Context

This would make struct (aka "record" / "nested") columns more useful for data read from BigQuery.

@tswast tswast added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 14, 2024
@rhshadrach rhshadrach added Needs Discussion Requires discussion from core team before further action Arrow pyarrow functionality and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Arrow pyarrow functionality Enhancement Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

No branches or pull requests

2 participants