You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It can sometimes be unfortunately easy to create arrays that contain null even when the field is marked non-nullable. An attempt to write one of these batches to lance will trigger a panic.
import pyarrow as pa
import lance
schema = pa.schema([pa.field("values", pa.utf8(), False)])
tab = pa.Table.from_pydict({"values": [None]}, schema=schema)
lance.write_dataset(tab, "/tmp/foo.dataset", mode='overwrite').to_table()
# thread '<unnamed>' panicked at /home/pace/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-array-50.0.0/src/array/struct_array.rs:90:46:
# called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Found unmasked nulls for non-nullable StructArray field \"min_value\"")
# note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
# Traceback (most recent call last):
# File "/tmp/repr.py", line 6, in <module>
# print(lance.write_dataset(tab, "/tmp/foo.dataset", mode='overwrite').to_table())
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/home/pace/dev/lance/python/python/lance/dataset.py", line 2347, in write_dataset
# _write_dataset(reader, uri, params)
# pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Found unmasked nulls for non-nullable StructArray field \"min_value\"")
The text was updated successfully, but these errors were encountered:
It can sometimes be unfortunately easy to create arrays that contain null even when the field is marked non-nullable. An attempt to write one of these batches to lance will trigger a panic.
The text was updated successfully, but these errors were encountered: