Cannot load the dataset split (in streaming mode) to extract the first rows.
Error code: StreamingRowsError
Exception: CastError
Message: Couldn't cast
episode_name: string
task: string
start_timestamp_ns: int64
end_timestamp_ns: int64
duration_ns: int64
duration_sec: double
subtasks: list<item: struct<task: string, timestamp_ns: int64>>
child 0, item: struct<task: string, timestamp_ns: int64>
child 0, task: string
child 1, timestamp_ns: int64
scene: int64
color: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 48 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
serial_number: string
ir2: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 230 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
child 1, extrinsics_to_color: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
child 2, extrinsics_to_ir1: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
ir1: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 140 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
child 1, extrinsics_to_color: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
to
{'color': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}}, 'ir1': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}, 'extrinsics_to_color': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}}, 'ir2': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}, 'extrinsics_to_color': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}, 'extrinsics_to_ir1': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}}, 'serial_number': Value('string')}
because column names don't match
Traceback: Traceback (most recent call last):
File "/src/services/worker/src/worker/utils.py", line 147, in get_rows_or_raise
return get_rows(
dataset=dataset,
...<4 lines>...
column_names=column_names,
)
File "/src/libs/libcommon/src/libcommon/utils.py", line 272, in decorator
return func(*args, **kwargs)
File "/src/services/worker/src/worker/utils.py", line 127, in get_rows
rows_plus_one = list(itertools.islice(safe_iter(ds, dataset=dataset), rows_max_number + 1))
File "/src/services/worker/src/worker/utils.py", line 478, in safe_iter
yield from ds.decode(False) if ds.features else ds
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2818, in __iter__
for key, example in ex_iterable:
^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2355, in __iter__
for key, pa_table in self._iter_arrow():
~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2380, in _iter_arrow
for key, pa_table in self.ex_iterable._iter_arrow():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 536, in _iter_arrow
for key, pa_table in iterator:
^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 419, in _iter_arrow
for key, pa_table in self.generate_tables_fn(**gen_kwags):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 343, in _generate_tables
self._cast_table(pa_table, json_field_paths=json_field_paths),
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 132, in _cast_table
pa_table = table_cast(pa_table, self.info.features.arrow_schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2369, in table_cast
return cast_table_to_schema(table, schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2297, in cast_table_to_schema
raise CastError(
...<3 lines>...
)
datasets.table.CastError: Couldn't cast
episode_name: string
task: string
start_timestamp_ns: int64
end_timestamp_ns: int64
duration_ns: int64
duration_sec: double
subtasks: list<item: struct<task: string, timestamp_ns: int64>>
child 0, item: struct<task: string, timestamp_ns: int64>
child 0, task: string
child 1, timestamp_ns: int64
scene: int64
color: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 48 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
serial_number: string
ir2: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 230 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
child 1, extrinsics_to_color: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
child 2, extrinsics_to_ir1: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
ir1: struct<intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: dou (... 140 chars omitted)
child 0, intrinsics: struct<width: int64, height: int64, fx: double, fy: double, ppx: double, ppy: double, model: string, (... 28 chars omitted)
child 0, width: int64
child 1, height: int64
child 2, fx: double
child 3, fy: double
child 4, ppx: double
child 5, ppy: double
child 6, model: string
child 7, coeffs: list<item: double>
child 0, item: double
child 1, extrinsics_to_color: struct<rotation: list<item: double>, translation: list<item: double>>
child 0, rotation: list<item: double>
child 0, item: double
child 1, translation: list<item: double>
child 0, item: double
to
{'color': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}}, 'ir1': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}, 'extrinsics_to_color': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}}, 'ir2': {'intrinsics': {'width': Value('int64'), 'height': Value('int64'), 'fx': Value('float64'), 'fy': Value('float64'), 'ppx': Value('float64'), 'ppy': Value('float64'), 'model': Value('string'), 'coeffs': List(Value('float64'))}, 'extrinsics_to_color': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}, 'extrinsics_to_ir1': {'rotation': List(Value('float64')), 'translation': List(Value('float64'))}}, 'serial_number': Value('string')}
because column names don't match