Skip to content

[Python] Cannot use sparse_union/dense_union types with pa.Table.from_pylist() #44182

Closed as duplicate

Description

@rustyconover

Describe the bug, including details regarding any error messages, version, and platform.

An exception is raised when attempting to use a sparse or dense union with pa.Table.from_pylist().

Steps to Reproduce:
Run the following Python code:

import pyarrow as pa

schema = pa.schema([
  pa.field("data", 
            pa.sparse_union([
              pa.field("age", pa.int32()), 
              pa.field("weight", pa.int32())
            ])
          )
  ])

data = [
    {"data": {"age": 20}},
]

table = pa.Table.from_pylist(data, schema=schema)

Expected Behavior:
The code should create a pa.Table using the provided schema and data.

Actual Behavior
The following exception is raised:

pyarrow.lib.ArrowNotImplementedError: sparse_union

With a traceback of:

Traceback (most recent call last):
  File "/Users/rusty/Development/flight-cloud/src/flight_cloud/example-union.py", line 7, in <module>
    table = pa.Table.from_pylist(data, schema=schema)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow/table.pxi", line 1984, in pyarrow.lib._Tabular.from_pylist
  File "pyarrow/table.pxi", line 6230, in pyarrow.lib._from_pylist
  File "pyarrow/table.pxi", line 4758, in pyarrow.lib.Table.from_arrays
  File "pyarrow/table.pxi", line 1562, in pyarrow.lib._sanitize_arrays
  File "pyarrow/array.pxi", line 398, in pyarrow.lib.asarray
  File "pyarrow/array.pxi", line 368, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 42, in pyarrow.lib._sequence_to_array
  File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: sparse_union

This occurs both when using pa.sparse_union and pa.dense_union.

PyArrow version: 18.0.0.dev

Component(s)

Python

Activity

  1. jonasdedden commented on Oct 28, 2025

    @jonasdedden
    Contributor

    Can confirm this issue in pyarrow version 22.0.0.

  2. thisisnic commented on Nov 20, 2026

    @thisisnic
    Member

    This still happens on 26.0.0: pa.array() and pa.Table.from_pylist() raise ArrowNotImplementedError for both sparse_union and dense_union types, because converting Python objects to union arrays has not been implemented. That is tracked in #19157, so I'm closing this as a duplicate of that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions