Repository navigation
Heterogeneous data sometimes detect wrong columns type #20
Description
There are combinations of heterogeneus data where the wrong column types are discovered.
For example:
using JSONTables
using Tables
nonhomogenous = """
[
{"a": 1, "b": 2, "c": 3},
{"b": 4, "c": 8, "d": 5}
]
"""
JSONTables.jsontable(nonhomogenous)
You got:
JSONTables.Table{false, JSON3.Array{JSON3.Object, Base.CodeUnits{UInt8, String}, Vector{UInt64}}}([:a, :b, :c, :d], Dict{Symbol, Type}(:a => Union{Missing, Int64}, :b => Int64, :d => Int64, :c => Int64), JSON3.Object[{
"a": 1,
"b": 2,
"c": 3
}, {
"b": 4,
"c": 8,
"d": 5
}])
d type is detected as Int64 and this throws an error when building a table:
ct = Tables.columntable(jt)
ERROR: MethodError: Cannot `convert` an object of type Missing to an object of type Int64
Activity
attdona commentedon Nov 29, 2021
if I understand correctly the culprit is:
JSONTables.jl/src/JSONTables.jl
Line 65 in 272624b
This fix should works:
quinnj commentedon Nov 30, 2021
Yes, I think that's correct; mind making a PR @attdona ?
quinnj commentedon Oct 15, 2026
[comment generated via AI] Fixed by #21, released in 1.0.3. With JSONTables 1.1.0 the example gives
d = Union{Missing, Int64}[missing, 5]for both string and JSON3 input (checked on Julia 1.12.6).