codehaus


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (ARROW-4135) [Python] Can't reload a pandas dataframe containing a list of datetime.time


Krisztian Szucs created ARROW-4135:
--------------------------------------

             Summary: [Python] Can't reload a pandas dataframe containing a list of datetime.time 
                 Key: ARROW-4135
                 URL: https://issues.apache.org/jira/browse/ARROW-4135
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
    Affects Versions: 0.11.1
            Reporter: Krisztian Szucs
             Fix For: 0.12.0


Ported from https://github.com/apache/arrow/issues/3223

This simple script writes a panda dataframe with a list of datetime.time. However, constructing back the pandas dataframe fails.

I initially realised that when doing a pd.read_parquet('example.parquet'), which fails with the same error.

This is using
pyarrow 0.11.1
pandas 0.23.4

{code}
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq

times = pd.to_datetime(['09:00', '09:30', '10:00', '10:30', '11:00', '11:30', '12:00']).time
df = pd.DataFrame({'Time': [times]})

table = pa.Table.from_pandas(df)
pq.write_table(table, 'example.parquet')

# works
table2 = pq.read_table('example.parquet')

# fails: ArrowNotImplementedError: Not implemented type for list in DataFrameBlock: time64[us]
df2 = pa.Table.to_pandas(table2)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)