dataframe values.tolist() datatype

Question

I have a dataframe like this:

This dataframe has several columns. Two are of type float: price and change, while volme and amountare of type int. I use the method df.values.tolist() change df to list and get the data:

datatmp = df.values.tolist()
print(datatmp[0])

[20160108150023.0, 11.12, -0.01, 4268.0, 4746460.0, 2.0]

The int types in df all change to float types. My question is why do int types change to the float types? How can I get the int data I want?

Answer 1

I think the pandas documentation helps:

DataFrame.values

Numpy representation of NDFrame

The dtype will be a lower-common-denominator dtype (implicit upcasting); that is to say if the dtypes (even of numeric types) are mixed, the one that accommodates all will be chosen. Use this with care if you are not dealing with the blocks.

So here apparently float is chosen to accomodate all component types. A simple method would be (however, most possibly there are more elegant solutions around, I'm not too familiar with pandas):

datatmp = map(lambda row: list(row[1:]), df.itertuples())

Here the itertuples() gives an iterator with elements of the form (rownumber, colum1_entry, colum2_entry, ...). The map takes each such tuple and applies the lambda function, which removes the first component (rownumber), and returns a list containing the components of a single row. You can also remove the list() invocation if it's ok for you to work with a list of tuples.

[Dataframe values property][1] "http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.values.html#pandas.DataFrame.values"

Answer 2

You can convert column-by-column:

by_column = [df[x].values.tolist() for x in df.columns]

This will preserve the data type of each column.

Than convert to the structure you want:

list(list(x) for x in zip(*by_column))

You can do it in one line:

list(list(x) for x in zip(*(df[x].values.tolist() for x in df.columns)))

You can check what datatypes your columns have with:

df.info()

Very likely your column amount is of type float. Do you have any NaN in this column? These are always of type float and would make the whole column float.

You can cast to int with:

df.values.astype(int).tolist()