Read XML file to Pandas DataFrame

Question

Can someone please help convert the following XML file to Pandas dataframe:

<?xml version="1.0" encoding="UTF-8" ?>
<root>
	<bathrooms type="dict">
		<n35237 type="number">1.0</n35237>
		<n32238 type="number">3.0</n32238>
		<n44699 type="number">nan</n44699>
	</bathrooms>
	<price type="dict">
		<n35237 type="number">7020000.0</n35237>
		<n32238 type="number">10000000.0</n32238>
		<n44699 type="number">4128000.0</n44699>
	</price>
	<property_id type="dict">
		<n35237 type="number">35237.0</n35237>
		<n32238 type="number">32238.0</n32238>
		<n44699 type="number">44699.0</n44699>
	</property_id>
</root>

It should look like this --

OUTPUT

This is the code I have written:-

import pandas as pd
import xml.etree.ElementTree as ET

tree = ET.parse('real_state.xml')
root = tree.getroot()

dfcols = ['property_id', 'price', 'bathrooms']
df_xml = pd.DataFrame(columns=dfcols)

for node in root:
    property_id = node.attrib.get('property_id')
    price = node.attrib.get('price')
    bathrooms = node.attrib.get('bathrooms')

    df_xml = df_xml.append(
            pd.Series([property_id, price, bathrooms], index=dfcols),
            ignore_index=True)


print(df_xml)

I am getting None everywhere, instead of the actual values. Can someone please tell how it can be fixed. Thanks!

Answer 1

I have had success using this function from the xmltodict package:

import xmltodict

xmlDict = xmltodict.parse(xmlData)
df = pd.DataFrame.from_dict(xmlDict)

What I like about this, is I can easily do some dictionary manipulation in between parsing the xml and making my df. Also, it helps to explore the data as a dict if the structure is wily.

Answer 2

if the data is simple, like this, then you can do something like:

from lxml import objectify
xml = objectify.parse('Document1.xml')
root = xml.getroot()

bathrooms = [child.text for child in root['bathrooms'].getchildren()]
price = [child.text for child in root['price'].getchildren()]
property_id = [child.text for child in root['property_id'].getchildren()]

data = [bathrooms, price, property_id]
df = pd.DataFrame(data).T
df.columns = ['bathrooms', 'price', 'property_id']

    bathrooms   price      property_id
0   1.0        7020000.0    35237.0
1   3.0        10000000.0   32238.0
2   nan        4128000.0    44699.0

if it is more complex then a loop is better. You can do something like

from lxml import objectify
xml = objectify.parse('Document1.xml')
root = xml.getroot()

data=[]
for i in range(len(root.getchildren())):
    data.append([child.text for child in root.getchildren()[i].getchildren()])

df = pd.DataFrame(data).T
df.columns = ['bathrooms', 'price', 'property_id']