I want to read a dbf
file of an ArcGIS shapefile and dump it into a pandas
dataframe. I am currently using the dbf package.
I have apparently been able to load the dbf
file as a Table, but have not been able to figure out how to parse it and turn it into a pandas dataframe. What is the way to do it?
This is where I am stuck at:
import dbf
thisTable = dbf.Table('C:\\Users\\myfolder\\project\\myfile.dbf')
thisTable.open(mode='read-only')
Python returns this statement as output, which I frankly don't know what to make of:
dbf.ver_2.Table('C:\\Users\\myfolder\\project\\myfile.dbf', status='read-only')
EDIT
Sample of my original dbf
:
FID Shape E N
0 Point 90089.518711 -201738.245555
1 Point 93961.324059 -200676.766517
2 Point 97836.321204 -199614.270439
... ... ... ...
Performance can be an issue. I tested a few of the libraries suggested above and elsewhere. For my test, I used a small dbf file of 17 columns and 23 records (7 kb).
Package simpledbf has a straightforward method to_dataframe(). And the practical aspect of the DBF table object of dbfread is the possibility to just iterate over it by adding it as an argument to Python's builtin function iter(), of which the result can be used to directly initialise a dataframe. In the case of pysal, I used the function dbf2DF as decribed here. The data from the other libraries I added to the dataframe by using the method shown above. However, only after retrieving the field names so that I could initialise the dataframe with the right column names first: from the fieldNames, _meta.keys and by means of the function ListFields respectively.
Probably adding records 1 by 1 is not the fastest way to obtain a filled dataframe, meaning that tests with dbfpy, dbf and arcpy would result in more favourable figures when a smarter way would be chosen to add the data to the dataframe. All the same, I hope the following table - with times in seconds - is useful:
As mmann1123 stated, you can use geopandas in order to read your dbf file. The Geopandas reads it even though it may or may not have geospatial data.
Assuming your data is only tabular data (no geographical coordinate on it), and you wish to read it and convert to a format which pandas library can read, I would suggest using geopandas.
Here is an example:
You might want to look at geopandas. It will allow you to do most important GIS operations
http://geopandas.org/data_structures.html
How about using dbfpy? Here's an example that shows how to load a dbf with 3 columns into a dataframe:
If necessary, you could find out the column names from db.fieldNames.
You should have a look at simpledbf:
This works for me with a little sample .dbf file. Hope that helps.