How to add meta_data to Pandas dataframe?

问题:

I use Pandas dataframe heavily. And need to attach some data to the dataframe, for example to record the birth time of the dataframe, the additional description of the dataframe etc.

I just can't find reserved fields of dataframe class to keep the data.

So I change the core\frame.py file to add a line _reserved_slot = {} to solve my issue. I post the question here is just want to know is it OK to do so ? Or is there better way to attach meta-data to dataframe/column/row etc?

#----------------------------------------------------------------------
# DataFrame class


class DataFrame(NDFrame):
    _auto_consolidate = True
    _verbose_info = True
    _het_axis = 1
    _col_klass = Series

    _AXIS_NUMBERS = {
        'index': 0,
        'columns': 1
    }

    _reserved_slot = {}  # Add by bigbug to keep extra data for dataframe

    _AXIS_NAMES = dict((v, k) for k, v in _AXIS_NUMBERS.iteritems()) 

EDIT : (Add demo msg for witingkuo's way)

>>> df = pd.DataFrame(np.random.randn(10,5), columns=list('ABCDEFGHIJKLMN')[0:5])
>>> df
        A       B       C       D       E
0  0.5890 -0.7683 -1.9752  0.7745  0.8019
1  1.1835  0.0873  0.3492  0.7749  1.1318
2  0.7476  0.4116  0.3427 -0.1355  1.8557
3  1.2738  0.7225 -0.8639 -0.7190 -0.2598
4 -0.3644 -0.4676  0.0837  0.1685  0.8199
5  0.4621 -0.2965  0.7061 -1.3920  0.6838
6 -0.4135 -0.4991  0.7277 -0.6099  1.8606
7 -1.0804 -0.3456  0.8979  0.3319 -1.1907
8 -0.3892  1.2319 -0.4735  0.8516  1.2431
9 -1.0527  0.9307  0.2740 -0.6909  0.4924
>>> df._test = 'hello'
>>> df2 = df.shift(1)
>>> print df2._test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\Python\lib\site-packages\pandas\core\frame.py", line 2051, in __getattr__
    (type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute '_test'
>>> 

回答1:

This is not supported right now. See https://github.com/pydata/pandas/issues/2485. The reason is the propogation of these attributes is non-trivial. You can certainly assign data, but almost all pandas operations return a new object, where the assigned data will be lost.

回答2:

Your _reserved_slot will become a class variable. That might not work if you want to assign different value to different DataFrame. Probably you can assign what you want to the instance directly.

In [6]: import pandas as pd

In [7]: df = pd.DataFrame()

In [8]: df._test = 'hello'

In [9]: df._test
Out[9]: 'hello'

回答3:

I think a decent workaround is putting your datafame into a dictionary with your metadata as other keys. So if you have a dataframe with cashflows, like:

df = pd.DataFrame({'Amount': [-20, 15, 25, 30, 100]},index=pd.date_range(start='1/1/2018', periods=5))

You can create your dictionary with additional metadata and put the dataframe there

out = {'metadata': {'Name': 'Whatever', 'Account': 'Something else'}, 'df': df}

and then use it as out[df]