How to add meta_data to Pandas dataframe?

metadata, pandas

Solution

This is not supported right now. See https://github.com/pydata/pandas/issues/2485. The reason is the propogation of these attributes is non-trivial. You can certainly assign data, but almost all pandas operations return a new object, where the assigned data will be lost.

Problem

I use Pandas dataframe heavily. And need to attach some data to the dataframe, for example to record the birth time of the dataframe, the additional description of the dataframe etc. I just can't find reserved fields of dataframe class to keep the data. So I change the `core\frame.py` file to add a line `_reserved_slot = {}` to solve my issue. I post the question here is just want to know is it OK to do so ? Or is there better way to attach meta-data to dataframe/column/row etc? ``` #---------------------------------------------------------------------- # DataFrame class class DataFrame(NDFrame): _auto_consolidate = True _verbose_info = True _het_axis = 1 _col_klass = Series _AXIS_NUMBERS = { 'index': 0, 'columns': 1 } _reserved_slot = {} # Add by bigbug to keep extra data for dataframe _AXIS_NAMES = dict((v, k) for k, v in _AXIS_NUMBERS.iteritems()) ``` EDIT : (Add demo msg for witingkuo's way) ``` >>> df = pd.DataFrame(np.random.randn(10,5), columns=list('ABCDEFGHIJKLMN')[0:5]) >>> df A B C D E 0 0.5890 -0.7683 -1.9752 0.7745 0.8019 1 1.1835 0.0873 0.3492 0.7749 1.1318 2 0.7476 0.4116 0.3427 -0.1355 1.8557 3 1.2738 0.7225 -0.8639 -0.7190 -0.2598 4 -0.3644 -0.4676 0.0837 0.1685 0.8199 5 0.4621 -0.2965 0.7061 -1.3920 0.6838 6 -0.4135 -0.4991 0.7277 -0.6099 1.8606 7 -1.0804 -0.3456 0.8979 0.3319 -1.1907 8 -0.3892 1.2319 -0.4735 0.8516 1.2431 9 -1.0527 0.9307 0.2740 -0.6909 0.4924 >>> df._test = 'hello' >>> df2 = df.shift(1) >>> print df2._test Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\Python\lib\site-packages\pandas\core\frame.py", line 2051, in __getattr__ (type(self).__name__, name)) AttributeError: 'DataFrame' object has no attribute '_test' >>> ```

Original source