Read a Latex table into a Pandas DataFrame
latex, pandas, python
Solution
The astropy module has a LaTeX table reader. But it doesn't support all LaTeX expressions. I had to remove \toprule, \midrule, and \bottomrule. That works for me.
from astropy.table import Table
tab = Table.read('table.tex').to_pandas()
Problem
Is there any easy way to read a Latex table, as generated by the DataFrame method to_latex(), back into another DataFrame?. In particular, I'm looking for something that handles Multiindex. For instance if we have the following file 'test.out': ``` \begin{tabular}{llllrrr} \toprule & & & 1 & 2 & 3 \\ \midrule a & 1 & 1.0 & 1898 & 1681 & 1.129090 \\ & & 0.1 & 1898 & 1349 & 1.406968 \\ & 10 & 1.0 & 8965 & 5193 & 1.726362 \\ & & 0.1 & 8965 & 1669 & 5.371480 \\ & 100 & 1.0 & 47162 & 22049 & 2.138963 \\ & & 0.1 & 47162 & 5732 & 8.227844 \\ b & 1 & 1.0 & 8316 & 7200 & 1.155000 \\ & & 0.1 & 8316 & 5458 & 1.523635 \\ & 10 & 1.0 & 43727 & 24654 & 1.773627 \\ & & 0.1 & 43727 & 6945 & 6.296184 \\ & 100 & 1.0 & 284637 & 137391 & 2.071730 \\ & & 0.1 & 284637 & 26364 & 10.796427 \\ \bottomrule \end{tabular} ``` my first attempt was to read it as ``` df = pd.read_csv('test.out', sep='&', header=None, index_col=(0,1,2), skiprows=4, skipfooter=3, engine='python') ``` which does not work correctly since `read_csv()` picks up the empty fields as new levels of the Multiindex: ``` In [4]: df.index Out[4]: MultiIndex(levels=[[u' ', u'a ', u'b '], [u' ', u' 1 ', u' 10 ', u' 100 '], [0.1, 1.0]], labels=[[1, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0], [1, 0, 2, 0, 3, 0, 1, 0, 2, 0, 3, 0], [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0]], names=[0, 1, 2]) ``` Is there any way to do this?