Can pandas handle variable-length whitespace as column delimiters

Question

I have a textfile where columns are separated by variable amounts of whitespace. Is it possible to load this file directly as a pandas dataframe without pre-processing the file? In the pandas documentation the delimiter section says that I can use a `'s*'` construct but I couldn't get this to work. ``` ## sample data head sample.txt # --- full sequence --- -------------- this domain ------------- hmm coord ali coord env coord # target name accession tlen query name accession qlen E-value score bias # of c-Evalue i-Evalue score bias from to from to from to acc description of target #------------------- ---------- ----- -------------------- ---------- ----- --------- ------ ----- --- --- --------- --------- ------ ----- ----- ----- ----- ----- ----- ----- ---- --------------------- ABC_membrane PF00664.18 275 AAF67494.2_AF170880 - 615 8e-29 100.7 11.4 1 1 3e-32 1e-28 100.4 7.9 3 273 42 313 40 315 0.95 ABC transporter transmembrane region ABC_tran PF00005.22 118 AAF67494.2_AF170880 - 615…

Accepted Answer

I think there's just a missing `\` in the docs (maybe because it was interpreted as an escape marker at some point?) It's a regexp, after all:

```
In [68]: data = read_table('sample.txt', skiprows=3, header=None, sep=r"\s*")

In [69]: data
Out[69]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 7 entries, 0 to 6
Data columns:
X.1     7  non-null values
X.2     7  non-null values
X.3     7  non-null values
X.4     7  non-null values
X.5     7  non-null values
X.6     7  non-null values
[...]
X.23    7  non-null values
X.24    7  non-null values
X.25    5  non-null values
X.26    3  non-null values
dtypes: float64(8), int64(10), object(8)
```

Because of the delimiter problem noted by @MRAB, it has some trouble with the last few columns:

```
In [73]: data.ix[:,20:]
Out[73]: 
   X.21  X.22           X.23                   X.24            X.25    X.26
0   315  0.95            ABC            transporter   transmembrane  region
1   527  0.93            ABC            transporter            None    None
2   408  0.86  RecF/RecN/SMC                      N        terminal  domain
3   575  0.85  RecF/RecN/SMC                      N        terminal  domain
4   556  0.72            AAA                 ATPase          domain    None
5   275  0.85      YceG-like                 family            None    None
6   200  0.85       Pyridine  nucleotide-disulphide  oxidoreductase    None
```

but that can be patched up at the end.

Can pandas handle variable-length whitespace as column delimiters

Solution

Problem

Related problems