PyBrain: Loading data with numpy.loadtxt?

Question

I have some working code which correctly loads data from a csv file into a PyBrain Dataset: ``` def old_get_dataset(): reader = csv.reader(open('test.csv', 'rb')) header = reader.next() fields = dict(zip(header, range(len(header)))) print header # assume last field in csv is single target variable # and all other fields are input variables dataset = SupervisedDataSet(len(fields) - 1, 1) for row in reader: #print row[:-1] #print row[-1] dataset.addSample(row[:-1], row[-1]) return dataset ``` Now I'm trying to rewrite this code to use numpy's loadtxt function instead. I believe addSample can take numpy arrays rather than having to add the data one row at a time. Assuming my loaded numpy array is m x n dimensional, how to I pass in the first m x (n-1) set of data as the first parameter, and the last column of data as the second parameter? This is what I'm trying: ``` def get_dataset(): array = numpy.loadtxt('test.csv', delimiter=',', skiprows=1) # assume last field in csv is single targe…

Accepted Answer

After a lot of experimenting and re-reading the dataset documentation, the following runs without error:

```
def get_dataset():

array = numpy.loadtxt('test.csv', delimiter=',', skiprows=1)

# assume last field in csv is single target variable
    # and all other fields are input variables
    number_of_columns = array.shape[1]
    dataset = SupervisedDataSet(number_of_columns - 1, 1)

print array[0]
    #print array[:,:-1]
    #print array[:,-1]
    #dataset.addSample(array[:,:-1], array[:,-1])
    #dataset.addSample(array[:,:-1], array[:,-2:-1])
    dataset.setField('input', array[:,:-1])
    dataset.setField('target', array[:,-1:])

return dataset
```

I have to double check that it's doing the right thing.

PyBrain: Loading data with numpy.loadtxt?

Solution

Problem