Using prepared data for Sci-kit classification

Question

I am trying to use the Sci-kit learn python library to classify a bunch of urls for the presence of certain keywords matching a user profile. A user has name, email address ... and a url assigned to them. I have created a txt with the result of each profile data match on each link so it is in the format: ``` Name Email Address 0 1 0 =>Relavent 1 1 0 =>Relavent 0 1 1 =>Relavent 0 0 0 =>Not Relavent ``` Where the 0 or 1 signifies that the attribute was found on the page(each row is a webpage) How do i give this data to the sci-kit so it can use it to run a classifier? The examples i have seen all have data coming from a predefined sch-kit library such as digits or iris or are being generated in the format i already have. I just dont know how to use the data format i have to provide to the library The above is a toy example and i have many more features than 3

Accepted Answer

The data needed is a `numpy` array (in this case a "matrix") with the shape `(n_samples, n_features)`.

A simple way to read the csv-file to the right format by using `numpy.genfromtxt`. Also refer this thread.

Let the contents of a csv file (say `file.csv` in the current working directory) be:

```
a,b,c,target
1,1,1,0
1,0,1,0
1,1,0,1
0,0,1,1
0,1,1,0
```

To load it we do

```
data = np.genfromtxt('file.csv', skip_header=True)
```

The `skip_header` is set to `True`, to prevent reading the header column (The `a,b,c,target` line). Refer numpy's documentation for more details.

Once you load the data, you need to do some pre-processing based on your input data format. The preprocessing could be something like splitting the input and the targets (classification) or splitting the whole dataset into a training and validation set (for cross-validation).

To split the input (feature matrix) from the output (target vector) we do

```
features = data[:, :3]
targets = data[:, 3]   # The last column is identified as the target
```

For the above given CSV data, the arrays will use will look like:

```
features = array([[ 0, 1, 0],
              [ 1, 1, 0],
              [ 0, 1, 1],
              [ 0, 0, 0]])  # shape = ( 4, 3)

targets = array([ 1, 1, 1, 0])  # shape = ( 4, )
```

Now these matrices are passed to the estimator objects `fit` function. If you are using the popular svm classifier then

```
>>> from sklearn.svm import LinearSVC
>>> linear_svc_model = LinearSVC()
>>> linear_svc_model.fit(X=features, y=targets) 
```

Using prepared data for Sci-kit classification

Solution

Problem

Related problems