Scikit Learn SVC decision_function and predict

Question

I'm trying to understand the relationship between decision_function and predict, which are instance methods of SVC (http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html). So far I've gathered that decision function returns pairwise scores between classes. I was under the impression that predict chooses the class that maximizes its pairwise score, but I tested this out and got different results. Here's the code I was using to try and understand the relationship between the two. First I generated the pairwise score matrix, and then I printed out the class that has maximal pairwise score which was different than the class predicted by clf.predict. ``` result = clf.decision_function(vector)[0] counter = 0 num_classes = len(clf.classes_) pairwise_scores = np.zeros((num_classes, num_classes)) for r in xrange(num_classes): for j in xrange(r + 1, num_classes): pairwise_scores[r][j] = result[counter] pairwise_scores[j][r] = -result[counter] counter += 1 index = np.argmax(pairw…

Accepted Answer

I don't fully understand your code, but let's go through the example in the documentation page you referenced:

```
import numpy as np
X = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]])
y = np.array([1, 1, 2, 2])
from sklearn.svm import SVC
clf = SVC()
clf.fit(X, y) 
```

Now let's apply both the decision_function() and predict() to the samples:

```
clf.decision_function(X)
clf.predict(X)
```

The output we get is:

```
array([[-1.00052254],
       [-1.00006594],
       [ 1.00029424],
       [ 1.00029424]])
array([1, 1, 2, 2])
```

And that is easy to interpret: The decision function tells us on which side of the hyperplane generated by the classifier we are (and how far we are away from it). Based on that information, the estimator then labels the examples with the corresponding label.

Scikit Learn SVC decision_function and predict

Solution

Problem