How much time does it take to train a SVM classifier?
machine-learning, python, scikit-learn, svm
Solution
SVM training can be arbitrarily long; this depends on dozens of parameters:
- `C` parameter - the greater the missclassification penalty, the slower the process
- Kernel - the more complicated the kernel, the slower the process (rbf is the most complex from the predefined ones)
- Data size/dimensionality - again, the same rule
In general, the basic SMO algorithm is O(n3), so in case of 30,000 datapoints it has to run a number of operations proportional to 2,700,000,000,000, which is a really huge number.
What are your options?
- Change the kernel to the linear one, 784 features is quite a lot, rbf can be redundant
- Reduce dimensionality of features (PCA?)
- Lower the `C` parameter
- Train the model on a subset of your data to find good parameters and then train the whole one on some cluster / supercomputer.
Problem
I wrote the following code and tested it on small data: ``` classif = OneVsRestClassifier(svm.SVC(kernel='rbf')) classif.fit(X, y) ``` Where `X, y` (X - 30000x784 matrix, y - 30000x1) are NumPy arrays. On small data, the algorithm works well and gives me correct results. But I started my program about 10 hours ago... And it is still in process. I want to know how long it will take, or it stuck in some way? (Laptop specs 4 GB Memory, Core i5-480M)