Support Vector Machines

$y = w^Tx+b$

y: label w: parameter to the plane

$y\in{\{-1,+1\}}$

边界线（hyperplane）：$y = w^Tx+b = 0$

plane1: $w^Tx_{1}+b=1$

plane2: $w^Tx_2+b=-1$

plane1 - plane2——maximize $\frac{2}{||w||}$

二次规划

通过二次规划，可以得出w，并且可以求出b

Linear Married

线形结合

$\Phi(q)=<q_1^2,q_2^2,\sqrt{2}q_1q_2>$

$\Phi(x)^T\Phi(y)=(x_1y_1+x_2y_2)^2$

kernel trick

$w(\alpha)=\sum{\alpha_i}-\frac{1}{2}\displaystyle\sum_{i,j}{\alpha_i\alpha_jy_iy_jk(x_i,x_j)}$

->simularity ->domain knowledge

$k=(x^Ty+c)^p$

$k=e^{-(|x-y|^2/2\sigma^2}$

...

Mercer Condition

What makes a good separating line

Maximizes Distance to nearest point === Margin

import sys
from class_vis import prettyPicture
from prep_terrain_data import makeTerrainData

import matplotlib.pyplot as plt
import copy
import numpy as np
import pylab as pl


features_train, labels_train, features_test, labels_test = makeTerrainData()


########################## SVM #################################
### we handle the import statement and SVC creation for you here
from sklearn.svm import SVC
clf = SVC(kernel="linear")


#### now your job is to fit the classifier
#### using the training features/labels, and to
#### make a set of predictions on the test data
clf.fit(features_train, labels_train)


#### store your predictions in a list named pred
pred = clf.predict(features_test)




from sklearn.metrics import accuracy_score
acc = accuracy_score(pred, labels_test)

def submitAccuracy():
    return acc

kernel, gamma and C

With larger C, we can have more trainning points correct

如果数据量很大，噪音很多，不适合用SVM