Scribd Logo
Upload

Welcome to Scribd!

  • 4 views

    ML Practical 3

    Uploaded by

    AIMLB119Akash Tiwari
    Machine learning Practicals for students

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    ML Practical 3

    Uploaded by

    AIMLB119Akash Tiwari
    4 views5 pages
    Machine learning Practicals for students

    Original Title

    ML Practical 3.docx

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Machine learning Practicals for students

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    4 views5 pages

    ML Practical 3

    Uploaded by

    AIMLB119Akash Tiwari
    Machine learning Practicals for students

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 5

    Experiment No: 03

    Title: Implementation of Support Vector Machines (SVM) for classifying images of handwritten
    digits into their respective numerical classes (0 to 9).

    Objective: We will develop a model using Support Vector Machine which should correctly
    classify the handwritten digits from 0-9 based on the pixel values given as features. Thus, this is
    a 10-class classification problem.

    Software Requirements: Open Source Python, Programming tool like Jupyter Notebook,
    Pycharm, Spyder.

    Operating System recommended: 64-bit Open source Linux or its derivative

    Theory:

    For this problem, we use the MNIST data which is a large database of handwritten digits. The
    'pixel values' of each digit (image) comprise the features, and the actual number between 0-9 is
    the label.

    Since each image is of 28 x 28 pixels, and each pixel forms a feature, there are 784 features.
    MNIST digit recognition is a well-studied problem in the ML community, and people have
    trained numerous models (Neural Networks, SVMs, boosted trees etc.) achieving error rates as
    low as 0.23% (i.e. accuracy = 99.77%, with a convolutional neural network).

    Before the popularity of neural networks, though, models such as SVMs and boosted trees were
    the state-of-the-art in such problems.

    We'll first explore the dataset a bit, prepare it (scale etc.) and then experiment with linear and
    non-linear SVMs with various hyperparameters.

    We'll divide the analysis into the following parts:

    Data understanding and cleaning Data preparation for model building Building an SVM
    model - hyperparameter tuning, model evaluation etc.

    SVM -

    Support Vector Machine or SVM is a Supervised Learning algorithm, which is used for
    Classification and Regression problems. but it is mostly used for Classification problems in
    Machine Learning.

    An SVM separates data across a decision boundary (plane) determined by only a small subset of
    the data (feature vectors). The data subset that supports the decision boundary is called the
    support vector. And this best decision boundary is called a hyper plane. Consider the below
    diagram in which two different categories are classified using a decision boundary or hyper
    plane:

    The distance between the decision–boundary and the closest data points is called the margin.

    SVM algorithm can be used for image classification, Face detection, text categorization, etc.

    SVM can be of two types:

    ● Linear SVM: if a dataset can be classified into two classes by using a single straight line,
    then such data is linearly separable data, and the classifier is used called a Linear SVM
    classifier. and the kernel is also used “linear”

    ● Non-linear SVM: if a dataset cannot be classified by using a straight line, then such data is
    non-linear data, and the classifier used is called a Non-linear SVM classifier. and the kernel
    is used “poly” or “rbf”.

    Python Implementation of Support Vector Machine of the Image Classification problem

    Now we will implement the SVM algorithm using Python. Here we will use the digit recognizer
    data .you can download the data-set from the kaggle itself

    We used 42000 samples and 28000 samples from the training and test data sets just to reduce the
    time of computation
    Input the data
    import numpy as np
    import pandas as pd
    import matplotlib.pyplot as plttrain_df = pd.read_csv("train.csv")
    test_df = pd.read_csv("test.csv")

    Extracting Independent and dependent Variable. Now we separate label and pixel columns
    and, the label is the 1st column of the data frame.
    X = train_df.drop('label', axis=1)
    y = train_df['label']

    Model Training

    Then I have separate training and test data with 30% samples reserved for test data.
    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 100)

    Data preprocessing

    I have used StandardScaler which standardizes features by removing the mean and scaling it to
    unit variance. StandardScaler is that it will transform your data such that its distribution will have
    a mean value of 0 and a standard deviation of 1
    from sklearn.preprocessing import StandardScaler
    scaler = StandardScaler()# fit_transform use to do some calculation and then do transformation
    X_train = scaler.fit_transform(X_train)
    X_test = scaler.transform(X_test)

    Model construct

    Now the training set will be fitted to the SVM classifier and construct the model. To create the
    SVM classifier, we will import the SVC class from sklearn.svm library. Below is the code for it:
    from sklearn.svm import SVC
    rbf_model = SVC(kernel=’rbf’)
    rbf_model.fit(X_train, y_train)

    In the above code, we have used kernel=’rbf’, as here we are creating SVM for linearly
    separable data. and the Fit function is used to adjust weights according to data values

    Model Testing

    Now, we will predict the output for the test set. For this, we will create a new variable
    y_rbf_pred. Below is the code for it:
    y_rbf_pred = rbf_model.predict(X_test)

    After getting the y_rbf_pred vector, we can compare the result of y_rbf_pred and y_test to
    check the difference between the actual value and the predicted value.
    print(“Predictad Values :\n “,y_rbf_pred[10:15])
    print (“Actual Values :\n”,y_test[10:15])

    Output:-

    The accuracy of the model can be calculated using the accuracy_score() method from
    sklearn.metrics from sklearn import metrics
    acc_rbf= metrics.accuracy_score(y_test, y_rbf_pred)
    print("accuracy:","{:.2f}".format(acc_rbf*100),"%")

    Output:- accuracy: 95.66 %

    We can now plot the digits using python matplotlib.pyplot . We use the prediction list and the
    pixel values from the test list for comparison.
    for i in (np.random.randint(0,270,4)):
    two_d = (np.reshape(X_test[i], (28, 28)))
    plt.title('predicted label: {0}'. format(y_rbf_pred[i])
    plt.imshow(two_d, cmap='gray')
    plt.show()

    Let me briefly explain the second line of the code. As the pixel values are arranged in a row with
    784 columns in the data set, first we use numpy ‘reshape’ module to arrange it in 28 X 28 array

    Output:-
    Conclusion:
    Hence here we implementation of Support Vector Machines (SVM) for classifying images of
    handwritten digits into their respective numerical classes (0 to 9).

    You might also like