IJRECE VOL.

7 ISSUE 2 (APRIL - JUNE 2019) ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE)

Automatic Assessment of Descriptive Answers for Online

Examination using Semantic Analysis
Vaishnavi Adhav1, Itisha Adhav2,Tanuja Bhujbal3, Sibiya S Babu4, Dr. S. N. Zaware5
DEPARTMENT OF COMPUTER ENGINEERING AISSMS Institute of Information Technology
Kennedy Road, Near R.T.O., Pune 411 001, Maharashtra (INDIA).
Abstract- Question Answering is a specialized form of rhetorical structure theory (RST) algorithm. LSA method
information retrieval. Given a collection of documents, a supports the semantic analysis of the essay, and the RST to
Question Answering system attempts to retrieve correct evaluate the writing method and the cohesion of the essay. The
answers to questions posed in natural language. Open-domain LSA method finds the similarity ratio among two texts even if
question answering requires question answering systems to be they do not include similar words. The system processes input
able to answer questions about any conceivable topic. Such essay into two phases is a training phase and testing phase.
systems cannot, therefore, rely on hand crafted domain The training phase is made up of three parts: calculating the
specific knowledge to find and extract the correct answers. average of words per essay, calculating the most ten visible
Closed-domain question answering deals with questions under words on a given topic and applying LSA algorithm. The
a specific domain and can be seen as an easier task because testing phase passes through a number of processes:1)
NLP systems can exploit domain-specific knowledge calculating LSA distance.2) calculating the number of a
frequently formalized in ontology’s. Alternatively, The vernacular.3) calculating a number of repeated sentences.4)
proposed system provides the semantic based QA evaluation calculating the length of the essay.5) calculating number of
using Artificial Neural Network (ANN). Our system also spelling mistakes.6) applying RST algorithm.7) checking
comes under closed domain QA where we are supposed for cohesion of essay related to the topic.
accessing online based subjective examination. Anirudh et al. [2] propose an automated evaluation system for
descriptive English answers that contains multiple sentences.
Keywords- Online Subjective Examination, Sentiment The system evaluates the student’s answer with an answer-key
Analysis, Machine Learning, Paraphrase, Evaluation process. for questions of professional courses. It depends on a group of
algorithms for natural language processing which are Wu and
I. INTRODUCTION Palmer, Longest Common Substring (LCS),the similarity
Question answering is a specified form of information score of LCS combined with a similarity score of Wu-Palmer
retrieval. Our work comes under closed domain question technique using the similarity matrix method. LSA uses SVD
answering. We are working on assessment of answer for on the similarity matrix that formed of both sentences. SVD
online subjective examination. Examination and evaluation are produces two vectors representing two sentences. The
part of every course module so are even in online examination, similarity between two sentences is computed using cosine
objective based examination are already available but similarity. Pure PMI-IR combines all similarity scores of word
subjective examination are in need of time as subjective pairs among sentences in one value using the similarity matrix
assessment is considered as best way of evaluation of ones method. The multi-class Logistic regression technique
subject understanding & knowledge. In our research work we combines results of all five techniques to produce a score for
have discussed two issues related to the answer method i.e. the answer.
length & paraphrasing. And have obtained a pattern extraction DimitriosAlikaniotis et. al. [3] Automatic Text Scoring Using
by creating a sequence for a given answer. Our system has a Neural Networks System introduce a model that forms word
centralized file system which includes the reference material representations by learning the extent to which specific words
as well as the model answer for questions. These are used for contribute to the text’s score. Using Long-Short Term
matching and evaluating a candidate’s answer. For every Memory networks to represent the meaning of texts, we
correct answer a confidence factor of being positive is demonstrate that a fully automated framework is able to
assigned when the required selective pattern of candidates achieve excellent results over similar approaches. In an
answer matches with the model answer. attempt to make our results more interpretable, and inspired by
recent advances in visualizing neural networks, we introduce a
II. LITARATURE SURVEY novel method for identifying the regions of the text that the
Maram et al. [1] introduces an Automatic evaluation of an model hasfound more discriminative.
essay (AEE) system which is written in Arabic. The system According to system [4] proposes an approach of evaluation of
presents a hybrid approach which integrates the LSA and online descriptive type students’ answers using Hyperspace

INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING

A UNIT OF I2OR 483 | P a g e
 IJRECE VOL. 7 ISSUE 2 (APRIL - JUNE 2019) ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE)
Analog to Language (HAL) procedure and Self-Organizing Natural Language Processing is the word segmentation, but
Map (SOM) method. To evaluate students’ answer, the student this problem solved by the Institute of Computing
writes the answer and sent as input to HAL. HAL constructs a Technology, the Chinese Lexical Analysis System
high dimensional semantic matrix from a collection of an n- (ICTCLAS). It assesses the student's answer to the standard
word vocabulary. Method for construct matrix through answer in two phases as follows:1) compute similarities
motivation a window of length "1" by the corpus through one- between two words depend on How-Net.
word increment. HAL ignores sentence boundaries, According to system [8] to compute the similarity of sentences
punctuation and converts each word to numeric vectors depending on dependency structure among words of a
expressing information on its meanings for words. Inside sentence. This phase parses the sentence by the language
window computes the distance between two words is "d", then technology platform (LTP) to find out the dependency
computes "(l-d+1)" which denotes the weight of an association structure of the sentence. The method of computing
among two words. dependency structure is finding a valid pair which is a noun,
kumaran and Sankar [5] propose a technique of an automated verb or subjective linking to the head of the sentence. Then,
system for assessing the short answers using ontology computing the sentence similarity based on dependency
mapping. Three stages of assessing the short answers are RDF structure as functions mentioned by this system.
sentence builder, ontology construction, and ontology Xia et al. [9] design automatic scoring algorithm for a
mapping. In the first stage, the system constructs the RDF subjective question. They use the idea of a one-way approach
sentence for every sentence in student answer and model degree depending on the closeness theory of fuzzy
answer after reading the model answer and student answer as mathematics. The authors are calculating the closeness of two
input in plaintext form. The system parses each sentence and fuzzy sets which are set "A" denoted by the standard answer
builds the grammatical relationships each sentence. It uses string and set "B" denoted by the student answer string. A
Stanford typed dependency parser to represent dependency fuzzy set is an ordered collection from a single character that
relationships. In the second stage, the RDF sentences are as decomposed from a string. To compute a one-way approach
input to ontology constructor to construct an ontology for degree between two fuzzy sets "A" and "B", "B" contain n
them. The authors use sequential and coordinate links to characters and one-way approach degree denoted by δ (B,
construct RDF graph for the RDF sentences. A)=m/n whereas m denotes by the effective sum number of
Raheel and Christopher [6] propose a system that provides a the set B in each element in the set A. δ (B, A) introduce B
novel approach for automated marking of short answer close to A unidirectional closeness. The introductory
questions. To compute the grade for the student's answer, algorithm provides the aim of the system.
authors introduce the architecture for the system that is Zhenming et al. [10] propose a novel web-based online
composed of three phases to address the student's answer. objective examination system for computer science education.
Three phases are 1) spell checking and correction that is This system conducts the examination and auto-marking of
implemented by an Open Source spell checker like JOrtho.2) objective questions and operating questions. The system
parsing the student's answer using the Stanford Parser. This transmits answers and questions into the bit stream after
statistical parser can be creating parses with high accuracy. encoding to ensure security and intrusion. It is the password
The parser offers the following results which are the part of protected system and camera are used to monitor the activities
speech tagged text and design dependency grammatical of students. The auto-grading system can automatically grade
relations among singular words. 3)The Third phase of the the answers, that are collected from the examination system.
processing answer is a comparison between the tagged text
with syntactical structures specified by authors in Question III. OBJECTIVES OF SYSTEM
and Answer Language. The Objective of the proposed application is as follows:
The Automatic marking system for a student's answer  To identify the correct answer possibility for all
examination of the short essay was introduced by Mohd et al. question set.
[7]. The system applied to sentences were written using the  Identify the patterns of answer and find the similarity
Malay language that requires technique to process it. The for classification
technique mentioned in which is the syntactic annotation and  Implement the NLP approach for identifying the
the dependency group to represent the Grammatical words probability.
Relations(GR) from Malay sentences.  Finally use the classification approach for label
A new automated assessment algorithm for assessing the classification.
Chinese subjective answers was proposed by Runhua et al. [8].
The algorithm called Automated Word and Sentence Scoring IV. PROPOSED METHODOLOGY
(AWSS) assesses the student answers for the level of word The proposed system will provide data preprocessing, data
and sentence. From fundamental problems of the Chinese, normalization, stop word removal, porter stemmer algorithm,

INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING

A UNIT OF I2OR 484 | P a g e
 IJRECE VOL. 7 ISSUE 2 (APRIL - JUNE 2019) ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE)
TF-IDF calculation etc. has used for features selection.This a number of ways. Finally it will shows the system accuracy
module will classify all answers base on similarity weights. with clustering result with accuracy as well as false positive
Confidence in the correctness of an answer can be increased in ratio.

Fig.1: Proposed System Architecture

Module 1: Question Answering This module will classify all answers base on similarity weights.
In this module, admin first upload the QA dataset which is Confidence in the correctness of an answer can be increased in a
technically verified according to the specific subject. The each number of ways. One way is to use a lexical resource like
question have specific marks as well as desired answer in answer WordNet (Synonyms) to verify that a candidate response was of
set. The connectivity of each question and answer has managed the correct answer type, and it will evaluate the marks
by different indicators. automatically according to the current weights (it will be 0.01 to
Module 2: User authentication and validation 0.99).
This module admin creates the multiple users and set the Module 6: Classification Results
authorization process, once user has validate by authentication Finally it will shows the system accuracy with clustering result
system, he can give the exam and add the answer for various with accuracy as well as false positive ratio.
questions.
Module 3: Subjective Examination V. SYSTEM ANALYSIS
Basically this module set the exam with time and scheduling, Algorithm1 : Stop word Removal Approach
according to that each user provide the answer for desired Input: Stop words list L[], String Data D for remove the stop
questions. words.
Module 4: Features extraction using NLP Output: Verified data D with removal all stop words.
This module will provide data preprocessing, data normalization, Step 1: Initialize the data string S[].
stop word removal, porter stemmer algorithm, and features
extraction etc has used for features selection. Once an answer has Step 2: initialize a=0,k=0
been identified, the shallow parsing performed is leveraged to
extract only the relevant word or phrase in answer to the Step 3: for each(read a to L)
question. The use of a part-of-speech tagger can help to enable
recognition of answer candidates within identified model answer. If(a.equals(L[i]))
Answer candidates can be ranked based on measures of distance
Then Remove S[k]
between keywords, numbers of keywords matched and other
similar heuristic metrics. End for
Module 5: Semantic Analysis
Step 4: add S to D.

INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING

A UNIT OF I2OR 485 | P a g e
 IJRECE VOL. 7 ISSUE 2 (APRIL - JUNE 2019) ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE)
Step 5: End Procedure Algorithm3 TF-IDF
Input : Each word from vector as Term T, All vectors
Algorithm2 Stemming Algorithm. V[i…n]
Output : TF-IDF weight for each T
Input : Word w Step 1 :Vector = {c1, c2, c3….cn}
Step 2 : Aspects available in each comment
Output : w with removing past participles as well. Step 3 : D = {cmt1, cmt2, cmt3, cmtn}
and comments available in each document
Step 1: Initialize w
Calculate the Tf score as
Step 2:Intialize all steps of Porter stemmer Step 4 :tf (t,d) = (t,d)
t=specific term
Step 3: for each (Char ch from w) d= specific document in a term is to be found.
Step 5 :idf = t  sum(d)
If(ch.count==w.length()) && (ch.equals(e))
Step 6: Return tf *idf
Remove ch from(w)
VI. RESULTS AND DISCUSSION
Step 4:if(ch.endswith(ed)) For the system performance evaluation, calculate the matrices for
accuracy. The system is executed on java 3-tier architecture
Remove ‘ed’ from(w) framework with INTEL 2.7 GHz i3 processor and 4 GB RAM
with supervised learning approach. We have tested around 100
Step 5: k=w.length() samples with of QA dataset using proposed NLP and ANN. The
below figure. 2 shows the accuracy of proposed system of
If(k (char) to k-3 .equals(tion)) experiment 1, it shows the accuracy as well as error rate of
system.
Replace w with te.

Step 6: end procedure

100 92
86 84 84
82
80

60
Accuracy
40

14 14 15 Error Rate
20 12
7
0
K. Meena [4] V. Kumar [5] AMSSAE [7] R. Li [8] Proposed ANN
Fig.2: Proposed overall system evaluation.

The figure 2 shows the system comparative performance with uses NLP approach for during the preprocessing phase and ANN
someexisting systems. Proposed system provides the accuracy for generate the similarity weight of system. According the
better than [4,5,7,8] gained weight system will assign the proportional marks to
specific answer and finally generate the class cluster for
VII. CONCLUSION individual student. Apply sentiment analysis with polarity
The proposed system used for semantic evaluation for subjective dataset is the future work of system.
answer using Artificial Neural Network (ANN).System consist
two modules admin and students, admin first upload the standard
QA dataset and student also load the answers dataset. System

INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING

A UNIT OF I2OR 486 | P a g e
 IJRECE VOL. 7 ISSUE 2 (APRIL - JUNE 2019) ISSN: 2393-9028 (PRINT) | ISSN: 2348-2281 (ONLINE)
VIII. REFERENCES
[1]. M.F. Al-Jouie, A.M. Azmi, "Automated Evaluation of School
Children Essays in Arabic," 3rd International Conference on
Arabic Computational Linguistics, 2017, vol.117, pp.19-22.
[2]. A.Kashi, S.Shastri and A. R.Deshpande, "A Score
Recommendation System Towards Automating Assessment In
Professional Courses," 2016 IEEE Eighth International Conference
on Technology forEducation,2016,pp.140-143.
[3]. Alikaniotis D, Yannakoudakis H, Rei M. Automatic text scoring
using neural networks. arXiv preprint arXiv:1606.04289. 2016 Jun
14.
[4]. K.Meena and R.Lawrance, “Evaluation of the Descriptive type
answers using Hyperspace Analog to Language and Self-
organizing Map”, Proc. IEEE International Conference on
Computational Intelligence and Computing Research, 2014,
pp.558-562.
[5]. Vkumaran and A Sankar," Towards an automated system for short-
answer assessment using ontology mapping," International Arab
Journal of e-Technology, Vol. 4, No. 1, January 2015.
[6]. R. Siddiqi and C. J. Harrison, “A systematic approach to the
automated marking of short-answer questions,” Proceedings of the
12th IEEE International Multi topic Conference (IEEE INMIC
2008),Karachi, Pakistan, pp. 329-332, 2008.
[7]. M. J. A. Aziz, F. D. Ahmad, A. A. A. Ghani, and R. Mahmod,
"Automated Marking System for Short Answer examination
(AMSSAE)," in Industrial Electronics & Applications, 2009.
ISIEA 2009. IEEE Symposium on, 2009, pp. 47-51.
[8]. R. Li, Y.Zhu and Z.Wu,"A new algorithm to the automated
assessment of the Chinese subjective answer," IEEE International
Conference on Information Technology and Applications, pp.228 –
231,Chengdu, China, 16-17 Nov. 2013.
[9]. X.Yaowen, L.Zhiping, L.Saidong and T.Guohua," ," 2nd
InternationalSymposium on Computer, Communication, Control
and Automation, Vols. 347-350, pp. 2647-2650, 2013.
[10]. Y. Zhenming, Z. Liang, and Z. Guohua, “A novel web-based
online examination system for computer science education,” in
2013 IEEE Frontiers in Education Conference (FIE), 2003, vol. 3,
pp. S3F7–10.

INTERNATIONAL JOURNAL OF RESEARCH IN ELECTRONICS AND COMPUTER ENGINEERING

A UNIT OF I2OR 487 | P a g e