1

Information Security Analysis and Audit

(CSE3501)
Fall Semester(2021-22)

Project Report

Accuracy Comparison Of Multiple

Network Intrusion Detection System

Team Members:
Vishnu Kalra- 19BCE0374
Trinav Rattan- 19BCE0493
Aditya Pagaria- 19BCE0348

Submitted To:
Prof. Raja S P

In
B.Tech Computer Science Engineering
 2

Abstract

A Network Intrusion Detection System (NIDS) helps system and network

directors to find network security breaches in their organisations. Identifying
anonymous and new attacks are one among the most difficult challenges. We
will be giving a comparison of multiple Network Intrusion Detection Systems
taking in account the following parameters: Accuracy, Precision, Recall, and
F-score.

Classification of Intrusion Detection System

1. Host based IDS: judge information obtained on one or multiple host systems
which incorporates contents of OS, system and applications. HIDS tends to be
additional correct and fewer false positive than network-based IDS since it
analyses the log files, and as a result, it will verify whether or not an attack
occurred.

2. Network based IDS: evaluates info obtained from the communication

between the networks, the obtained packets are analysed. Sensors capture the
network traffic packets. The matter with NIDS is that it's restricted visibility
within the host machine, and there's no effective mechanism to analyse
encrypted network traffic to discover an attack. Therefore, until now, several
researches progressed to develop effective ways for NIDS to discover attacks.

3. Vulnerability Assessment IDS: Internal network and firewall vulnerabilities

have been identified.
a. Misuse Detection Model
b. Anomaly Detection Model
 3

Evaluation of Results

Machine Learning is a huge field and consists of a range of algorithms that

might perform supervised similarly to unsupervised tasks.
For the problem statement mentioned, we've planned to use some of the
supervised Algorithms, i.e., KNN, Decision tree, Regression, Naive Bayes to
find anomalies within the network activity.

How does network intrusion work?

Attackers will take many different approaches while trying to penetrate a

system. With network intruder detection software packages, understanding what
styles of attacks are used is critically necessary for fixing effective interference.
In several cases of network intrusion, the attack involves flooding or
overloading the network, gathering information regarding the network to attack
it from a vulnerable point later, or inserting data into the network to spread and
gain access from within. It’s necessary to have hacker detection tools active, so
you'll be able to stop these vulnerabilities from getting in your system.

We were able to classify the attack varieties as:

• Denial of service attack (Dos)
• probing Attack (Probe)
• User to Root Attack (U2R)
• Remote to local Attack (R2L)

Denial of Service (DoS) attack

A Denial-of-Service (DoS) attack is an attack meant to stop a machine or
network, resulting in its accessibility to its supposed users. DoS attacks
 4

accomplish this by flooding the target with traffic, or sending it data that
triggers a crash. In either case, the DoS attack prevents legitimate users
from using the service or resource they expected.

There are 2 general ways of DoS attacks: flooding services or crashing

services.
Flood attacks occur once the system receives an excessive amount of
traffic for the
server to buffer, inflicting them to cut down and eventually stop. standard
flood attacks
include:
• Buffer overflow attacks
• ICMP flood
• SYN flood

Probing Attack
Probe-response attacks are a new threat to collective intrusion detection
systems. A probe is an attack that is deliberately crafted so its target
detects and reports it with a recognizable "fingerprint" within the report.
The attacker then uses the collective infrastructure to find out the
detector's location and defensive capabilities from this report.

User to root attack

User to root attack (u2r) is launched for getting the root’s privileges
illegally while accessing a local machine legally.

Remote to local attack

Remote to local attack (r2l) is well known to be launched by an attacker
to achieve unauthorised access to a victim machine within the entire
 5

network.
One approach for detecting each attack is to formulate each issue as a
binary classification problem by deciding whether or not to accept or
reject access requests from remote sites to local user machines or by
accepting or rejecting access as root makes an attempt.

Techniques

Evasion Techniques:
Being responsive to the techniques accessible to cyber criminals who are
attempting to breach a secure network will facilitate IT departments perceive
how IDS systems are often tricked into not missing actionable threats.

Fragmentation:
Sending fragmented parcels permits the assailant to remain under the radar,
bypassing the identification framework's capacity to identify the assault
signature.

Avoiding defaults:
A port utilised by a protocol doesn't continuously give an indication to the
protocol that’s being transported. If a hacker had modified it to use another
port,the IDS might not be ready to find the presence of a trojan.

Coordinated low-bandwidth attacks:

Coordinating a scan among various attackers, or perhaps allocating varied ports
or hosts to various attackers. This makes it troublesome for the IDS to correlate
the captured packets and deduce that a network scan is ongoing.
 6

Address spoofing/proxying:
Attackers will obscure the source of the attack by victimising poorly secured or
incorrectly organized proxy servers to bounce an attack. If the source is spoofed
and bounced by a server, it makes it very tough to find.

Pattern modification evasion:

IDS depends on pattern matching to sight attacks. By creating slight
modifications to the attack structure, detection may be avoided.

Mathematical Model

The attack classes are divided into 4 categories. These 4 categories are:
● DOS
● Probe
● R2L
● U2R

The following models will be used to compare the intrusion detection systems:
● Naive Bayes
● Decision Trees
● KNN
● Logistic Regression
 7

Performance Evaluation

Accuracy: The accuracy of a machine learning model is a metric for

determining which algorithm is the strongest at recognising correlations
between variables in a dataset based on the input, or training, data.

Precision and recall: Precision and recall are data retrieval performance
measurements that refer to a collection, corpus, or sample space.

F1-score: F1 Score is the 2*((precision*recall)/(precision+recall)). The F Score

or F Measure is another name for it.It is a measurement of a test's efficiency in
statistical analysis of binary categorization.

Literature Survey

1. Intrusion detection system: A comprehensive review ;

Authors: Hung-Jen Liao, Chun-Hung Richard Lin, Ying-Chih Lin, Kuang-Yuan
Tung

Introduction: Intrusion, intrusion detection, intrusion detection system (IDS),

and intrusion prevention system are all defined in this study (IPS). This paper
also talk about any malicious intrusion or attack on the network vulnerabilities,
computers or information systems which may give rise to serious disasters, and
violate the computer security policies, i.e.Confidentiality, Integrity and
Availability.

Conclusion: The purpose of this study is to provide an overview of IDS

detection strategies, approaches, and technology. Each strategy has advantages
 8

and disadvantages, so we must be cautious while choosing ways. Also, several

rule-based approaches to detect unknown attacks have been
proposed.Furthermore, this paper briefly introduce two famous and open source
tools for studying IDSs

2. NetSTAT: A network-based intrusion detection system ;

Authors: Giovanni Vigna, Richard A. Kemmerer

Introduction: This paper talks about how networks have grown in both size
and importance. In particular TCP/IP networks, and most notably the
world-wide Internet, have become the main means to exchange data and carry
out transactions. The source of events for analysis in Network-oriented Intrusion
Detection Systems (NIDSs) is a distributed system made up of multiple hosts
and network links. The purpose of NIDSs is to identify network-based threats
that may span several hosts. The NetSTAT technique describes network attacks
as state transition diagrams, where states and transitions are defined in a
networked context, as shown in this study.

Conclusion: This paper presents a further application of the state transition

analysis approach to detect network-based intrusions. The approach is based on
formal models of attack scenarios and of the network itself. These two models
were created in order to figure out how to configure and locate intrusion
detection probes.

3. Network Intrusion Detection system using Fuzzy Logic;

Authors: R. Shanmugavadivu, Dr.N.Nagarajan

Introduction: Because it is theoretically impossible to build up a system

without any weaknesses, intrusion detection has arisen as a prominent subject of
 9

research. One of the most difficult challenges in intrusion detection is detecting

hidden attacks among a huge number of ordinary communication events.
Several machine learning (ML) algorithms, for instance Neural Network,
Support Vector Machine, Genetic Algorithm, Fuzzy Logic, and Data Mining
and more have been extensively employed to detect intrusion activities both
known and unknown from large quantities of complex and dynamic datasets.

Conclusion: This paper has developed an anomaly based intrusion detection

system in detecting the intrusion behaviour within a network. Using the fuzzy
inference approach, a fuzzy decision-making module was created to make the
software more efficient for attack detection.

4. Network intrusion detection;

Author: Biswanath Mukherjee, L. Todd Heberlein, and Karl N. Levitt

Introduction: The accuracy, fidelity, non-corruptibility, and believability of

information transfer between peer entities are all concerns of the data and
communications integrity service, according to this study. This service must
guarantee that the system hardware and firmware are operating properly, as well
as secure data and labels from unauthorised alteration. This paper also outlines
the number of limitations to this prevention-based approach for computer and
network security.

Conclusion: The conclusion of this study is that intrusion detection systems are
based on host audit trails and network traffic analysis, with the purpose of
detecting attacks in real time. A number of prototype intrusion detection
systems have been built, and this concept has been proven to be extremely
promising. In the future, it is expected that the current prototypes will be
developed further in order to turn them into production-quality systems.
 10

5. A survey of Intrusion Detection System;

Author: Teresa F. Lunt

Introduction: The timely detection of unwanted entrants into computers and

networks is a growing concern, according to this paper. It also outlines how
audit trails can establish accountability of users for their actions, and have been
viewed as the final defence, not only because of their deterrent value but
because in theory they can be pursued for suspicious events and used to provide
evidence to establish the guilt or innocence of suspected individuals. To make
audit trails useful for security purposes, automated tools are needed to analyse
the audit data so as to assist in the detection of suspicious events.

Conclusion: Departures from users' regular behaviour patterns can be used to

detect intrusions. In addition, a rule-based approach, in which rules
characterising intrusive behaviour arc constructed for evaluation against
observed user behaviour, can be used. This paper also successfully concludes
the strengths and weaknesses of all the systems outlined in the paper.

6. A survey of intrusion detection system

Author: Putra Wanda, Huang Jin Jie

Introduction: This study gave an assessment on intrusion detection systems

(IDS) in a variety of sectors, including Web applications, cloud environments,
the Internet of Things (IoT), mobile ad hoc networks (MANETs), wireless
sensor networks (WSNs), and Voice over Internet Protocol (VoIP) (VOIP). It
collects data from many sources and using alert filtering techniques to
discriminate between malicious and false alarms. Although both firewalls and
intrusion detection systems are concerned with network security, an IDS varies
 11

from a firewall in that a firewall searches outwards for intrusions in order to

prevent them. To deter intrusion, a firewall restricts access between networks.
IDS are classified based on the location of detection (network or host) and the
detection method used.

Conclusion: A computer network is made up of two parts: hardware and

software. Both of these components may come with their own set of dangers
and weaknesses. The authors of this research examined various types of
Intrusion Detection Models in various scenarios. Intrusion detection systems
(IDS) were discussed in numerous areas in this article. Some of the components
include web applications, cloud environments, the Internet of Things (IoT),
mobile ad hoc networks (MANETs), wireless sensor networks (WSNs), and
voice over internet protocol (VoIP) (VOIP). They came to the conclusion that an
IDS plays an important role in network security.

7. Diverse Malicious Attacks and security Analysis on MQTT protocol in IoT

Author: Bhanujyothi H C, Vidya J, Swasthika Jain T J, Sahana D S

Introduction: This article discusses various attack scenarios in the Message

Queuing Telemetry Protocol (MQTT) protocol, as well as the protocol's security
requirements and concerns. For bidirectional communication and remote
control, most IoT devices operate behind firewalls and use middleware or
message brokers [9]. Several protocols have been created to enable bidirectional
communication between IoT devices (D2D) and between devices and
servers/clouds (D2S). Message Queue Telemetry Transport (MQTT) has
become the most extensively used of these protocols. MQTT's low overhead
and power consumption are the main reasons for this. MQTT facilitates the
exchange of messages between clients, which are often IoT devices, smart
 12

phones, and computers, by using an Internet-facing broker server. As a result,

security concerns in the MQTT protocol must be recognised in order to defend
the IoT environment based on this protocol.

Conclusion: In IoT systems, MQTT is a frequently used application protocol.

Providing security to MQTT protocol is very important compare to all other
protocols because of its simplicity and scalability. The summary of this paper
tells that it is necessary to protect the IoT connected devices from malicious
attacks and misuse which could prevent the evolution of IoT as a secure and
reliable paradigm. Before providing security need to know about what are the
different security scenarios, This paper gives idea about different attacking
scenarios to detect different attacks that target the IoT connected devices in
MQTT protocol. Also discussed about the requirements to provide security for
MQTT protocol.

[8] Using artificial neural network in intrusion detection systems to computer

networks.
Author: L. P. Dias; J. J. F. Cerqueira; K. D. R. Assis; R. C. Almeida

Introduction: The study described in this paper presents an IDS system based
on the KDDCUP'99 dataset and an artificial neural network (ANN).
Experiments clearly show that the suggested system can achieve an overall
accuracy of 99.9% when it comes to classifying predefined kinds of intrusion
attempts, which is a very good result when compared to existing methods. The
system has the ability of classifying these networks and packets according to a
set of predetermined parameters. The method of detection used in IDS systems
can be basically classified into two distinct types: Misuse or Signature Detection
and Anomaly Detection
 13

Conclusion: The findings of this research show that the proposed neural
network can detect and classify normal or intrusive actions with a high correct
detection rate (average detection rate of 99.9%) using connection characteristics
and network packet analysis. The results suggest that IDS systems based on
anomaly are in fact a great alternative to widespread IDS systems based on
signature. With the exception of the R2L attack type, which only showed
median results in terms of mean squared error, a low index of false negatives
and false positives was still observed, resulting in an increase in network
manager productivity due to the reduction in false generated alarms analysis that
the IDS system would require.

[9] Fuzzy ARTMAP Neural Network IDS Evaluation applied for real IEEE
802.11 w data base
Author: Douglas W. F. L. Vilela; Anna Diva P. Lotufo; Carlos R. Santos

Introduction: The building of a genuine database for a wireless network using

the IEEE 802.11w security amendment is described in this paper. The Fuzzy
ARTMAP classifier IDS was evaluated using the database. The methodology
demonstrates the use of a Fuzzy ARTMAP neural network as an IDS. The same
real-world environment is described in this study, but with network devices
upgraded to accommodate IEEE 802.11w amendments and new DoS threats. As
a result, it is possible to investigate the amendment's flaws, collect network
data, and assess an artificial intelligence system as a detection approach.

Conclusion: The tagging method during database preprocessing is the paper's

contribution. Each database sample has a unique identifier based on the type of
traffic. The implementation of IEEE 802.11w as a security amendment, as well
as the collecting of intruder traffic, is another contribution. The original
 14

database created 20 subgroups in order to evaluate the IDS performance. The

Fuzzy ARTMAP IDS achieves good classification results across all subgroups.
Individual averages and overall accuracy both show this. As a result, the results
show that using the Fuzzy ARTMAP as an IDS to supplement the IEEE
802.11w security amendment is a viable option.

[10] Intrusion Detection and Prevention Systems: Technologies and Challenges

Authors: Azhagiri Mahendiran, Rajesh Appusamy, Karthik S

Introduction: This paper provides an overview of IDPS technology. Next, the

key features of each of the major classes of IDPS technology are highlighted. It
also discusses different types of IDPS security features, technology limitations,
and challenges. Intrusion Detection System Software is used to automates the
intrusion detection process. Intrusion prevention is an intrusion detection
process that seeks to stop potential incidents detected. The fundamental goals of
intrusion detection and prevention systems are to identify potential occurrences,
log information about them, try to stop them, and report to security
administrators (IDPS).

Conclusion: As confidence in computers and electronic transactions grows,

information security has become a legitimate concern for both enterprises and
computer users. Various techniques are used to protect an organization from
threats and attacks. Meanwhile, attackers are discovering new technologies and
ways to break these security guidelines. The four major types of IDPS
technology (network-based, wireless, NBA, and host-based) each offer radically
different capabilities. Each type of technology has advantages over other
technologies. For example, it detects attacks that other attacks cannot detect,
detects some attacks more accurately, and works without significantly impacting
 15

the performance of protected hosts. As a result, by utilising a wider range of

IDPS technologies, you will be able to detect and prevent malicious activities in
a more comprehensive and accurate manner.

Code snippets
16
17
18
19
 20

Graphical Outputs

Feature Importance
 21

Frequencies of the attacks

Final Output
22
 23

Conclusion
Intrusion Detection Systems (IDS) are the second layer of defence. It detects the
presence of attacks at intervals traffic that penetrates through the holes punched
into the firewall. An Intrusion Detection System (IDS) continuously observes
actions in a given environment to determine whether they are part of a potential
hostile attack or a legitimate use of the environment. The intrusion discovery
and intrusion avoidance fields are amazingly powerful, with new discoveries,
capacities, and models being made constantly. A lot of research on information
representation techniques for intrusion location information is additionally right
now being led. The study's findings revealed that data mining approaches
produce fascinating rules that are critical for intrusion detection and prevention
in the networking business. We showed however intrusion detection will have
the benefit of high-performance computing techniques. This project attempts to
address the problem of intrusion attack detection with the use of a data mining
supervised model. In conclusion, the findings of this study will help to improve
networking security standards.

Future Work
In future, it is possible to provide extensions or modifications to the proposed
clustering and classification algorithms using intelligent agents to achieve
further increased performance. Apart from the experimented combination of
data mining techniques, further combinations such as artificial intelligence, soft
computing and other deep learning algorithms can be used to improve the
detection accuracy and to reduce the rate of false negative alarm and false
positive alarm. Finally, the intrusion detection system can be extended as an
intrusion prevention system to enhance the performance of the system.
 24

References
[1] Liao, H.J., Lin, C.H.R., Lin, Y.C. and Tung, K.Y., 2013. Intrusion detection
system: A comprehensive review. Journal of Network and Computer
Applications, 36(1), pp.16-24.

[2] Vigna, G. and Kemmerer, R.A., 1999. NetSTAT: A network-based intrusion

detection system. Journal of computer security, 7(1), pp.37-71.

[3] Shanmugavadivu, R. and Nagarajan, N., 2011. Network intrusion detection

system using fuzzy logic. Indian Journal of Computer Science and Engineering
(IJCSE), 2(1), pp.101-111.

[4] Mukherjee, B., Heberlein, L.T. and Levitt, K.N., 1994. Network intrusion
detection. IEEE network, 8(3), pp.26-41.

[5] Wanda, P., 2020. A survey of intrusion detection system. International

Journal of Informatics and Computation, 1(1), pp.1-10.

[6] Bhanu Jyothi, H. C., J. Vidya, Swasthika Jain TJ, and D. S. Sahana. "Diverse
Malicious Attacks and security Analysis on MQTT protocol in IoT." (2017)

[7] Karaçay, Leyli, Erkay Savaş, and Halit Alptekin. "Intrusion Detection Over
Encrypted Network Data." The Computer Journal 63, no. 1 (2020): 604-619.
(2020)

[8] Agrawal, Gaurav, Shivank Kumar Soni, and Chetan Agrawal. "A survey on
attacks and approaches of intrusion detection systems." International Journal of
Advanced Research in Computer Science 8, no. 8 (2017): 499.
 25

[9] Vilela, Douglas WFL, Anna Diva P. Lotufo, and Carlos R. Santos. "Fuzzy
ARTMAP Neural Network IDS Evaluation applied for real IEEE 802.11 w data
base." In 2018 International Joint Conference on Neural Networks (IJCNN), pp.
1-7. IEEE, 2018.

[10] Lokesak, Brandon. "A Comparison Between Signature Based and Anomaly
Based Intrusion Detection Systems." PPT). www. iup. Edu (2019)

[11] Vasisht, Samrat. "Method, system and device for automatically configuring
a communications network." U.S. Patent 9,363,709, issued June 7, 2016.

[12] Dias, L. P., Jés de Jesus Fiais Cerqueira, Karcius DR Assis, and Raul C.
Almeida. "Using artificial neural network in intrusion detection systems to
computer networks." In 2017 9th Computer Science and Electronic Engineering
(CEEC), pp. 145-150. IEEE, 2017.

[13] Teresa F. Lunt, A survey of intrusion detection techniques, Computers &

Security, Volume 12, Issue 4, 1993, Pages 405-418, ISSN 0167-4048,
https://doi.org/10.1016/0167-4048(93)90029-5.
(https://www.sciencedirect.com/science/article/pii/0167404893900295)

[14] Mahendiran, Azhagiri & Appusamy, Rajesh & S, Karthik. (2015). Intrusion
Detection and Prevention System: Technologies and Challenges. International
Journal of Applied Engineering Research. 10. 1 - 12.