Different Artificial Neural

Networks Architectures
 By Group 12-
Harsh Dadich (2021UEA4374)
Harsh Deshwal (2021UEA7003)
Prem Dev Singh (2021UEA7024)
Babloo (2021UEA7036)
Aditya Pandey (2021UEA7050)
Artificial Neural
Networks
Artificial Neural Networks (ANNs) are
computational models inspired by the
structure and function of the human
brain. They consist of interconnected
nodes, called neurons, which process and
transmit information. ANNs can be
classified into several types, each with its
unique architecture and characteristics.
Let's explore some of the most prominent
types of neural networks.
Neural Network
Architectures
Single Layer Feedforward
Architecture
Multi-layer Feedforward Architecture
Recurrent Network Architecture
Dynamic Neural Networks
Feedforward
Neural Network
The Feedforward Neural Network is the
simplest and most common type of
neural network. It consists of multiple
layers of neurons, with information
flowing only in one direction, from the
input layer to the output layer. This
architecture makes it suitable for tasks
like pattern recognition and classification.
Single Layer Feedforward
Neural Network
It is the simplest and most basic architecture of
ANN’s. It consists of only two layers- the input
layer and the output layer. The input layer
consists of ‘m’ input neurons connected to each
of the ‘n’ output neurons. The connections carry
weights w11 and so on. The input layer of the
neurons doesn’t conduct any processing – they
pass the i/p signals to the o/p neurons. The
computations are performed in the output
layer. So, though it has 2 layers of neurons, only
one layer is performing the computation.
Single Layer Feedforward Neural Network (contd.)
The net signal input to output neurons is given by:

The signal output from each

output neuron will depend on the
activation function used.
Modelling SLFFNN
In a single-layer neural network, the inputs x1, x2, x3........xm are connected to
layers of neurons thrpught the weight matrix W. The weight matrix Wmxn can
be represented as:
Characteristics of SLFFNN
1. Single Layer: SLFFNN has only one layer of neurons,
often used for simple linear classification tasks.
2. Linear Separability: It is suitable for problems with
linearly separable data.
3. Linear Activation: Neurons use a linear activation
function, typically the step function.
4. Binary Classification: Primarily used for binary
classification tasks.
5. Simple Learning: Training is achieved with a simple
learning algorithm, such as the perceptron rule.
6. Lack of Depth: SLFFNN lacks hidden layers, limiting
its ability to model complex relationships.
7. Limited Applicability: Best suited for basic problems
with linear separability, not for more complex tasks.
Advantages of SLFFNN
1. Simplicity 4. Transparency
2. Quick Training 5. Binary Classification
3. Low Computational Requirements 6. Easy Implementation

Disadvantages of SLFFNN
1. Limited Expressiveness 6. Binary Classification Only
2. Linear Separability Requiremen 7. Limited Generalization
3. Inadequate for Complex Tasks 8. Lack of Depth
4. Lack of Hidden Layers 9. Overspecialization
5. Perceptron Convergence Issues
Applications of SLFFNN
1. Basic Binary Classification: Used for simple linear binary classification
tasks like spam detection and basic anomaly detection.
2. Pattern Recognition: Applied in cases of straightforward pattern
recognition tasks such as basic optical character recognition (OCR) and
shape recognition.
3. Control Systems: Utilized in control systems where linear relationships
exist between inputs and control actions.
4. Educational and Demonstrative Purposes: Employed in educational
settings to teach the fundamentals of neural networks and their
architecture.
5. Simple Function Approximation: Useful for approximating simple
mathematical functions in basic regression problems with linear input-
output relationships.
Multi-layer Feedforward
Neural Network
The multi-layer feed-forward network is quite
similar to the single-layer feed-forward
network, except for the fact that there are one
or more intermediate layers of neurons
between the input and output layer. Hence,
the network is termed as multi-layer. Each of
the layers may have a varying number of
neurons. For example, the one shown in the
diagram on next slide has ‘m’ neurons in the
input layer and ‘r’ neurons in the output layer
and there is only one hidden layer with ‘n’
neurons.
Multi-layer Feedforward Neural Network (contd.)

For the kth hidden layer neuron. The net signal input to the neuron in the
output layer is given by:
Characteristics of MLFFNN
1. Multiple Layers: Comprises input, hidden, and output layers for
processing data.
2. Non-Linear Activation Functions: Uses non-linear activation functions in
hidden layers.
3. Backpropagation: Trained using backpropagation for minimizing
prediction errors.
4. Universal Function Approximator: Can approximate any continuous
function.
5. Deep Learning Capability: Can have multiple hidden layers for
hierarchical feature learning.
6. Pattern Recognition and Classification: Used for tasks like image
classification and speech recognition.
7. Generalization: Aims to make accurate predictions on unseen data.
Advantages of MLFFNN
1. Versatility 6. Generalization
2. Non-Linear Mapping 7.Scalability
3. Feature Learning 8.Parallel Processing
4. Universal Function Approximators 9.Wide Adoption
5. Deep Learning Capability 10.Transfer Learning

Disadvantages of MLFFNN
1. Complexity 6. Hyperparameter tuning
2. Overfitting 7. Vanishing and Exploding Gradients
3. Large datasets 8. Data Preprocessing
4. Computationally Intensive 9. Not always suitable
5. Lack of Transparency
Applications of MLFFNN
1. Image Classification: MLFFNNs are commonly used in image recognition
tasks, such as classifying objects in photos.
2. Natural Language Processing: They play a role in tasks like text
classification, sentiment analysis, and language generation.
3. Speech Recognition: MLFFNNs are used for converting spoken language
into text, such as in voice assistants.
4. Financial Forecasting: They can be applied for stock market prediction
and other financial time series analysis.
5. Recommendation Systems: Used in personalized recommendation
systems, like those employed by streaming services and e-commerce
platforms.
Recurrent Neural Network
A Recurrent Neural Network is a type of artificial neural network that takes
the output of the previous step and feeds it as the input of the current step.
Unlike traditional neural networks, all the new outputs in the network are
dependent on the previously generated output. It is mostly used to generate
random sentences; for example, to predict the next word of a sentence, the
previous words are required, and there is a need to remember the previously
generated words. This issue is solved by the hidden layer of an RNN. It would
not be wrong to say that the hidden state/layer is the most important feature
of an RNN, which is used to memorize information about the sequence.
Working of RNN
The Working of RNN can be understood by the following example:

Suppose there is a deeper network with one input layer, three hidden layers
and one output layer. Then like other neural networks, each hidden layer will
have its own set of weights and biases, let's say, for hidden layer 1 the weights
and biases are (w1, b1), (w2, b2) for second hidden layer and (w3, b3) for third
hidden layer. This means that each of these layers are independent of each
other, i.e. they do not memorize the previous outputs.
Working of RNN (contd.)
Now the RNN will do the following:
• RNN converts the independent activations into dependent activations by
providing the same weights and biases to all the layers, thus reducing the
complexity of increasing parameters and memorizing each previous output
by giving each output as input to the next hidden layer.
• Hence these three layers can be joined together such that the weights and
bias of all the hidden layers is the same, into a single recurrent layer
Characteristics of RNN
1. Sequential Data Processing: RNNs are designed for processing
sequential data, making them suitable for tasks like time series analysis,
natural language processing, and speech recognition.
2. Recurrent Connections: RNNs have recurrent connections that allow
information to be passed from one step of the sequence to the next,
enabling memory of previous inputs.
3. Temporal Dependency: They capture temporal dependencies and
context by maintaining an internal hidden state that evolves as new
input is processed.
4. Vanishing and Exploding Gradients: RNNs can suffer from vanishing or
exploding gradient problems during training, which affect their ability to
capture long-term dependencies.
Advantages of RNN
1. Sequential Data Handling 6. Applications
2. Temporal Dependencies 7. Natural Language Processing
3. Variable Sequence Length 8. Short-Term Dependencies
4. Online Learning 9. Adaptive Learning
5. Stateful Memory 10. Transfer Learning

Disadvantages of RNN
1. Vanishing and Exploding Gradients 6. Training Challenges
2. Difficulty in Capturing Long-Term 7. Limited Memory Capacity
Dependencies 8. Gradient Instability
3. Computationally Intensive 9. Sensitivity to Hyperparameters
4. Sequential Processing Limitation 10. Model Complexity
5. Lack of Parallelization
Applications of RNN
1. Speech Recognition: RNNs are used to convert spoken language into
text.
2. Natural Language Processing: They power text generation, sentiment
analysis, and language translation.
3. Time Series Analysis: RNNs are applied in forecasting, stock market
prediction, and weather modeling.
4. Handwriting Recognition: Used in recognizing and converting
handwritten text into digital format.
5. Sequential Data Generation: For tasks like music composition and video
captioning.
Dynamic Neural
Network
A Dynamic Neural Network(DNN) is a type of
neural network that is designed to handle
sequential or time-dependent data. Unlike
traditional feedforward neural networks, which
process data independently and do not
maintain any internal state, dynamic neural
networks can maintain the memory of past
inputs and make use of this information when
processing new data points. Dynamic neural
networks are particularly well-suited for tasks
involving sequences, such as time series
analysis, natural language processing, speech
recognition, and video analysis.
Features of DNN
1. Recurrent Connections: Dynamic neural networks incorporate recurrent
connections, which allow information to flow from one time step to the next.
This enables them to capture and model temporal dependencies in
sequential data.
2. Temporal Memory: These networks maintain a form of memory or hidden
state that evolves as new input data is processed over time. This memory
can retain information from previous time steps.
3. Variable Sequence Length: Dynamic neural networks can handle
sequences of varying lengths, making them versatile for different types of
data.
4. Online Learning: They support online learning, meaning they can update
their internal state and predictions in real time as new data arrives.
5. Stateful Memory: The ability to retain information from previous time steps
is a form of stateful memory, which is crucial for modeling temporal
relationships in sequential data.
Artificial Neural
Networks in Practice
Artificial Neural Networks have
revolutionized various fields, including
computer vision, natural language
processing, and robotics. Their ability to
learn from data and make intelligent
decisions has led to advancements in
autonomous vehicles, medical diagnosis,
and financial prediction. By understanding
the architectural diversity of neural
networks, we can harness their potential
to drive innovation and solve complex
problems.
Conclusion
In conclusion, the architectural diversity of
Artificial Neural Networks offers a wide
range of possibilities in the field of artificial
intelligence. From feedforward networks
to recurrent networks, each type has its
unique strengths and applications. By
leveraging the power of neural networks,
we can continue to push the boundaries of
technology and unlock new opportunities
in various domains.
 Thanks for
being a patient
listener!