NN11

Neural Networks
Course Outline:
Introduction and historical background, Feedback and Network
Architectures, Learning, Perceptrons, Perceptron Convergence
Theorem, Correlation Matrix, Least Mean Square Algorithm, Back
Propagation, Multilayer Perceptrons, Cross Validation / Function
Approximation, Adapting Learning Rates, Fuzzy Learning / Radial
Basis Function, Radial Basis Function Networks, Hopfield
Networks, Simulated Annealing, Boltzmann Machines, Mean Field
Theory, Self Organizing Systems, Self-Organizing Feature Map,
Information Theoretic Models, Introduction to Genetic Algorithm,
Application of Neural Network to Image Processing and Power
System Analysis
Shashidhar Ram Joshi
Institute of Engineering
References:
1.Simon Haykin, Neural Networks: A Comprehensive Foundation, Pearson
Education Asia, 2001.
2.Bart Kosko, Neural Networks and Fuzzy System, Prentice-Hall of India Private
Limited, 2001.
3.B. Yegnanarayana, Artificial Neural Networks, Prentice-Hall of India Private
Limited, 2001.
4.Robert J. Schalkoff, Artificial Neural Networks, McGraw-Hill International
Editions, 1997.
Evaluation
1.
2.
3.
4.
External Examination:
60%
Internal Examination:
Assignment
Test
Project
Seminar Presentation
40%
10%
10%
10%
10%
Human Brain
Computers and the Brain: A Contrast
Arithmetic:
1 brain = 1/10 pocket calculator
Vision:
1 brain = 1000 super computers
Memory of arbitrary details: computer wins
Memory of real-world facts: brain wins
A computer must be programmed explicitly
The brain can learn by experiencing the world
Other Comparisons
ITEM
COMPUTER
BRAIN
Complexity
ordered structure
serial processor
Processor Speed
10,000,000 operations
per second
10^10 neuron
processors
10^4 connections
100 operations per
second
Computational
Power
one operation at a
time
1 or 2 inputs
millions of operations
at a time
thousands of inputs
Computer Operations
Traditionally computers execute a sequence of
instructions to accomplish a set task
This is a powerful technique if you know the algorithm
Its not very useful if you dont !!
There are many interesting tasks where the

algorithm is either unknown or unclear
Recognizing handwriting - Pattern recognition
Playing table tennis - Interacting with the environment
Balancing activities - Optimization
The Question
Humans find these tasks relatively simple
We learn by example
The brain is responsible for our computing power
If a machine were constructed using

the fundamental building blocks
found in the brain could it learn to
do difficult tasks ???
Definition
. . . Neural nets are basically mathematical
models of information processing . . .
. . . (neural nets) refer to machines that have a
structure that, at some level, reflects what is
known of the structure of the brain . . .
A neural network is a massively parallel
distributed processor . . .
Neural Net Concept

Artificial Neural Systems are called:
neurocomputers
neural networks
parallel distributed processors
connectionists systems
Basic Philosophy
large number of simple neuron-like processors
which execute global or distributed computation
Neural Nets vs Standard

Computing
NN are different from
ParallelProcessors-ifyoulookatone
processorinatypicalparallelsystem,its
workmakessense
ContemporaryAInoalgorithms
Processing Comparisons
INPUT
(Type)
(Procedural)
Data Input
1, 2, 3, etc.
(Logical)
Facts
(A is True)
(Statistical)
Patterns
OUTPUT
Conventional Processing
Procedures applied to data
Data Output
Expert/Knowledge-Based System
Inference Engine &

Knowledge (Rules)
Decision
Recommend
Neural Networks
Network Algorithm Identifies

Patterns
Pattern
recognized
10
History of NN
Creation:
1890: William James - defined a neuronal process of learning
1911: Ramon y Cajal (1911) introduced the idea of a neuron (brain cell)
Promising Technology:
1943: McCulloch and Pitts - earliest mathematical models

1954: Hebb and IBM research group - earliest simulations
1958: Frank Rosenblatt - The Perceptron
Disenchantment:
1969: Minsky and Papert - perceptrons have severe limitations
Re-emergence:
1985: Multi-layer nets that use back-propagation
1986: PDP Research Group - multi-disciplined approach
11
Biological Neurons
A neuron consists of two main parts:
Axon
one per neuron
excites up to 104 other neurons
all or nothing output signal
Dendrites
1 to 104 per neuron
Theconnectionbetweenanaxonofone
neuronandthedendriteofanotheriscalled
asynapse
12
Two Neurons
13
Properties of the Brain

Architectural
80,000neuronspersquaremm
1011neurons-1015connections
Mostaxonsextendlessthan1mm(localconnections)
Operational
Highlycomplex,nonlinear,
parallelcomputer
Operatesatmillisecondspeeds
14
Interconnectedness
Each neuron may have over a thousand
synapses
Some cells in cerebral cortex may have
200,000 connections
Total number of connections in the brain
network is astronomicalgreater than
the number of particles in known universe
15
Artificial Neuron
An artificial neuron is designed to mimic
the first-order characteristics of a
biological neuron
Asetofinputs,xi
Eachinputismultiplied
bythecorresponding
weight,wi
Alloftheweightedinputs
aresummedtodetermine
theactivationlevel
Anactivationfunctionisappliedtodetermine
theneuronoutput Shashidhar Ram Joshi
16
Mathematical Structure
A neuron is a simple sum and compare
device
u=x1w1+x2w2+...xnwn-T
y=f(u)
threshold
Onepossiblefunctionforyisastepfunction:
Iftheweightedsumoftheinputsisgreater
thanthethreshold,thentheneuronfires
17
Applications of Neural Nets

Applications 1
There are a variety of applications (both

research and commercial) of neural nets
Simulations
specialsoftwarethat
implementsNNon
standardcomputers
Hardwareemulation
specialhardware
coprocessors
18
Applications 2
Diagnosis
Closest to pure concept learning and classification
Some ANNs can be post-processed to produce probabilistic diagnoses
Prediction and Monitoring

aka prognosis (sometimes forecasting)
Predict a continuation of (typically numerical) data
Decision Support Systems

aka recommender systems
Provide assistance to human subject matter experts in making decisions
Design (manufacturing, engineering)
Therapy (medicine)
Crisis management (medical, economic, military, computer security)
Control Automation
Mobile robots
Autonomic sensors and actuators
Many, Many More (ANNs for Automated Reasoning, etc.)
19
Applications 3
Other applications include:
StockPricePrediction:
Dataforpaststockpricesarelearnedbyneural
networks,andtheirtrendsforoneorafewdays
aheadarepredicted.
SignatureCheck:
Forcheckingasignatureofaperson,firstwetrain
theneuralnetworkstorecognizethatsignatureby
learningseveralwritingfeaturessuchasstrengthof
thepen,direction,andanyspecialpointofthat
sample.Afterlearning,theneuralnetworkcan
checkwhetherasignaturebelongstothesame
personornot.
20
Applications 4
The Boeing Airplane Company uses an ART1 neural network system (NIRS) for the
identification and retrieval of 2-D and 3-D
representations of engineering designs.
Avoids redesign of existing parts and tools
Production solid model data base > 55,000
entries
2-D data base > 95,000 entries
21
Applications 5
Credit Card Fraud Detection
The probability of fraud is
calculated with a neural network
with each card transaction. When
the probability of fraud reaches a
critical threshold, the case is sent
to one of the retailer's fraud
analysts for action
.
ac
22
Possible Quiz
Whatarethetwopartsofaneuron?
Whatisitthathumansdomuchbetterthanmachines?
Whatisoneapplicationofaneuralnet?
SUMMARY
IntroductiontoNeuralNets
ApplicationsofNeuralNets
23

NN11

Uploaded by

Copyright:

Available Formats

NN11

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

NN11

Uploaded by

Copyright:

Available Formats

Neural Networks

Shashidhar Ram Joshi

Shashidhar Ram Joshi

Shashidhar Ram Joshi

There are many interesting tasks where the

Shashidhar Ram Joshi

If a machine were constructed using

Shashidhar Ram Joshi

Neural Net Concept

Shashidhar Ram Joshi

Neural Nets vs Standard

Shashidhar Ram Joshi

Procedures applied to data

Inference Engine &

Network Algorithm Identifies

1943: McCulloch and Pitts - earliest mathematical models

Shashidhar Ram Joshi

Properties of the Brain

Applications of Neural Nets

There are a variety of applications (both

Prediction and Monitoring

Decision Support Systems

Many, Many More (ANNs for Automated Reasoning, etc.)

Shashidhar Ram Joshi

Shashidhar Ram Joshi

Shashidhar Ram Joshi

Shashidhar Ram Joshi

You might also like