L1 - Image Acquisition and Pre-Processing

Department of Mechatronics
Image Acquisition and Pre-processing

MACHINE VISION AND IMAGE PROCESSING
(MTE 4075)
Year/ Semester : IVth / VII th
Name of Faculty:
Dr. Umesh Sahu
(Assistant Professor)
Manipal Institute of Technology, Manipal, India
−1−
Syllabus
Image acquisition and pre-processing: [15]
Vision system components, Image acquisition and analysis, Image formation
- Sampling & Quantization, simple operations on image, Image
enhancement: Spatial Domain and frequency domain methods, Image noise,
image restoration , morphological operations, Segmentation, image analysis,
representation of regions, Feature extraction, fundamentals of color image
processing, conversion of color spaces.

−2−
Content
▪Introduction: Vision in humans
▪Vision and Imaging Sensors/Detectors
▪Simple model of image formation
▪What is Digital Image Processing?
▪Image Acquisition Process
▪Image Sampling and Quantization
▪Key Stages in Digital Image Processing
▪Applications & Research Topics (Self Learning)
▪Task
−3−
Vision in humans
• Vision is our most powerful sense. Half of primate cerebral cortex is devoted to visual processing
• The retina is ~1,000 𝑚𝑚2. Contains 130 million photoreceptors (120 mil. rods for low light vision
and 10 mil. cones for color sampling) covering a visual field of 𝟐𝟐𝟎 × 𝟏𝟑𝟓 degrees
• Provides enormous amount of information: data-rate of ~3GBytes/s
• To match the eye resolution, we would need a 500 Megapixel camera. But in practice the acuity of
an eye is 8 Megapixels within a 18-degree field of view (5.5 mm diameter) around a region called
fovea
Manipal Institute of Technology, Manipal, India 4

−4−
Vision in humans: how we see
• The area we see in focus and in full color represents the part of the visual field that is covered by
the fovea
• The fovea is 0.35 mm in diameter, covers a visual field of 1-2 degrees, has high density of cone
cells
• Within the rest of the peripheral visual field, the image we perceive becomes more blurry (rod
cells)
How we actually see. This principle is used in

foveated rendering
Manipal Institute of Technology, Manipal, India 19
−5−
Vision and Imaging Sensors/Detectors
• Electronic devices that detect the presence of objects or colors within their fields of view and
convert this information into a visual image for display
• They usually integrate a camera, lights, and controller in a single unit
• Employed in many manufacturing operations for quality control, presence sensing,
positioning and orienting, sorting, labeling inspecting, gripping and guiding, etc
• Some sensors send output signals to control systems to direct actions on parts. Such systems
are termed “embedded vision” systems to describe a device that encompasses not only the
image acquisition components but the image processing system as well, sometimes called
a “smart” camera (color and monochrome sensors)
• Lighting options in image acquisition: light rings around their lenses, Lights can be white,
red, blue, or green, Backlighting to enhance the silhouette of a complex shape, Infrared and
ultraviolet lighting
• Lenses: wide-angle or telephoto images Machine Vision and Computer Vision
– are they both same?
https://www.thomasnet.com/articles/instruments-controls/all-about-vision-and-imaging-sensors/

−6−
Contd..
• Area sensor - to detect missing features in a machined part, such as a hole or multiple holes.
• Blemish sensors - to locate scratches on a surface or foreign matter on packaging material
• Match sensors -to compare patterns such as inscriptions to reference patterns
• Sort sensor - to check a part against multiple patterns for orienting it.
• Inspection: Line scan cameras
Application?

−7−
−8−
A Simple model of image formation
▪ The scene is illuminated by a single source.
▪ The scene reflects radiation towards the camera.
▪ The camera senses it via chemicals on film.

−9−
Light
Is Imaging
possible only in
visible light ?

− 10 −
X-Ray Visible Infrared SAR image

− 11 −
Images in different walks of life
Entertainment Medical Field
Image
Remote Sensing Forensic Field

− 12 −
Image Types

− 13 −
Image Digitization
▪ Why do we need image digitization ?
– To store an image in digital computer, since there are infinite points between two numbers and
representation of all points will take infinite space in the memory.
– Image need to be represented using finite 2-D matrix.
▪ What is image digitization?

– Image is represented in 2-D finite matrix.
– Each element of matrix represented by finite set of discrete value.
▪ How to digitize an image?

– sampler followed by quantization

− 14 −
What is Digital Image Processing?
Digital Image
— a two-dimensional function
x and y are spatial coordinates f ( x, y)
The amplitude of f is called intensity or gray level at the point (x, y)
Digital Image Processing

— process digital images by means of computer, it covers low-, mid-, and high-level processes
low-level: inputs and outputs are images
mid-level: outputs are attributes extracted from input images
high-level: an ensemble of recognition of individual objects
Pixel
— the elements of a digital image

− 15 −
Image Acquisition Process

− 16 −
Image Sampling and Quantization
Plot of Intensity
(amplitude) value
along the AB
Digitizing the
coordinate
values
Digitizing the
amplitude
values
Gonzalez_Digital Image Processing(2007)

− 17 −
Image Sampling and Quantization

− 18 −
Gonzalez_Digital Image Processing(2007)

− 19 −
Representing Digital Images
▪ The representation of an M×N numerical array as
Image element/ picture element/

pixel/ pel
 f (0, 0) f (0,1) ... f (0, N − 1) 

 f (1, 0) f (1,1) ... f (1, N − 1) 
f ( x, y ) = 
 ... ... ... ... 
 
 f ( M − 1, 0) f ( M − 1,1) ... f ( M − 1, N − 1) 

− 20 −
▪ The representation of an M×N numerical array as
 a0,0 a0,1 ... a0, N −1 

 a a1,1 ... a1, N −1  
A=  1,0
 ... ... ... ... 

 
 aM −1,0 aM −1,1 ... aM −1, N −1 

− 21 −
▪ The representation of an M×N numerical array in MATLAB
 f (1,1) f (1, 2) ... f (1, N ) 

 f (2,1) f (2, 2) ... f (2, N ) 
f ( x, y ) = 
 ... ... ... ... 
 
 f ( M ,1) f ( M , 2) ... f (M , N )

− 22 −
How are images represented in the computer?

− 23 −
Color images

− 24 −
Digital Image
Digital image = a multidimensional

array of numbers (such as intensity image)
or vectors (such as color image)
Each component in the image 10 10 16 28

 9 656 7026
56  43
37  78
called pixel associates with  32 99 70 56
15  256013 902296
54 96 67 
67
the pixel value (a single number in 3  21  54 47  42 
 2 158587 853943  92
the case of intensity images or a 54  65 65 39 
vector in the case of color images). 32 65 87  99

− 25 −
What is DIP? (cont…)
▪The continuum from image processing to computer vision can be broken up into low-,
mid- and high-level processes
Low Level Process Mid Level Process High Level Process
Input: Image Input: Image Input: Attributes

Output: Image Output: Attributes Output: Understanding
Examples: Noise removal, Examples: Object Examples: Scene
image sharpening recognition, segmentation understanding, autonomous
navigation

− 26 −
Key Stages in Digital Image Processing
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition
Problem Domain Representation

& Description
Colour Image Image
Processing Compression

− 27 −
Key Stages in Digital Image Processing: Image Acquisition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 28 −
Key Stages in Digital Image Processing: Image Enhancement
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 29 −
Key Stages in Digital Image Processing: Image Restoration
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 30 −
Key Stages in Digital Image Processing: Morphological Processing
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 31 −
Key Stages in Digital Image Processing: Segmentation
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 32 −
Key Stages in Digital Image Processing: Object Recognition
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 33 −
Key Stages in Digital Image Processing: Representation & Description
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 34 −
Key Stages in Digital Image Processing: Image Compression
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 35 −
Key Stages in Digital Image Processing: Colour Image Processing
Morphological
Image Restoration
Processing
Image
Segmentation
Enhancement
Object
Image Acquisition
Recognition

& Description
Colour Image Image

− 36 −
Applications
&
Research Topics

− 37 −
Signature Verification

− 38 −
Biometrics

− 39 −
Fingerprint Identification Research at UNR
Minutiae Matching
Delaunay Triangulation

− 40 −
Target Recognition
▪ Department of Defense (Army, Airforce, Navy)

− 41 −
Autonomous Vehicles
▪ Land, Underwater, Space

− 42 −
Traffic Monitoring

− 43 −
Face Detection/Recognition Research at UNR

− 44 −
Facial Expression Recognition

− 45 −
Hand Gesture Recognition
▪Smart Human-Computer User ▪Sign Language Recognition
Interfaces

− 46 −
Human Activity Recognition

− 47 −
Medical Applications
▪ skin cancer breast cancer

− 48 −
Morphing

− 49 −
Inserting Artificial Objects into a Scene

− 50 −
Robotic Vision
Programming a robot to see Sense the information
through camera
What you need to know

Principles of vision
capture images, camera, pre processing of an image, image transformations, identify
the shapes, colours transformation, morphology operations, camera calibration,
stereo vision, feature extraction, detection and classification
Mathematical knowledge action
linear algebra, probability
Programming skills
Program to plan
Python (OpenCV)
MATLAB (image processing and computer vision toolbox
Mathematica
Lab View
Third party APIs- sherlocks

− 51 −
Applications of Machine Vision in
Industry

− 52 −
Applications of Machine Vision in Industry
Remote monitoring
Bottle verification Medicine inspection
Food grade inspection
Coloured object separation Dimension measurement

Pick and place applications

− 53 −
Glance on these techniques
Character recognition
Remote inspection
Barcode detection
Object detection
Object tracking Agriculture machines
Speed estimation
Measure the dimension of object
Find the distance of the object from camera
Stereo vision Robotic surgery

− 54 −
Glance on these techniques
piston ring counting
Contains more than 100 rings (width of 0.29mm-0.79mm Camera- 5MP

Labour intensive Red diffused light
Sherlock software
Counts the piston rings
vision system design magazine

− 55 −
Turtlebot
Camera- 3D vision
Iben robot with Orbbec Astra stereo camera to sanitize the hospitals
Robot Operating System
Algorithms for object detection and tracking
Uses OpenCV
Sensors for obstacle avoidance
for vision applications
Can be programmed to move from location A to B
Dept of Mechatronics, MIT Manipal
− 56 −
Plant disease monitor: virus & bacteria infections potatoes Requires detection and recognition of diseases
Removes the diseased plants Actions through actuators
Deep learning approaches on hyperspectral image data

− 57 −
Vision guided drone for flood monitoring system
Pipe inspection in oil refinery
Infrared cameras for wild life behaviour


− 58 −
Logistics and warehouses
sensors gyroscope, accelerometer, encoders, laser scanners
Obstacle avoidance algorithms
Proximity sensors
Intel’s D435 USB 3.0 cameras

− 59 −
Practice Problem
Gonzalez, Rafael C., and Richard E. Woods. "Digital Image Processing,
New Jersey.", 2007. 3rd edition.
Example 2.1, 2.2, 2.3, 2.4

− 60 −
Task:
Difference between Computer
Vision and Machine Vision?
Brainstorm your own

Application.
(for Assignment)

− 61 −
Thank you
for your attention

− 62 −

L1 - Image Acquisition and Pre-Processing

Uploaded by

Copyright:

Available Formats

L1 - Image Acquisition and Pre-Processing

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

L1 - Image Acquisition and Pre-Processing

Uploaded by

Copyright:

Available Formats

Department of Mechatronics

Image Acquisition and Pre-processing

Year/ Semester : IVth / VII th

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India 4

How we actually see. This principle is used in

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Entertainment Medical Field

Remote Sensing Forensic Field

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

▪ What is image digitization?

▪ How to digitize an image?

Manipal Institute of Technology, Manipal, India

Digital Image Processing

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Gonzalez_Digital Image Processing(2007)

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

▪ The representation of an M×N numerical array as

Image element/ picture element/

 f (0, 0) f (0,1) ... f (0, N − 1) 

Manipal Institute of Technology, Manipal, India

▪ The representation of an M×N numerical array as

 a0,0 a0,1 ... a0, N −1 

 ... ... ... ... 

Manipal Institute of Technology, Manipal, India

▪ The representation of an M×N numerical array in MATLAB

 f (1,1) f (1, 2) ... f (1, N ) 

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Manipal Institute of Technology, Manipal, India

Digital image = a multidimensional

Each component in the image 10 10 16 28

Manipal Institute of Technology, Manipal, India

Input: Image Input: Image Input: Attributes

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India

Problem Domain Representation

Manipal Institute of Technology, Manipal, India