Lecture11 Lossless

ECE472/572 - Lecture 11
Image Compression – Fundamentals

and Lossless Compression
Techniques
11/03/11
Roadmap
Preprocessing – low level
Image Image Image

Enhancement Restoration Segmentation
Image Image Image

Acquisition Compression Coding Representation
& Description
Morphological Wavelet Recognition &

Image Processing Analysis Interpretation
Knowledge Base
Roadmap
• Color image processing • Image compression
– Interpreting color – Data vs. information
– Pseudo-color IP – Entropy
• Intensity slicing – Data redundancy
• Transform in spatial domain • Coding redundancy
• Interpixel redundancy
• Transform in frequency
• Psycho-visual redundancy
domain
– Fidelity measurement
– Full-color IP
• Tonal correction • Lossless compression
• Color correction – Variable length coding
• Enhancement (I vs. RGB • Huffman coding
channels) – LZW
• Restoration (I vs. RGB – Bitplane coding
channels) – Binary image compression
• Color image edge detection • Run length coding
– Lossless predictive coding
• Lossy compression
Questions
• What is the difference between data and information?
• How to measure data redundancy?
• How many different types of data redundancy? Explain
each.
• What is entropy? What is the first/second order estimate of
entropy?
• Understand the two criteria that all coding mechanisms
should satisfy.
• Understand Huffman coding
• How do you explain the average coding length from
Huffman coding is always greater than the entropy?
• What image format uses which coding scheme?
• Understand RLC, differential coding
Data and Information
• Data and information
– Different data set can represent the same kind of
information
• Data redundancy
– Relative data redundancy
– CR: compression ratio

– n1, n2: number of information carrying units in two data
sets that represent the same information
• Data compression
– Reducing the amount of data required to represent a
given quantity of information
Data Redundancy
• Coding redundancy
• Interpixel redundancy
• Psychovisual redundancy
Coding Redundancy
• In general, coding redundancy is
present when the codes assigned to
a set of events (such as gray-level
values) have not been selected to
take full advantage of the
probabilities of the events.
• In most images, certain gray levels
are more probable than others
Coding Redundancy - Example
Interpixel Redundancy
• Because the value of any
pixel can be reasonably
predicted from the value of
its neighbors, much of the
visual contribution of a single
pixel to an image is
redundant, it could have
been guessed on the basis
of its neighbors’ values.
• Include spatial redundancy,
geometric redundancy,
interframe redundancy
Interpixel Redundancy - Example
Psychovisual Redundancy
• The eye does not respond with equal
sensitivity to all visual information. Certain
information simply has less relative
importance than other information in
normal visual processing
• In general, an observer searches for
distinguishing features such as edges or
textural regions and mentally combines
them into recognizable groupings.
Psychvisual Redundancy
Improved Gray Scale Quantization (IGS)

Evaluation Metrics
• Fidelity criteria
• Measure information
– Entropy
Fidelity Criteria
• Objective fidelity criteria
– Root-mean-square error (erms)
– Mean-square signal-to-noise ratio (SNRms)
– Root-mean-square SNRms (SNRrms)

Measure Information
• A random event E that occurs with
probability P(E) is said to contain
– I(E): self-information of E
– The amount of self-information attributed to
event E is inversely related to the probability of
E
– If base 2 is used, the unit of information is
called a bit
– Example: if P(E) = ½, then I(E) = -log2P(E) = 1.
Entropy
• Average information per source output, or
uncertainty, or entropy of the source
• How to interpret the increase of entropy?

• What happens if the events are equally
probable?
Example
• Estimate the information content (entropy)
of the following image (8 bit image):
• First-order estimate of the entropy

(bits/pixel)
• Second-order estimate of the entropy (bits

every two pixels)
What do these two numbers tell you?

Compression Approaches
• Error-free compression or lossless
compression
– Variable-length coding
– Bit-plane coding
– Lossless predictive coding
• Lossy compression
– Lossy predictive coding
– Transform coding
Variable-length Coding
• Only reduce code redundancy
• Assign the shortest possible code
words to the most probable gray
levels
• Huffman coding
– Can obtain the optimal coding scheme
Revisit Example
Can we do
this?
Huffman Coding
• Uniquely decodable
• Instantaneous coding
• Source reduction
– Order the probabilities of symbols
– Combine the lowest probability symbols into a
single symbol that replaces them in the next
source reduction
• Code each reduced source, starting with
the smallest source and working back to
the original source
Example
R1 0.25 (01) 0.25 (01) 0.25 (01) 0.25 (01) 0.35 (00) 0.4 (1) 0.6 (0)
R2 0.21 (10) 0.21 (10) 0.21 (10) 0.21 (10) 0.25 (01) 0.35 (00) 0.4 (1)
R0 0.19 (11) 0.19 (11) 0.19 (11) 0.19 (11) 0.21 (10) 0.25 (01)
R3 0.16 (001) 0.16 (001) 0.16 (001) 0.19 (000) 0.19 (11)
R4 0.08 (0001) 0.08 (0001) 0.11 (0000) 0.16 (001)
R5 0.06 (00000) 0.06 (00000) 0.08 (0001)
R6 0.03 (000010) 0.05 (00001)
R7 0.02 (000011)
Entropy of the source:
-0.25*log2(0.25) - 0.21*log2(0.21) - 0.19*log2(0.19) - 0.16*log2(0.16)
-0.08*log2(0.08) - 0.06*log2(0.06) - 0.03*log2(0.03) - 0.02*log2(0.02)
= 2.6508 bits/pixel
Average length of Huffman code:

2*0.25 + 2*0.21 + 2*0.19 + 3*0.16 + 4*0.08 + 5*0.06 + 6*0.03 + 6*0.02
= 2.7 bits
LZW
• Lempel-Ziv-Welch
• Attack interpixel redundancy - spatial
redundancy
• Assigns fixed-length code words to variable
length sequences of source symbols
• Requires no a-priori knowledge of the
probability of occurrence of the symbols
• Used in GIF, TIFF, PDF
• Coding book is created while the data are
being encoded
39 39 126 126
39 39 126 126
39 39 126 126
Example 39 39 126 126
Bit-plane Coding
• Attack inter-pixel redundancy
• First decompose the original image
into bit-plane
• Binary image compression approach
– run-length coding (RLC)
Bit-plane Decomposition
• Bit-plane slicing
• Problem: small changes in gray level can have a
significant impact on the complexity of the bit
plane
– 127 vs. 128  0111 1111 vs. 1000 0000
• Solution:
• Example:
– 127  0111 1111  0100 0000
– 128  1000 0000  1100 0000
Binary Image Compression
- RLC
• Developed in 1950s
• Standard compression approach in FAX
coding
• Approach
– Code each contiguous group of 0’s or 1’s
encountered in a left to right scan of a row by
its length
– Establish a convention for determining the
value of the run
– Code black and white run lengths separately
using variable-length coding
A Complete Example
2
Original image 127 128
3 32 33
0000 0010 0111 1111 1000 0000

Binary code
0000 0011 0010 0000 0010 0001
0000 0011 0100 0000 1100 0000

XOR binary 0000 0010 0011 0000 0011 0001 Run-length coding
001 011 000 000 000 000 100 100 21 12 3 3 3 3 012 012
8 bit planes 000 000 011 011 000 000 100 001 3 3 12 12 3 3 012 21
Huffman 1 0.34 (1) 0.66 (0) 3 1.00 (0) 0 0.34 (1) 1 2 0.4 (1) 0.6 (0)
coding 2 0.33 (00) 0.34 (1) 1 0.33 (00) 2 2 0.4 (00) 0.4 (1)
3 0.33 (01) 2 0.33 (01) 0 1 0.2 (01)
001 100 01 01 0 0 10001 01100

Final code 01 01 100 100 0 0 10001 001
Lossless Predictive Coding
• Do not need to decompose image into bit planes
• Eliminate interpixel redundancy
• Code only the new information in each pixel
• The new information is the difference between
the actual and predicted value of that pixel
Previous Pixel Coding
(differential coding)
• First-order linear predictor / previous

pixel predictor uses differential
coding
126 127 128
126 1 1
Lossless Compression
Approaches
Averagely, use
Variable-length coding shorter-length
Code code to
redundancy Huffman coding convey the
same amount of
information
Bit-plane coding
Run-length coding
Interpixel LZW
redundancy
Predictive coding

Lecture11 Lossless

Uploaded by

Copyright:

Available Formats

Lecture11 Lossless

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture11 Lossless

Uploaded by

Copyright:

Available Formats

ECE472/572 - Lecture 11

Image Compression – Fundamentals

Image Image Image

Image Image Image

Morphological Wavelet Recognition &

– CR: compression ratio

Improved Gray Scale Quantization (IGS)

– Mean-square signal-to-noise ratio (SNRms)

– Root-mean-square SNRms (SNRrms)

• How to interpret the increase of entropy?

• First-order estimate of the entropy

• Second-order estimate of the entropy (bits

What do these two numbers tell you?

Average length of Huffman code:

0000 0010 0111 1111 1000 0000

0000 0011 0100 0000 1100 0000

001 100 01 01 0 0 10001 01100

• First-order linear predictor / previous

You might also like