Scribd Logo
Upload

Welcome to Scribd!

  • 90 views

    Video Processing Communications Yao Wang Chapter8a

    Uploaded by

    Ashoka Vanjare
    This document provides an overview of video coding systems and lossless encoding techniques. It discusses the components of video coding, including binary encoding, entropy coding with Huffman coding and arithmetic coding. Huffman coding generates prefix codes to encode symbols but requires large vectors to approach entropy rates. Arithmetic coding can encode single symbols and achieves entropy rates, but is more complex to implement than Huffman coding. The document also covers modeling of random sources, information content characterization, and the bounds of lossless coding rates.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd

    Video Processing Communications Yao Wang Chapter8a

    Uploaded by

    Ashoka Vanjare
    90 views19 pages
    This document provides an overview of video coding systems and lossless encoding techniques. It discusses the components of video coding, including binary encoding, entropy coding with Huffman coding and arithmetic coding. Huffman coding generates prefix codes to encode symbols but requires large vectors to approach entropy rates. Arithmetic coding can encode single symbols and achieves entropy rates, but is more complex to implement than Huffman coding. The document also covers modeling of random sources, information content characterization, and the bounds of lossless coding rates.

    Copyright

    © © All Rights Reserved

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides an overview of video coding systems and lossless encoding techniques. It discusses the components of video coding, including binary encoding, entropy coding with Huffman coding and arithmetic coding. Huffman coding generates prefix codes to encode symbols but requires large vectors to approach entropy rates. Arithmetic coding can encode single symbols and achieves entropy rates, but is more complex to implement than Huffman coding. The document also covers modeling of random sources, information content characterization, and the bounds of lossless coding rates.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    90 views19 pages

    Video Processing Communications Yao Wang Chapter8a

    Uploaded by

    Ashoka Vanjare
    This document provides an overview of video coding systems and lossless encoding techniques. It discusses the components of video coding, including binary encoding, entropy coding with Huffman coding and arithmetic coding. Huffman coding generates prefix codes to encode symbols but requires large vectors to approach entropy rates. Arithmetic coding can encode single symbols and achieves entropy rates, but is more complex to implement than Huffman coding. The document also covers modeling of random sources, information content characterization, and the bounds of lossless coding rates.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    of 19

    Foundation of Video Coding Overview and Binary Encoding

    Sections 8.1-8.4 (lossless encoding)

    Yao Wang Polytechnic University, Brooklyn, NY11201

    Outline
    Overview of video coding systems
    Couple words on probability and information theory concepts Binary encoding Information theory bounds Huffman coding Arithmetic coding Look ahead Lossless coding: now - Section 8.4 Basic lossy coding: scalar and vector quantization Section 8.5, 8.6 Waveform coding: KLT, JPEG Chapter 9 Content-dependent video coding, e,g Facial Animation Parameters (FAPs) in MPEG4 Chapter 10 Video Compression Standards: MPEG-1, MPEG-2, MPEG-4, MPEG7 Chapter 13

    Components in a Coding System

    Video Coding Techniques Based on Different Source Models

    Waveform-based techniques

    Content-dependent-techniques

    H.261, H.263, MPEG-1, MPEG-2, MPEG-4* Transform coding + predictive coding

    Statistical Characterization of Random Sources


    Source: a random sequence (discrete time random process),
    Ex 1: an image that follows a certain statistics
    Fn represents the possible value of the n-th pixel of the image, n=(m,n) fn represents the actual value taken

    Ex 2: a video that follows a certain statistics


    Fn represents the possible value of the n-th pixel of a video, n=(k,m,n) fn represents the actual value taken

    Continuous source: Fn takes continuous values (analog image) Discrete source: Fn takes discrete values (digital image) Stationary source: statistical distribution invariant to time (space) shift

    Probability distribution
    probability mass function (discrete) (pmf) or probability density (continuous) function (pdf): Joint pmf or pdf and Conditional pmf or pdf

    Information Content Characterization of Discrete RVs


    Entropy of one RV:
    Entropy is a measure of uncertainty or information content Very uncertain -> high information content

    Joint entropy of two RVs:


    Uncertainty of two RVs together

    Conditional entropy between two RVs:


    Uncertainty of one RV given the other RV

    Mutual information between two RVs :


    Information provided by G about F

    Information Content Characterization of Discrete RVs


    Relations between Entropy (H), Conditional Entropy and Mutual Information (I)
    symmetric of course

    symmetric of course

    symmetric symmetric

    Lossless Coding (Binary Encoding)

    Binary encoding is a necessary step in any coding system


    Applied to
    original symbols (e.g. image pixels) in a discrete source, or converted symbols (e.g. quantized transformed coefficients) from a continuous or discrete source

    Binary encoding process (scalar coding)

    Symbol ai

    Binary Encoding

    Codeword ci (bit length li)

    Probability table pi Bit rate (bit/symbol):

    Bound for Lossless Coding


    Scalar coding: Assign one codeword to one symbol at a time Problem: could differ from the entropy by up to 1 bit/symbol Vector coding: Assign one codeword for each group of N symbols (vector) Larger N -> Lower Rate, but higher complexity

    R1 - minimum bit rate

    Conditional coding (context-based coding) alternative to vector coding The codeword for current symbol depends on the pattern (context) formed by the previous M symbols

    Binary Encoding: Requirement

    A good code should be:


    Uniquely decodable Instantaneously decodable prefix code (no prefix is another code word!)

    10

    Huffman Coding

    Idea: more frequent symbols -> shorter codewords Algorithm:

    Huffman coding generate prefix code Can be applied to one symbol at a time (scalar coding), or a group of symbols (vector coding), or one symbol conditioned on previous symbols (conditional coding)
    11

    Huffman Coding Example: Source description


    Four symbols: a,b,c,d pmf: pT [0.5000,0.2143,0.1703,0.1154]


    1st order conditional pmf:

    2nd order pmf:

    Ex. p("ab") p("a")q("b" /"a") 0.5 * 0.1875 0.0938

    12

    Huffman Coding Example: Scalar Coding

    13

    Huffman Coding Example: Vector Coding

    Huffman Coding Example: Conditional Coding

    RC ,"a" 1.5625, RC ,"b" RC ,"c" RC ,"d " 1.9375, RC ,1 1.7500 H C ,"a" 1.5016, H C ,"b" H C ,"c" H C ,"d " 1.8829, H C ,1 1.6922
    different codebook depending on previous symbols this example for previous symbol b
    15

    Arithmetic Coding

    Basic idea: Represent a sequence of symbols by an interval with length equal to its probability The interval is specified by its lower boundary (l), upper boundary (u) and length d (=probability) The codeword for the sequence is the common bits in binary representations of l and u The interval is calculated sequentially starting from the first symbol The initial interval is determined by the first symbol The next interval is a subinterval of the previous one, determined by the next symbol

    pl - probabilit y of symbolal ql - cumulative probabilit y up to lth symbol

    16

    Encoding:

    Decoding:

    Huffman vs. Arithmetic Coding

    Huffman coding
    Convert a fixed number of symbols into a variable length codeword Efficiency: To approach entropy rate, must code a large number of symbols together Used in all image and video coding standards

    Arithmetic coding
    Convert a variable number of symbols into a variable length codeword Efficiency: in limit same as Huffman N is sequence length Can approach the entropy rate by processing one symbol at a time Easy to adapt to changes in source statistics Complex, but integer implementation is available No codebook needed Question on precision of computation (finite-precision methods exist) Used as advanced options in image and video coding standards
    18

    Summary

    Coding system:
    original data -> model parameters -> quantization-> binary encoding Waveform-based vs. content-dependent coding

    Lossless coding
    Bit rate bounded by entropy rate of the source Huffman coding:
    Scalar, vector, conditional coding can achieve the bound only if a large number of symbols are coded together Huffman coding generates prefix code (instantaneously decodable)

    Arithmetic coding
    Can achieve the bound by processing one symbol at a time More complicated than scalar or short vector Huffman coding
    19

    You might also like