MMAT5390: Mathematical Image Processing

Chapter 2: Review of Basic Mathematical Concepts

As described in the last chapter, a digital image can be considered as a big two-dimensional matrix.
As a image resolution increases, a digital image can be regarded as a discretization of a continuous
function defined on a two-dimensional rectangular domain. Mathematical image processing are
mainly based on two mathematical tools, namely, advanced linear algebra and advanced calculus.
In this chapter, we will review some basic mathematical concepts necessary for our discussions.

1 Basic linear algebra

We will first review some basic mathematical concepts related to linear algebra, which will be used
in our discussions about mathematical imaging. The proofs for the theorems are omitted.

1.1 Determinant of a matrix

Definition 1.1. Let A be a n × n matrix. If n = 1, so that A = (A11 ), we define det(A) = A11 .
For n ≥ 2, we define det(A) recursively as
n
X
det(A) = (−1)1+j A1j det(Ã1j ),
j=1

where Ãij is the (n − 1) × (n − 1) matrix obtained from removing row i and column j of A (called
the minor of the entry of A in row i, column j). The scalar det(A) is called the determinant of
A and is also denoted by |A|. The scalar

(−1)i+j det(Ãij )

is called the cofactor of the entry of A in row i, column j.

Theorem 1.2. The determinant of a n × n matrix is a linear function of each row when the
remaining rows are held fixed. That is, for 1 ≤ r ≤ n, we have
     
a1 a1 a1
 ..   ..   .. 
 .   .   . 
     
 ar−1  ar−1  ar−1 
     
u + kv = det  u  + k det  v 
det      
 ar+1  ar+1  ar+1 
     
 .   .   . 
 ..   ..   .. 
an an an

whenever k is a scalar and u, v and each ai are row vectors in Rn .

Theorem 1.3. The determinant of a square matrix can be evaluated by cofactor expansion along
any row. That is, if A is a n × n matrix, then for any integer i (1 ≤ i ≤ n),
n
X
det(A) = (−1)i+j Aij det(Ãij ).
j=1

Corollary. If a square matrix A has two identical rows, then det(A) = 0.

Theorem 1.4. If A is a square matrix and B is a matrix obtained from A by interchanging any
two rows of A, then det(B) = −det(A).

1
Theorem 1.5. Let A be a square matrix, and let B be a matrix obtained by adding a multiple of
one row of A to another row of A. Then det(B) = det(A).

Corollary. If a n × n matrix has rank less than n, then det(A) = 0.

Theorem 1.6. For any two n × n matrices A and B, det(AB) = det(A)det(B).
Corollary. A square matrix is invertible if and only if det(A) 6= 0. Furthermore, if A is invertible,
1
then det(A−1 ) = .
det(A)
Theorem 1.7. For any square matrix A, det(AT ) = det(A).

1.2 Eigenvalues and Eigenvectors

Definition 1.8. Let A be a n × n matrix. A nonzero vector v ∈ Rn is called an eigenvector
of A if there exists a scalar λ such that Av = λv. The scalar λ is called the eigenvalue of A
corresponding to the eigenvector v.

Definition 1.9. Let A be a n × n matrix. The polynomial f (t) = det(A − tIn ) is called the
characteristic polynomial of A.
Definition 1.10. Let A be a m × n matrix. We define the conjugate transpose or adjoint of
A to be the n × m matrix A∗ such that (A∗ )ij = Aji for all i, j.
Definition 1.11. Let A be a n × n matrix. A is said to be normal if AA∗ = A∗ A; in particular,
A is said to be unitary if AA∗ = A∗ A = In .
Definition 1.12. Let A and B be n × n matrices. A is said to be similar to B if there exists an
invertible matrix C such that B = C −1 AC. In particular, A is said to be unitarily equivalent
to B if C is unitary.

1.3 Diagonalization and Jordan Canonical Form

Definition 1.13. Let A be a n × n matrix. A is said to be diagonalizable if A is similar to a
diagonal matrix; in particular, A is said to be unitarily diagonalizable if A is unitarily equivalent
to a diagonal matrix.

Theorem 1.14. A square matrix is normal if and only if it is unitarily diagonalizable.

Theorem 1.15. All eigenvalues of a real symmetric matrix are real.
Theorem 1.16. For every n × n complex
 matrix A, thereexists a n × n complex invertible matrix
A1 0 · · · 0
 0 A2 · · · 0
B such that B −1 AB is of the form  . .. , where each matrix block Ai is of the
 
 .. ..
. . 
0 0 · · · Ak
 
λi 1 0 · · · 0 0
 0 λi 1 · · · 0 0 
 
 0 0 λi · · · 0 0 
form  . .. .
 
 .. ..
 . . 

 0 0 0 · · · λi 1 
0 0 0 · · · 0 λi

Definition 1.17. The product B −1 AB in Theorem 1.16 is said to be a Jordan canonical form
of A.

2
2 Vector calculus in Euclidean space Rn
In this short section, we shall introduce some basic concepts about the vector calculus, which are
widely used in engineering, physics and other areas requiring mathematics.
Consider the Euclidean space Rn , with a rectangular coordinate system formed by the x1 -, x2 -,
· · · , and xn -coordinate axis. The vector calculus is about some basic operations of a vector-valued
function in Rn .
For any given vector-valued function v(x) in Rn with n components, we shall write v(x) as

v(x) = (v1 (x), v2 (x), · · · , vn (x))T with x = (x1 , x2 , · · · , xn )T .

2.1 Gradient and divergence

∂u ∂u ∂u

T
Gradient vector. For each scalar function u(x) in Rn , we can define a vector v = ∂x1 , ∂x2 , · · · , ∂x n
.
This function is called the gradient vector of u(x). We often write

v = grad u = ∇u .

Example 2.1. Find the gradient of u for u(x1 , x2 , x3 ) = x21 + x22 + sin(πx3 ).
Divergence. For any given vector-valued function v = (v1 , v2 , · · · , vn )T , we can define a scalar
∂v1 ∂v2 ∂vn
function w(x) = ∂x 1
+ ∂x2
+ · · · + ∂xn
. This function is called the divergence of function v. And
we often write
∂v1 ∂v2 ∂vn
div v = + + ··· + ≡∇·v
∂x1 ∂x2 ∂xn
It is easy to verify that

∂2u ∂2u ∂2u

div grad u = ∇ · ∇u = + + · · · + ,
∂x21 ∂x22 ∂x2n

and we often write

∂2u ∂2u ∂2u
∆u = 2 + 2 + ··· + .
∂x1 ∂x2 ∂x2n
∆ is called the Laplacian operator. The notation ∆ is often extended to vector-valued functions:
for a vector-valued function v(x) in Rn , we write

∆v = (∆v1 , ∆v2 , · · · , ∆vn )T .

2.2 Vorticity and curl operations

Three dimensions
Curl vector (vorticity). For a given vector-valued function v = (v1 , v2 , v3 )T in R3 , we can define
the following special vector-valued function
 
∂v3 ∂v2
∂x2 − ∂x3
 ∂v1 ∂v3 
curl v ≡ ∇ × v =  ∂x3
− ∂x1

∂v2 ∂v1
∂x1 − ∂x2

This vector is called the curl vector of v or the vorticity of v. It may help remember the curl
operation if we compare it with the vector cross-product a × b.
Two dimensions
In two dimensions, curl operation is also frequently used. But its operation is very different from
three dimensions. For any given scalar function v(x, y), we define

∂v ∂v
curl v = ( , − )T .
∂y ∂x
Sometimes, we write
curl v = ∇⊥ v ,

3
as ∇ and ∇⊥ are orthogonal in the sense that

∇v · ∇⊥ v = 0 .

It is interesting to notice that the Euclidean norm of curl v is the same as the gradient ∇v. But
this is only valid in two dimensions. (Why?)
One can also define another curl operation in two dimensions. For any vector-valued function
v(x, y), we define
∂v2 ∂v1
curl v = − .
∂x ∂y
This maps a vector-valued function into a scalar function.

2.3 Some relations between gradient, divergence and curl operations

Example 2.2. Verify that the curl of the gradient of any function u(x1 , x2 , x3 ) is zero.
Solution. By the definition, we can check
 ∂   ∂u 
∂x1 ∂x1
∂  ∂u 
∇ × ∇u =  ∂x2
 ×
∂x2 =0.
∂ ∂u
∂x3 ∂x3

The student may work out the detail. 

• Find ∇ × ∇u when 2 2
u(x1 , x2 , x3 ) = e−x1 +x2 +sin(πx3 ) sin(πx1 x2 x3 ).
• Think about the operation ∇ (∇ × u).

Example 2.3. Verify that the divergence of the curl of any vector-valued function v(x1 , x2 , x3 ) is
zero.
Solution. The student may work out the detail for

div curl v = ∇ · (∇ × v) = 0 .


• Think about the operation curl (div u). It does not make sense in 3D. What about this operation
in 2D ?
Example 2.4. For any vector-valued function v(x1 , x2 , x3 ), verify the following relation

∇ × (∇ × v) = ∇(∇ · v) − ∆v .

The following conclusions are widely used to simplify some mathematical models:
Theorem 2.5. If a vector-valued function v(x1 , x2 , x3 ) is divergence-free, that is, ∇ · v = 0, then
there exists a vector-valued function w(x1 , x2 , x3 ) such that

v(x1 , x2 , x3 ) = ∇ × w(x1 , x2 , x3 ) .

On the other hand, if a vector-valued function v(x1 , x2 , x3 ) is vorticity-free, that is, ∇ × v = 0,

then there exists a scalar field φ(x1 , x2 , x3 ) such that

v(x1 , x2 , x3 ) = ∇φ(x1 , x2 , x3 ) .

Finally, the students may try themselves to verify the following property:
Example 2.6. For any scalar function u(x1 , x2 , · · · , xn ) and any vector-valued function v(x1 , x2 , · · · , xn ),
we have
∇ · (u v) = ∇u · v + u(∇ · v) .

4
What is the operation ∇ × (u v) ? Work out a convenient formula for evaluating such operation.
The following integration by parts formulae are widely used in deriving variational formulations
for various partial differential equations:
For two scalar functions u and v, we have
Z Z Z
uxi v dx = − u vxi dx + u v ni ds .
Ω Ω ∂Ω

where Ω ⊂ Rk , n = (n1 , n2 , · · · , nk )T is the unit outward normal direction to the boundary ∂Ω of

Ω. Hence for a vector-valued function u and a scalar function v, we have
Z Z Z
(∇ · u) v dx = − (u · ∇v) dx + (u · n) v ds .
Ω Ω ∂Ω

For two vector-valued functions u and v, we have

Z Z Z
(∇ × u) · v dx = u · (∇ × v) dx − (u × n) · v ds .
Ω Ω ∂Ω