Chapter 3. Algorithms

Algorithms The Growth of Functions Complexity of Algorithms Problems
Chapter 3. Algorithms
Lai Văn Phút
Ngày 8 tháng 6 năm 2022
Lai Văn Phút Chapter 3. Algorithms

Contents
1. Algorithms

Contents
1. Algorithms
2. The Growth of Functions

Contents
1. Algorithms
2. The Growth of Functions
3. Complexity of Algorithms

Example.
• Finding the Maximum Element in a Finite Sequence

{8, 4, 11, 3, 10}.

Example.

{8, 4, 11, 3, 10}.
• To search for 19 in the list
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22

Example.

{8, 4, 11, 3, 10}.
• To search for 19 in the list
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22
• ...

Definition
An algorithm is a finite sequence of precise instructions

(mã lệnh xác định) for performing a computation or for
solving a problem.

PROPERTIES OF ALGORITHMS
• Input. An algorithm has input values from a specified
set.

set.
• Output. From each set of input values an algorithm
produces output values from a specified set (solution).

set.
• Definiteness. The steps of an algorithm must be
defined precisely.

set.
defined precisely.
• Correctness. An algorithm should produce the correct
output values for each set of input values.

set.
defined precisely.
• Finiteness. An algorithm should produce the desired
output after a finite (but perhaps large) number of
steps for any input in the set.

set.
defined precisely.
• Effectiveness. It must be possible to perform each step
of an algorithm exactly and in a finite amount of time.

set.
defined precisely.
• Effectiveness. It must be possible to perform each step
of an algorithm exactly and in a finite amount of time.
• Generality. The procedure should be applicable for all
problems of the desired form.
Finding the Maximum Element in a Finite

Sequence.


Sequence.
Example. Finding the Maximum Element in a Finite

Sequence {8, 4, 11, 3, 10}.


Sequence.

Sequence {8, 4, 11, 3, 10}.
procedure max (a1 , a2 , a3 , a4 , a5 : integers)


Sequence.

Sequence {8, 4, 11, 3, 10}.
max:=a1


Sequence.

Sequence {8, 4, 11, 3, 10}.
max:=a1
for i:=2 to 5


Sequence.

Sequence {8, 4, 11, 3, 10}.
max:=a1
for i:=2 to 5
if max < ai then max:= ai


Sequence.

Sequence {8, 4, 11, 3, 10}.
max:=a1
for i:=2 to 5
if max < ai then max:= ai
return max {max is the largest element}
Example.
{8, 4, 11, 3, 10}

Max=8

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11
Thus, Max=11

The Linear Search Algorithm

The Linear Search Algorithm
Example. List all the steps used to search for 9 in the

sequence 2, 3, 4, 5, 6, 8, 9, 11 using a linear search. How
many comparisons required to search for 9 in the
sequence.

Solution
Below is the linear search algorithm in pseudocode

2, 3, 4, 5, 6, 8, 9, 11
procedure linear search (x: integer, a1 , a2 , ..., a8 :
distinct integers)

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1
while (i ≤ 8 and x 6= ai )

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1
i := i + 1

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1
i := i + 1
if i ≤ 8 then location: = i

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1
i := i + 1
else location: = 0

Solution

2, 3, 4, 5, 6, 8, 9, 11
distinct integers)
i := 1
i := i + 1
else location: = 0
return location {location is the subscript of the term
that equals x, or is 0 if x is not found}

2, 3, 4, 5, 6, 8, 9, 11

The Binary(nhị phân) Search Algorithm

The Binary(nhị phân) Search Algorithm
Example. To search for 19 in the list

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
procedure binary search ( x:integer, a1 , a2 , . . . , a16 :
increasing integers)

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
i := 1 { i is left endpoint of search interval}

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
j := 16 { j is right endpoint of search interval}

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c
if x > am then i := m + 1

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c
else j := m

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c
else j := m
if x = ai then location: = i

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c
else j := m
else location: = 0

Solution
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22. Search 19
while i < j
m := b(i + j )/2c
else j := m
else location: = 0
return location {location is the subscript of the term
that equals x, or is 0 if x is not found}

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13, 19 6= 18 →location:=0
13 + 16
m=b c = b14.5c = 14
2
19 ≤ 19 → j = 14,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13, 19 6= 18 →location:=0
13 + 16
m=b c = b14.5c = 14
2
19 ≤ 19 → j = 14,

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13, 19 6= 18 →location:=0
13 + 16
m=b c = b14.5c = 14
2
19 ≤ 19 → j = 14, 19 6= 18 →location:=0
13 + 14
m=b c = b13.5c = 13
2
19 > 18 → i = 13 + 1 = 14

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22.

Search 19
i=1
j = 16
while i < j
1 + 16
m=b c = b8.5c = 8
2
19 > 16 → i = 8 + 1 = 9, 19 6= 12 →location:=0
9 + 16
m=b c = b12.5c = 12
2
19 > 16 → i = 12 + 1 = 13, 19 6= 18 →location:=0
13 + 16
m=b c = b14.5c = 14
2
19 ≤ 19 → j = 14, 19 6= 18 →location:=0
13 + 14
m=b c = b13.5c = 13
2
19 > 18 → i = 13 + 1 = 14
19 = 19 → location = 14
Sorting
Sorting is putting the elements into a list in which the

elements are in increasing order.

Sorting
Sorting is putting the elements into a list in which the

elements are in increasing order.
For instance, sorting the list 7, 2, 1, 4, 5, 9 produces the list
1, 2, 4, 5, 7, 9. Sorting the list d, h, c, a, f (using
alphabetical order) produces the list a, c, d, f, h.

The Bubble Sort

The Bubble Sort
Example. Use the bubble sort to put 3, 2, 4, 1, 5 into

increasing order.

The Bubble Sort

increasing order.
procedure bubble sort (a1 , a2 , . . . , a5 :real numbers
with 5 ≥ 2)

The Bubble Sort

increasing order.
with 5 ≥ 2)
for i := 1 to 4

The Bubble Sort

increasing order.
with 5 ≥ 2)
for i := 1 to 4
for j := 1 to 5 − i

The Bubble Sort

increasing order.
with 5 ≥ 2)
for i := 1 to 4
for j := 1 to 5 − i
if aj > aj +1 then interchange aj and aj +1

The Bubble Sort

increasing order.
with 5 ≥ 2)
for i := 1 to 4
for j := 1 to 5 − i
if aj > aj +1 then interchange aj and aj +1
{a1 , a2 , . . . , a5 is in increasing order}
Use the bubble sort to put 3, 2, 4, 1, 5 into

increasing order
i=1


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3
j = 1 : 2 > 1 → 1, 2, 3, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3
j = 1 : 2 > 1 → 1, 2, 3, 4, 5
j = 2 : 2 ≤ 3 → 1, 2, 3, 4, 5


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3
j = 1 : 2 > 1 → 1, 2, 3, 4, 5
j = 2 : 2 ≤ 3 → 1, 2, 3, 4, 5
i=4


increasing order
i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3
j = 1 : 2 > 1 → 1, 2, 3, 4, 5
j = 2 : 2 ≤ 3 → 1, 2, 3, 4, 5
i=4
j = 1 : 1 ≤ 2 → 1, 2, 3, 4, 5

increasing order

The Insertion Sort
(i < j, while True->compute, False->next)

The Insertion Sort
(i < j, while True->compute, False->next)

Example. Use the insertion sort to put the elements of the
list 3, 2, 4, 1, 5 in increasing order.
Use the insertion sort to put the elements of the list

3, 2, 4, 1, 5 in increasing order
procedure insertion sort (a1 , a2 , . . . , a5 :real numbers
with 5 ≥ 2)


with 5 ≥ 2)
for j := 2 to 5 { j: position of the examined element }
{ finding out the right position of aj }


with 5 ≥ 2)
i := 1


with 5 ≥ 2)
i := 1
while aj > ai


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
{ moving j-i elements backward }


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
for k := 0 to j − i − 1


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
for k := 0 to j − i − 1
aj −k := aj −k −1


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
for k := 0 to j − i − 1
aj −k := aj −k −1


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
for k := 0 to j − i − 1
aj −k := aj −k −1
{move aj to the position i}
ai := m


with 5 ≥ 2)
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
for k := 0 to j − i − 1
aj −k := aj −k −1
{move aj to the position i}
ai := m
{a1 , a2 , . . . , a5 is increasing order}
3, 2, 4, 1, 5
j=2

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1

3, 2, 4, 1, 5
j=2
i = 1, 2 ≤ 3 → i = 1, m = 2
k = 0 → a2 = 3, a1 = 2
→ 2, 3, 4, 1, 5
j=3
i = 1, 4 > 2
i = 2, 4 > 3
→ 2, 3, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5

3, 2, 4, 1, 5
j=4

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1
i = 2, 5 > 2

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1
i = 2, 5 > 2
i = 3, 5 > 3

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1
i = 2, 5 > 2
i = 3, 5 > 3
i = 4, 5 > 4

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1
i = 2, 5 > 2
i = 3, 5 > 3
i = 4, 5 > 4

3, 2, 4, 1, 5
j=4
i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a4 = 4
k = 1 → a3 = 3
k = 2 → a2 = 2
a1 = 1
→ 1, 2, 3, 4, 5
j=5
i = 1, 5 > 1
i = 2, 5 > 2
i = 3, 5 > 3
i = 4, 5 > 4
Thus, → 1, 2, 3, 4, 5

Introduction

Example.
(constant that mean f (n) = k, ∀n. In this ex, k = 1)

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!
• Growth of logn < growth of n

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!
• Growth of logn < growth of n

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!
• Growth of logn < growth of n → logn is O (n)
• In general, growth of f (n) ≤ growth of g (n)

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!
• In general, growth of f (n) ≤ growth of g (n)

Example.

→ 1 < log2 n(≡ logn) < n < n2 < 2n < n!
• In general, growth of f (n) ≤ growth of g (n) → f (n) is
O (g (n))
Big-O Notation

Big-O Notation

Example.
Show that f (x ) = x 2 + 2x + 1 is O (x 2 ).

Example.
Solution.
∀x > 1 =⇒ x 2 > 1 ∧ x 2 > x

Example.
Solution.
∀x > 1 =⇒ x 2 > 1 ∧ x 2 > x
f (x ) = x 2 + 2x + 1 < x 2 + 2x 2 + x 2 = 4x 2

Example.
Solution.
∀x > 1 =⇒ x 2 > 1 ∧ x 2 > x
f (x ) = x 2 + 2x + 1 < x 2 + 2x 2 + x 2 = 4x 2
Let g (x ) = x 2

Example.
Solution.
∀x > 1 =⇒ x 2 > 1 ∧ x 2 > x
f (x ) = x 2 + 2x + 1 < x 2 + 2x 2 + x 2 = 4x 2
Let g (x ) = x 2
We have C = 4, k = 1, |f (x )| ≤ C |g (x )|

Example.
Solution.
∀x > 1 =⇒ x 2 > 1 ∧ x 2 > x
f (x ) = x 2 + 2x + 1 < x 2 + 2x 2 + x 2 = 4x 2
Let g (x ) = x 2
We have C = 4, k = 1, |f (x )| ≤ C |g (x )|
Thus, f (x ) is O (x 2 )


The growth of functions commonly used in big-O

estimates.


estimates.
1 < logn < n < nlogn < n2 < 2n < n!


estimates.
1 < logn < n < nlogn < n2 < 2n < n!


estimates.
1 < logn < n < nlogn < n2 < 2n < n!
How about n3 , 3n and 4n2 ?


estimates.
1 < logn < n < nlogn < n2 < 2n < n!
How about n3 , 3n and 4n2 ?

Note. Cf (n) ∼ f (n) , C is constant.

Big-O theorem
1. Let f (x ) = an x n + an−1 x n−1 + ... + a1 x + a0 where

a1 , a2 , ....an−1 , an are real numbers. Then, f (x ) is O (x n ).

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))
COROLLARY. Suppose that f1 (x ) and f2 (x ) are both

O (g (x )). Then (f1 + f2 )(x ) is O (g (x )).

Example.
1. Estimate big-oh of functions 100n2 + 23nlogn + 2019

Example.

2. Give a big-O estimate for (2n2 + 17n)(7logn + 15)

Example.

3. Give a big-O estimate for
f (n) = 3nlog (n!) + (n2 + 3)logn, where n is a positive
integer.

Example.

f (n) = 3nlog (n!) + (n2 + 3)logn, where n is a positive
integer.
f (x ) = (x + 1)log (x 2 + 1) + 3x 2 .

Solution.

Quizz

Quizz
Ans: C

Quizz

Quizz
Ans: a,b (hàm nào có O(n))(f (x ) = O (g (x )) mean f(x) is

big-oh of g(x))

Quizz

Quizz
Ans: a
Big-Omega and Big-Theta Notation
• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

• f (x ) = O (g (x )) ∧ f (x ) = Ω(g (x )) → f (x ) = Θ(g (x ))

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))


• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

Example. Show that f (x ) = 1 + 2 + ... + n is Θ(n2 )

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
1. f (x ) is Ω(g (x )) if and only if g (x ) is O (f (x )).

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
2. If f (x ) = Θ(g (x )) then f(x) is of order g(x) or f(x) and
g(x) are of the same order (cùng độ tăng).

• ∃C > 0, k : x ≥ k ∧ |f (x )|≥C |g (x )| → f (x ) = Ω(g (x ))

n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
2. If f (x ) = Θ(g (x )) then f(x) is of order g(x) or f(x) and
g(x) are of the same order (cùng độ tăng).
a1 , a2 , ....an−1 , an are real numbers. Then, f (x ) is Θ(x n ).

Introduction
• Computational complexity = Time complexity +

space complexity (not be considered).

Introduction

• Time complexity can be expressed in terms of the
number of operations used by the algorithm.

Introduction

1. Worst-case complexity (tells us how many operations
an algorithm requires to guarantee that it will produce
a solution.).

Introduction

1. Worst-case complexity (tells us how many operations
an algorithm requires to guarantee that it will produce
a solution.).
2. Average-case complexity (the average number of
operations used to solve the problem over all possible
inputs of a given size).

Example.
{8, 4, 11, 3, 10}

Max=8

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11

Example.
{8, 4, 11, 3, 10}

Max=8
i = 2(8 ≥ 4) → Max=8
i = 3(8 < 11) → Max=11
i = 4(11 ≥ 3) → Max=11
i = 5(11 ≥ 10) → Max=11
Thus, Max=11

Example.
Describe the time complexity (Worst-case) of the

algorithm for finding the largest element in a set.

Example.

→ Number of comparisons f (n) = 2(n − 1) + 1.

Example.

→ Number of comparisons f (n) = 2(n − 1) + 1.Thus,

f (x ) = Θ (n).

Example.

Example.
Ans:f(n)=n+n=2n is O(n)

Example.

Example.
Ans: f (n) = n.n = n2 is O (n2 )

Example.

Example.
Ans: f (n) = n + n2 is O (n2 )

Quizz

Quizz
Ans: b (Number of comparisons f (n) = 2n + 1)

Quizz

Quizz
Ans: a (f (n) = n)

Algorithms

Algorithms

The Growth of Functions

The Growth of Functions

Complexity of Algorithms



Chapter 3. Algorithms

Uploaded by

Copyright:

Available Formats

Chapter 3. Algorithms

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 3. Algorithms

Uploaded by

Copyright:

Available Formats

Algorithms The Growth of Functions Complexity of Algorithms Problems

Lai Văn Phút

Ngày 8 tháng 6 năm 2022

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Lai Văn Phút Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Lai Văn Phút Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Lai Văn Phút Chapter 3. Algorithms

An algorithm is a finite sequence of precise instructions

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

Lai Văn Phút Chapter 3. Algorithms

Finding the Maximum Element in a Finite

Example. Finding the Maximum Element in a Finite

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Lai Văn Phút Chapter 3. Algorithms

The Linear Search Algorithm

Lai Văn Phút Chapter 3. Algorithms

The Linear Search Algorithm

Example. List all the steps used to search for 9 in the

Lai Văn Phút Chapter 3. Algorithms

Below is the linear search algorithm in pseudocode