1

Chapter 3: Brute Force and Exhaustive Search

Introduction

The subject of this chapter is brute force and its important special case, exhaustive
search. Brute force can be described as follows:
Brute force is a straightforward approach to solving a problem, usually directly
based on the problem statement and definitions of the concepts involved.

Example: Computing 𝑥 𝑛 .
By the definition of exponentiation,
𝑥𝑛 = ⏟ 𝑥 × …× 𝑥
𝑛 𝑡𝑖𝑚𝑒𝑠
𝑛
This suggests simply computing 𝑥 by multiplying 1 by 𝑥 𝑛 times.

The role of brute force

1. Brute-force is applicable to a very wide variety of problems.
2. A brute-force algorithm can still be useful for solving small-size instances of a
problem.
3. The expense of designing a more efficient algorithm may be unjustifiable if only a
few instances of a problem need to be solved and a brute-force algorithm can solve
those instances with acceptable speed.
4. A brute-force algorithm can serve an important theoretical or educational purpose
as a yardstick with which to judge more efficient alternatives for solving a problem.
5. A first application of the brute-force approach often results in an algorithm that can
be improved with a modest amount of effort.

Example: Bubble sort - Θ(𝑛2 )

Bubblesort(a[1 .. n]) {
for (i = 2; i  n; i++)
for (j = n; j  i; j--)
if (a[j - 1] > a[j])
a[j - 1]  a[j];
}
Note: We can improve the above algorithm by exploiting some observations and an
upgrade version of Bubble sort is Shake sort.
 2

Example: Brute force string matching

SequentialStringSearch(T[1 .. n], P[1 .. m]) {
count = 0;
for (i = 1; i ≤ n – m + 1; i++) {
j = 0;
while (j < m) && (P[1 + j] == T[i + j])
j++;
if (j == m)
count++;
}
return count;
}

The best-case efficiency: 𝐵(𝑛) =

The worst-case efficiency: 𝑊(𝑛) =
The average-case efficiency: 𝐴(𝑛) =

Find the subsequence with largest sum of elements in an array

Given an array of 𝑛 integers 𝑎1 , 𝑎2 , … , 𝑎𝑛 . The task is to find indices 𝑖 and 𝑗 with

𝑗
1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛, such that the sum ∑𝑘=𝑖 𝑎𝑘 is as large as possible. If the array contains all
non-positive numbers, then the largest sum is 0.

Brute force version with running time Θ(𝑛3 )

MaxContSubSum(a[1 .. n]) {
maxSum = 0;
for (i = 1; i ≤ n; i++)
for (j = i; j ≤ n; j++) {
curSum = 0;
for (k = i; k ≤ j; k++)
curSum += a[k];
if (curSum > maxSum)
maxSum = curSum;
}
return maxSum;
}
 3

Upgrade version with running time Θ(𝑛2 )

MaxContSubSum(a[1 .. n]) {
maxSum = 0;
for (i = 1; i ≤ n; i++) {
curSum = 0;
for (j = i; j ≤ n; j++) {
curSum += a[j];
if (curSum > maxSum)
maxSum = curSum;
}
}
return maxSum;
}

Dynamic programming version with running time Θ(𝑛)

MaxContSubSum(a[1 .. n]) {
maxSum = curSum = 0;
for (j = 1; j ≤ n; j++) {
curSum += a[j];
if (curSum > maxSum)
maxSum = curSum;
else
if (curSum < 0)
curSum = 0;
}
return maxSum;
}

The change-making problem

Given 𝑘 denominations: 𝑑1 < 𝑑2 < ⋯ < 𝑑𝑘 where 𝑑1 = 1. Find the minimum

number of coins (of certain denominations) that add up to a given amount of money 𝑛.
 4

Example: Suppose that there are 4 denominations: 𝑑1 = 1, 𝑑2 = 5, 𝑑3 = 10, 𝑑4 =

25 and the amount of money 𝑛 = 72. The minimum number of coins is 6 including two
pennies, two dimes, and two quarters.

Idea: Find all 𝑘-tuples 〈𝑐1 , 𝑐2 , … , 𝑐𝑘 〉 such that:

𝑐1 × 𝑑1 + 𝑐2 × 𝑑2 + ⋯ + 𝑐𝑘 × 𝑑𝑘 = 𝑛
where 𝑐𝑖 is the number of coins of denomination 𝑑𝑖 . Among these 𝑘-tuples we choose the
one with the smallest sum ∑𝑘𝑖=1 𝑐𝑖 .

The efficiency of this algorithm is as follows:

𝑛 𝑛 𝑛 𝑛𝑘
𝑇(𝑛) = × ×…× = ∈ Θ(𝑛𝑘 )
𝑑1 𝑑2 𝑑𝑘 𝑑1 × 𝑑2 × … × 𝑑𝑘
because 𝑑1 , 𝑑2 , … , 𝑑𝑘 are constants.

Closest-Pair Problem

This problem calls for finding the two closest points in a set of 𝑛 points in the plan.

Algorithm with running time Θ(𝑛2 )

BruteForceClosestPoints(P[1 .. n]) {
dmin = ;
for (i = 1; i ≤ n - 1; i++)
for (j = i + 1; j ≤ n; j++) {
d = √(𝑃[𝑖]. 𝑥 − 𝑃[𝑗]. 𝑥)2 + (𝑃[𝑖]. 𝑦 − 𝑃[𝑗]. 𝑦)2 ;
if (d < dmin) {
dmin = d;
point1 = P[i];
point2 = P[j];
}
}
return <point1, point2>;
}
 5

Convex-hull problem

Intuitively, the convex-hull of a set of 𝑛 points in the plane is the smallest convex
polygon that contains all of them either inside or on its boundary.
Note: Mathematicians call the vertices of such a polygon “extreme points.” Finding these
points is the way to construct the convex hull for a given set of 𝑛 points.

A brute force approach is based on the following observation:

“A line segment 𝑝𝑞̅̅̅ connecting two points 𝑝 and 𝑞 of a set of 𝑛 points is a part of
the convex hull’s boundary if and only if all the other points of the set lie on the same side
of the straight line through these two points.”

It’s known that the straight line through two points (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) in the
coordinate plane can be defined by the equation
𝑎𝑥 + 𝑏𝑦 = 𝑐
where 𝑎 = 𝑦2 − 𝑦1 , 𝑏 = 𝑥1 − 𝑥2 , 𝑐 = 𝑥1 𝑦2 − 𝑥2 𝑦1 .
Second, such a line divides the plane into two half-planes: for all the points in one
of them, 𝑎𝑥 + 𝑏𝑦 > 𝑐, while for all the points in the other, 𝑎𝑥 + 𝑏𝑦 < 𝑐. For the points on
the line itself, 𝑎𝑥 + 𝑏𝑦 = 𝑐.

Brute force algorithm with running time Θ(𝑛3 )

for (each point pi  S: i = 1 → n - 1)
for (each point pj  S: j = i + 1 → n) {
Construct the line 𝑝̅̅̅̅̅;
𝑖 𝑝𝑗
if (all the other points in S lie on the same side of
𝑝
̅̅̅̅̅)
𝑖 𝑝𝑗
Store the pair of two points <pi, qj>;
}
 6

Now, let’s consider another brute force approach whose output is a list of the
extreme points in a counterclockwise order.

Initially, the point with the lowest y-value must be the first extreme point in the
result list. If there are two points with the same lowest y-value then the leftmost one is the
first extreme point.

Assume that 𝑝 is the current extreme point, how to find out the next one?

Let’s denote 𝑐𝑢𝑟𝐴𝑛𝑔𝑙𝑒 the current angle (initially, 𝑐𝑢𝑟𝐴𝑛𝑔𝑙𝑒 = 0). The next
extreme point 𝑞 must satisfy the following condition:

̂
𝑐𝑢𝑟𝐴𝑛𝑔𝑙𝑒 ≤ (𝑝𝑞
̅̅̅, ̂
0𝑥) < (𝑝𝑟
̅̅̅, 0𝑥), ∀𝑟 ∈ 𝑆: 𝑟 ≠ 𝑝, 𝑟 ≠ 𝑞

Note: The value of 𝑐𝑢𝑟𝐴𝑛𝑔𝑙𝑒 is increased from 0 to 2𝜋.

 

  

Algorithm

computeAngle(point from, point to) {

angle = atan2(to.y - from.y, to.x - from.x);
if (angle < 0)
angle += 2 * ;
return angle;
}
 7

findNextExtremePoint(S, cur, curAngle) {

minAngle = 2 * ;
S \= cur;
for (each point p in S) {
angle = computeAngle(cur, p);
if (angle < minAngle && angle ≥ curAngle) {
next = p;
minAngle = angle;
}
}
S = cur;
return [next, minAngle];
}

computeConvexHull(S) {
convexHull = ;
// Let “first” be the first extreme point
convexHull = first;
curAngle = 0;
point cur = first;
while (true) {
[next, curAngle] = findNextExtremePoint(S, cur,
curAngle);
if (first == next)
break;
convexHull = next;
cur = next;
}
return convexHull;
}

What is the efficiency of this algorithm? In the worst case, the running time is
2 ).
Θ(𝑛 Otherwise?
 8

Exhaustive Search

Exhaustive search is simply a brute-force approach to combinatorial problems.

Algorithm for finding all subsets of a set of size 𝑛

for (k = 0; k < 2n; k++)
Output the binary string of length n representing k;

Algorithm for generating all permutations of a set

Permutation(pivot, a[1 .. n]) {
if (pivot == n)
Output(a);
else
for (i = pivot; i  n; i++) {
a[pivot]  a[i];
Permutation(pivot + 1, a);
a[pivot]  a[i];
}
}
Permutation(1, a);

Example: Let’s consider all permutations of four elements 𝑎1 , 𝑎2 , 𝑎3 , 𝑎4

𝑎1 , 𝑎2 , 𝑎3 , 𝑎4 𝑎2 , 𝑎1 , 𝑎3 , 𝑎4 𝑎3 , 𝑎2 , 𝑎1 , 𝑎4 𝑎4 , 𝑎2 , 𝑎3 , 𝑎1
𝑎1 , 𝑎2 , 𝑎4 , 𝑎3 𝑎2 , 𝑎1 , 𝑎4 , 𝑎3 𝑎3 , 𝑎2 , 𝑎4 , 𝑎1 𝑎4 , 𝑎2 , 𝑎1 , 𝑎3
𝑎1 , 𝑎3 , 𝑎2 , 𝑎4 𝑎2 , 𝑎3 , 𝑎1 , 𝑎4 𝑎3 , 𝑎1 , 𝑎2 , 𝑎4 𝑎4 , 𝑎3 , 𝑎2 , 𝑎1
𝑎1 , 𝑎3 , 𝑎4 , 𝑎2 𝑎2 , 𝑎3 , 𝑎4 , 𝑎1 𝑎3 , 𝑎1 , 𝑎4 , 𝑎2 𝑎4 , 𝑎3 , 𝑎1 , 𝑎2
𝑎1 , 𝑎4 , 𝑎3 , 𝑎2 𝑎2 , 𝑎4 , 𝑎3 , 𝑎1 𝑎3 , 𝑎4 , 𝑎1 , 𝑎2 𝑎4 , 𝑎1 , 𝑎3 , 𝑎2
𝑎1 , 𝑎4 , 𝑎2 , 𝑎3 𝑎2 , 𝑎4 , 𝑎1 , 𝑎3 𝑎3 , 𝑎4 , 𝑎2 , 𝑎1 𝑎4 , 𝑎1 , 𝑎2 , 𝑎3

Let’s denote 𝑇(𝑛) the number of times the basic operation must be executed to
generate all permutations of a set of 𝑛 elements. The recurrence relation is as follows:
𝑇(𝑛) = 𝑛𝑇(𝑛 − 1) + Θ(𝑛)
with the initial condition 𝑇(1) = 0.
Hint: 𝑇(𝑛) ∈ Ω(𝑛!)
 9

Traveling Salesman Problem

The problem asks to find the shortest tour through a given set of 𝑛 cities that visits
each city exactly once before returning to the city where it started.

Note: The problem can also be stated as the problem of finding the shortest Hamiltonian
circuit of a weighted graph.

Hamiltonian circuit: Given a graph of 𝑛 vertices. A Hamiltonian circuit is defined

as a sequence of 𝑛 + 1 vertices: 𝑣𝑖0 , 𝑣𝑖1 , …, 𝑣𝑖𝑛−1 , 𝑣𝑖𝑛 such that:
1. 𝑣𝑖0 ≡ 𝑣𝑖𝑛
2. ∀𝑗 ∈ [0, 𝑛 − 1]: 𝑣𝑖𝑗 is adjacent to 𝑣𝑖𝑗+1
3. ∀𝑗, 𝑘 ∈ [0, 𝑛 − 1]: 𝑣𝑖𝑗 ≠ 𝑣𝑖𝑘 iff 𝑗 ≠ 𝑘

In words, a Hamiltonian circuit is defined as a cycle that passes through all the
vertices of the graph exactly once.

Idea: With no loss of generality, we can assume that all circuits start and end at one
particular vertex 𝑣𝑖0 . Thus, we can get (𝑛 − 1)! potential tours by generating all the
permutations of 𝑛 − 1 intermediate cities and attach 𝑣𝑖0 to the beginning and the end of
each permutation. Then, we verify if each potential tours is a Hamiltonian circuit? If it’s
true then we compute the tour lengths, and find the shortest among them.

The time efficiency of this algoritm is Θ(𝑛!).

Sum of subsets problem

The sum of subsets problem consists of finding all subsets of a given set A =
{𝑎1 , 𝑎2 , … , 𝑎𝑛 } of 𝑛 distinct positive integers that sum to a positive integer 𝑘.

Example: If A = {3,5,6,7,8,9,10} and 𝑘 = 15, there will be more than one subset
whose sum is 15. The subsets are {3,5,7}, {7,8}, {6,9}, …

A brute force approach is to generate all subsets of the given set A and compute the
sum of each subset.
The time efficiency of this algoritm is Θ(2𝑛 ).
 10

Knapsach problem

Given 𝑛 items of known weights 𝑤1 , 𝑤2 , … , 𝑤𝑛 and values 𝑣1 , 𝑣2 , … , 𝑣𝑛 and a

knapsack of capacity 𝐶, find the most valuable subset of the items that fit into the knapsack.

The knapsack problem can also be formally stated as follows:

𝑛

𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 ∑ 𝑣𝑖 𝑥𝑖
𝑖=1
𝑛

𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ∑ 𝑤𝑖 𝑥𝑖 ≤ 𝐶 𝑤ℎ𝑒𝑟𝑒 𝑥𝑖 ∈ {0, 1}, 𝑖 = 1, . . , 𝑛

𝑖=1

A simplistic approach to solving this problem would be to enumerate all subsets of

the 𝑛 items, and select the one that satisfies the constraints and maximizes the profits.
The obvious problem with this strategy is the running time which is at least Θ(2𝑛 )
corresponding to the power set of 𝑛 items.

Assignment problem

There are 𝑛 people who need to be assigned to execute 𝑛 jobs, one person per job.
The cost if the 𝑖 𝑡ℎ person is assigned to the 𝑗 𝑡ℎ job is a known quantity 𝐶𝑖,𝑗 for each pair
𝑖, 𝑗 = 1,2, … , 𝑛. The problem is to find an assignment with the minimum total cost.

Example:
𝐶 Job 1 Job 2 Job 3 Job 4
Person 1 9 2 7 8
Person 2 6 4 3 7
Person 3 5 8 1 8
Person 4 7 6 9 4

We can describe feasible solutions to the assignment problem as 𝑛-tuples

〈𝑗1 , 𝑗2 , … , 𝑗𝑛 〉: ∀𝑖 ∈ [1, 𝑛], 𝑗𝑖 ∈ [1, 𝑛] in which the 𝑖𝑡ℎ component (𝑗𝑖 ) indicates the column
of the element selected in the 𝑖𝑡ℎ row.

Example: 〈2,4,1,3〉 indicates the assignment of person 1 to job 2, person 2 to job 4,

person 3 to job 1, and person 4 to job 3.
 11

The exhaustive-search approach to this problem would require generating all the
permutations of integers 1,2, … , 𝑛, computing the total cost of each assignment by
summing up the corresponding elements of the cost matrix, and finally selecting the one
with the smallest sum.

Of course, the running time of this approach is Θ(𝑛!).

 1

Chapter 4: Backtracking

Introduction

Backtracking is a more intelligent variation of the exhaustive-search technique. This

approach makes it possible to solve some large instances of difficult combinatorial
problems, though, in the worst case, we still face the same curse of exponential explosion
encountered in exhaustive search.
The principal idea of backtracking is to construct solutions one component at a time
and if no potential values of the remaining components can lead to a solution, the remaining
components are not generated at all.

The state-space tree

Backtracking is based on the construction of a state-space tree whose nodes reflect

specific choices made for a solution’s components.
The root of this tree represents an initial state before the search for a solution begins.
The nodes of the first level in the tree represent the choices made for the first component
of a solution, the nodes of the second level represent the choices for the second component,
and so on. A node in a state-space tree is said to be promising if it corresponds to a partially
constructed solution that may still lead to a complete solution; otherwise, it is called
nonpromising. Leaves represent either nonpromising dead ends or complete solutions
found by the algorithm.
Backtracking technique terminates a node as soon as it can be guaranteed that no
solution to the problem can be obtained by considering choices that correspond to the
node’s descendants.

Backtracking algorithm – the first version

Backtracking(u) {
if (promising(u))
if (there is a solution at u)
Output the solution;
else
for (each child v of u)
Backtracking(v);
}
 2

Backtracking algorithm – the second version

Backtracking(u) {
for (each child v of u)
if (promising(v))
if (there is a solution at v)
Output the solution;
else
Backtracking(v);
}

The 𝒏-Queens problem

The 𝑛-Queens is the problem of placing 𝑛 chess queens on an 𝑛 × 𝑛 chessboard so

that no two queens attack each other.

(a) (h)
1,1 1,2

(b) (d) (i)

2,3 2,4 2,4

(e) (j)
3,2 3,1
(c) (g)

(k)
4,3
(f)
 3

Algorithm
promising(i) {
j = 1; flag = true;
while (j < i && flag) {
if (col[i] == col[j] || abs(col[i] - col[j]) == i - j)
flag = false;
j++;
}
return flag;
}

Version 1
n_Queens(i) {
if (promising(i))
if (i == n)
print(col[1 .. n]);
else
for (j = 1; j ≤ n; j++) {
col[i + 1] = j;
n_Queens(i + 1);
}
}
n_Queens(0);

Version 2
n_Queens(i) {
for (j = 1; j ≤ n; j++) {
col[i] = j;
if (promising(i))
if (i == n)
print(col[1 .. n]);
else
n_Queens(i + 1);
}
}
n_Queens(1);
 4

The Knight’s tour problem

A knight is placed on the first cell 〈𝑟0 , 𝑐0 〉 of an empty board of the size 𝑛 × 𝑛 and,
moving according to the rules of chess, must visit each cell exactly once.

Note: Numbers in cells indicate move number of knight.

 4  3  -2
5 2 -1
  0
6 1 1
 7 0  2
-2 -1 0 1 2

Algorithm
KnightTour(i, r, c) {
for (k = 1; k ≤ 8; k++) {
u = r + row[k];
v = c + col[k];

if ((1  u, v  n) && (cb[u][v] == 0)) {

cb[u][v] = i;

if (i == n2)
print(h);
else
KnightTour(i + 1, u, v);

cb[u][v] = 0;
}
}
}

cb[r0][c0] = 1;
KnightTour(2, r0, c0);
 5

Maze problem

A robot is asked to navigate a maze. It is placed at a certain position (the starting

position) in the maze and is asked to try to reach another position (the goal position).
Positions in the maze will either be open or blocked with an obstacle. Of course, the robot
can only move to positions without obstacles and must stay within the maze.
At any given moment, the robot can only move 1 step in one of 4 directions: North,
East, South, and West.
The robot should search for a path from the starting position to the goal position
(a solution path) until it finds one or until it exhausts all possibilities. In addition, it should
mark the path it finds (if any) in the maze.

# S # # . # North


# . . . . #
. # . # . #
. . . . # # West East
. . # # . G
# . . . . #
# # # . # # South
(a)
# S # # . # # S # # . #
#     # #   . . #
. # . # . # . #  # . #
. . . . # # .   . # #
. . # # . G .  # #  G
# . . . . # #     #
# # # . # # # # # . # #
(b) (c)

1 2 3 4 5 6
#### #### #### #### #### ####
##..# ##..# ##..# ##..# ##..# ##..#
##..# ##..# ##..# ##..# ##..# ##..#
##.# ##.# ##.# ##.# ##.# #.#.#
###... ###... ###... ###... ###... ###...
G...## G...## G...## G...## G...## G...##
 6

Algorithm
bool Find_Path(r, c) {
if ((r, c)  Maze)
return false;
if (Maze[r][c] == ‘G’)
return true;
if (Maze[r][c] == ‘’)
return false;
if (Maze[r][c] == ‘#’)
return false;

Maze[r][c] = ‘’;

if (Find_Path(r - 1, c) == true)
return true;
if (Find_Path(r, c + 1) == true)
return true;
if (Find_Path(r + 1, c) == true)
return true;
if (Find_Path(r, c - 1) == true)
return true;

Maze[r][c] = ‘.’;
return false;
}
Find_Path(r0, c0);
 7

Hamiltonian Circuit Problem

Algorithm
bool promising(int pos, int v) {
if (pos == n && G[v][path[1]] == 0) // (3)
return false;
else
if (G[path[pos - 1]][v] == 0) // (2)
return false;
else
for (int i = 1; i < pos; i++) // (4)
if (path[i] == v)
return false;
return true;
}
Hamiltonian(bool G[1..n][1..n], int path[1..n], int pos) {
if (pos == n + 1)
print(path);
else
for (v = 1; v  n; v++)
if (promising(pos, v)) {
path[pos] = v;
Hamiltonian(G, path, pos + 1);
}
}
path[1 .. n] = -1;
path[1] = 1;
Hamiltonian(G, path, 2);
 8

Sum of Subsets Problem

Find a subset of a given set W = {𝑤1 , 𝑤2 , … , 𝑤𝑛 } of 𝑛 positive integers whose sum

is equal to a given positive integer 𝑡.

Note: It is convenient to sort the set’s elements in increasing order. So, we will assume that
𝑤1 < 𝑤2 < ⋯ < 𝑤𝑛

The first approach

The solution 𝑆 is a vector of the size 𝑛: {𝑠1 , 𝑠2 , … , 𝑠𝑛 } where 𝑠𝑖 ∈ {0,1}. For each
𝑖 ∈ {1,2, … , 𝑛}, the value of 𝑠𝑖 indicates whether 𝑤𝑖 is in the subset or not.

Example: W = {3,5,6,7} và 𝑡 = 15.

Level 0 0
+3 +0

Level 1 3 0

+5 +0 +5 +0

Level 2 8 3 5 0

+6 +0 +6 +0 +6 +0 

Level 3 14 8 9 3 11 5
 +7
   
+0

Level 4 15 8
✓ 

Algorithm
bool s[1 .. n] = {false};
total = ∑𝑛𝑖=1 𝑤[𝑖];
sort(w);
if (w[1]  t  total)
SoS(1, 0, total, w, s);
…
 9

SoS(k, sum, total, w[1 .. n], s[1 .. n]) {

if (sum == t)
print(s);
else
if ((sum + total  t) && (sum + w[k]  t)) {
s[k] = true;
SoS(k + 1, sum + w[k], total - w[k], w, s);
s[k] = false;
SoS(k + 1, sum, total - w[k], w, s);
}
}

The second approach

The solution 𝑆 is the set of selected items.

Assume that initially the given set has 4 items W = {𝑤1 , 𝑤2 , 𝑤3 , 𝑤4 }. The state-space
tree will be constructed as follows:

0
+ w1 + w4
+ w2 + w3

1 2 3 4
+ w2 + w4 + w3 + w4 + w4
+ w3

5 6 7 8 9 10

+ w3 + w4 + w4 + w4

11 12 13 14

+ w4

15
 10

Algorithm
SoS(s[1 .. n], size, sum, start) {
if (sum == t)
print(s, size);
else
for (i = start; i  n; i++) {
s[size] = w[i];
SoS(s, size + 1, sum + w[i], i + 1);
}
}
s[1 .. n] = {0};
total = ∑𝑛𝑖=1 𝑤[𝑖];
if (min(w)  t && t  total)
SoS(s, 1, 0, 1);

Algorithm (upgraded version)

SoS(s[1 .. n], size, sum, start, total) {
if (sum == t)
print(s, size);
else {
lost = 0;
for (i = start; i  n; i++) {
if ((sum + total – lost ≥ t) && (sum + w[i] ≤ t)) {
s[size] = w[i];
SoS(s, size+1, sum+w[i], i+1, total–lost - w[i]);
}
lost += w[i];
}
}
}
s[1 .. n] = {0};
total = ∑𝑛𝑖=1 𝑤[𝑖];
sort(w);
if (w[1]  t  total)
SoS(s, 1, 0, 1, total);
 1

Chapter 5: Divide-and-Conquer

Introduction

Divide-and-conquer is probably the best known general algorithm design technique.

Divide-and-conquer algorithms work according to the following general plan:
Step 1. A problem is divided into several subproblems of the same type, ideally of
about equal size.
Step 2. The subproblems are solved (typically recursively, though sometimes a
different algorithm is employed, especially when subproblems become small
enough).
Step 3. The solutions to the subproblems are combined to get a solution to the original
problem.

Example: Finding the maximum value from an array of 𝑛 numbers (for simplicity,
𝑛 is a power of 2).

The general divide-and-conquer recurrence

In the most typical case of divide-and-conquer, a problem’s instance of size 𝑛 is

divided into 𝑎(> 1) instances of size 𝑛⁄𝑏 where 𝑏 > 1. For simplicity, assuming that size
𝑛 is a power of 𝑏; we get the following recurrence for the running time 𝑇(𝑛):

𝑇(𝑛) = 𝑎𝑇(𝑛⁄𝑏) + 𝑓(𝑛)

where 𝑓(𝑛) is a function that accounts for the time spent on dividing an instance of size 𝑛
into instances of size 𝑛⁄𝑏 and combining their solutions. This recurrence is called the
general divide-and-conquer recurrence.

The efficiency analysis of many divide-and-conquer algorithms is greatly simplified

by the following theorem:
Master theorem: Given the divide-and-conquer recurrence 𝑇(𝑛) = 𝑎𝑇(𝑛⁄𝑏) + 𝑓(𝑛). If
𝑓(𝑛) ∈ Θ(𝑛𝑑 ) where 𝑑 ≥ 0 then:
Θ(𝑛𝑑 ) 𝑎 < 𝑏𝑑
𝑇(𝑛) ∈ {Θ(𝑛𝑑 log 𝑛) 𝑎 = 𝑏 𝑑
Θ(𝑛log𝑏 𝑎 ) 𝑎 > 𝑏 𝑑
Analogous results hold for the Ο and  notations, too.
 2

Example: Finding the maximum value from an array of 𝑛 numbers (for simplicity,
𝑘
𝑛 = 2 ).
findMax(a, l, r) {
if (l == r) return a[l];
m = (l + r) / 2;
return max(findMax(a, l, m), findMax(a, m + 1, r));
}
The divide-and-conquer recurrence is as follows:
𝑛
2𝑇 ( ) + Θ(1) 𝑛 > 1
𝑇(𝑛) = { 2
0 𝑛=1

Example: Finding simultaneously the maximum and minimum values from an array
of 𝑛 numbers.

Algorithm
MinMax(l, r, & min, & max) {
if (l ≥ r - 1)
if (a[l] < a[r]) {
min = a[l];
max = a[r];
}
else {
min = a[r];
max = a[l];
}
else {
m = (l + r) / 2;
MinMax(l, m, minL, maxL);
MinMax(m + 1, r, minR, maxR);
min = (minL < minR) ? minL : minR;
max = (maxL < maxR) ? maxR : maxL;
}
}

The divide-and-conquer recurrence is as follows:

𝑛 𝑛
𝐶 (⌊ ⌋) + 𝐶 (𝑛 − ⌊ ⌋) + 2 𝑛>2
𝐶(𝑛) = { 2 2
1 𝑛≤2
 3

Mergesort

This approach sorts a given array 𝑎1 , 𝑎2 , … , 𝑎𝑛 by dividing it into two halves:

𝑎1 , 𝑎2 , … , 𝑎⌊𝑛⌋
2
𝑎⌊𝑛⌋+1 , 𝑎⌊𝑛⌋+2 , … , 𝑎𝑛
2 2
sorting each of them recursively, and then merging the two smaller sorted arrays into a
single sorted one.

32749168

3274 9168
Tách

32 74 91 68

3 2 7 4 9 1 6 8

23 47 19 68
Trộn

2347 1689

12346789

Algorithm
mergeSort(a[1 .. n], low, high) {
if (low < high) {
mid = (low + high) / 2;
mergeSort(a, low, mid);
mergeSort(a, mid + 1, high);
merge(a, low, mid, high);
}
}
 4

merge(a[1 .. n], low, mid, high) {

i = low;
j = mid + 1;
k = low;
while (i  mid) && (j  high)
if (a[i]  a[j])
buf[k ++] = a[i ++];
else
buf[k ++] = a[j ++];

if (i > mid)
buf[k .. high] = a[j .. high];
else
buf[k .. high] = a[i .. mid];

a[low .. high] = buf[low .. high];

}
mergeSort(a, 1, n);

How efficient is mergesort?

• Assuming that the key comparison is the basic operation:

In the best case:
𝑛 𝑛 𝑛
𝑇 (⌊ ⌋) + 𝑇 (⌈ ⌉) + ⌊ ⌋ 𝑛>1
𝑇(𝑛) = { 2 2 2
0 𝑛=1
Hint: 𝑇(𝑛) ∈ Θ(𝑛 log 𝑛)

In the worst case:

𝑛 𝑛
𝑇 (𝑛 ) = {
𝑇 (⌊ ⌋) + 𝑇 (⌈ ⌉) + (𝑛 − 1) 𝑛 > 1
2 2
0 𝑛=1
Hint: 𝑇(𝑛) ∈ Θ(𝑛 log 𝑛)
• Assuming that the assignment statement is the basic operation:
𝑛 𝑛
𝑀 (⌊ ⌋) + 𝑀 (⌈ ⌉) + 𝑛 𝑛 > 1
𝑀(𝑛) = { 2 2
0 𝑛=1
Hint: 𝑀(𝑛) ∈ Θ(𝑛 log 𝑛)
 5

Quicksort

Unlike mergesort, which divides its input elements according to their position in the
array, quicksort divides them according to their value. This process is called partition.
A partition is an arrangement of the array’s elements so that all the elements to the
left of some element 𝑎𝑠 are less than or equal to 𝑎𝑠 , and all the elements to the right of 𝑎𝑠
are greater than or equal to it:
{𝑎1 … 𝑎𝑠−1 } ≤ 𝑎𝑠 ≤ {𝑎𝑠+1 … 𝑎𝑛 }
After a partition is achieved, 𝑎𝑠 will be in its final position in the sorted array, and
we can continue sorting the two subarrays to the left and to the right of 𝑎𝑠 independently
by the same method.

Algorithm
Quicksort(a[left .. right]) {
if (left < right){
s = Partition(a[left .. right]);
Quicksort(a[left .. s – 1]);
Quicksort(a[s + 1 .. right]);
}
}
Partition(a[left .. right]) {
p = a[left];
i = left;
j = right + 1;
do {
do i++; while (a[i] < p);
do j--; while (a[j] > p);
swap(a[i], a[j]);
} while (i < j);

swap(a[i], a[j]);
swap(a[left], a[j]);

return j;
}

Does this design work?

 6

Analysis of Quicksort

For simplicity, assuming that the sequence 𝑎1 , 𝑎2 , … , 𝑎𝑛 contains no duplicate

values and the size 𝑛 is a power of 2: 𝑛 = 2𝑘 . Two comparisons in loops are the basic
operation.

In the best case:

𝐶𝑏 (𝑛) ∈ Θ(𝑛 log 𝑛)

In the worst case:

𝐶𝑤 (𝑛) ∈ Θ(𝑛2 )

In the average case:

𝐶(𝑛) ≈ 1.39𝑛 log 2 𝑛
 7

Multiplication of Large Integers

A simple quadratic-time algorithm for multiplying large integers is one that mimics
the standard way learned in school. We will develop one that is better than quadratic time.

The basic idea: Observing the multiplication of two complex numbers

(𝑎 + 𝑏𝑖)(𝑐 + 𝑑𝑖) = (𝑎𝑐 − 𝑏𝑑) + (𝑏𝑐 + 𝑎𝑑)𝑖
K. F. Gauss perceived that:
𝑏𝑐 + 𝑎𝑑 = (𝑎 + 𝑏)(𝑐 + 𝑑) − (𝑎𝑐 + 𝑏𝑑)

We assume that the data type large_integer representing a large integer was
constructed. It is not difficult to write linear-time algorithms for three operations:
mul 10m, div 10m, and mod 10m.
Let’s consider the algorithm that implements the multiplication of two large
integers: 𝑢 × 𝑣
Algorithm
large_integer MUL(large_integer u, v) {
large_integer x, y, w, z;

n = max(number of digits in u, number of digits in v);

if (u == 0 || v == 0)
return 0;
else
if (n  )
return u × v; // built-in operator
else {
m = n / 2;
x = u div 10m; y = u mod 10m;
w = v div 10m; z = v mod 10m;
return MUL(x, w) mul 102m +
(MUL(x, z) + MUL(y, w)) mul 10m +
MUL(y, z);
}
}
 8

Analysis of the algorithm

The divide-and-conquer recurrence is as follows:
𝑛
4𝑇 ( ) + Θ(𝑛) 𝑛 > 𝛼
𝑇(𝑛) = { 2
1 𝑛≤𝛼
The Master theorem implies that 𝑇(𝑛) ∈ Θ(𝑛2 ).

Algorithm (upgraded version)

large_integer MUL(large_integer u, v, n) {
n = max(number of digits in u, number of digits in v);

if (u == 0 || v == 0)
return 0;
else
if (n  )
return u × v;
else {
m = n / 2;
x = u div 10m; y = u mod 10m;
w = v div 10m; z = v mod 10m;
r = MUL(x + y, w + z);
p = MUL(x, w);
q = MUL(y, z);

return p mul 102m + (r – p – q) mul 10m + q;

}
}
In this case, the divide-and-conquer recurrence is as follows:
𝑛
3𝑇 ( ) + Θ(𝑛) 𝑛 > 𝛼
𝑇(𝑛) = { 2
1 𝑛≤𝛼

Since 𝑑 = 1, 𝑎 = 3, 𝑏 = 2, the Master theorem implies that 𝑇(𝑛) ∈ Θ(𝑛log2 3 ) ≈

Θ(𝑛1.585 ).
 9

Extension: Multiplication of two positive integers of 𝑛 bits. Assuming that 𝑛 is the power
of 2.

Let’s 𝑥 and 𝑦 be two positive integers of 𝑛 bits. Obviously:

𝑥 = 2𝑛/2 𝑥𝐿 + 𝑥𝑅
𝑦 = 2𝑛/2 𝑦𝐿 + 𝑦𝑅
where 𝑥𝐿 , 𝑥𝑅 are two positive integers represented by 𝑛⁄2 leftmost bits and 𝑛⁄2 rightmost
bits of 𝑥 , respectively; similarly, 𝑦𝐿 , 𝑦𝑅 are two positive integers represented by 𝑛⁄2
leftmost bits and 𝑛⁄2 rightmost bits of 𝑦, respectively.

Example: Given 𝑥 = 13510 = 100001112. Then,

𝑥𝐿 = 810 (= 10002 )
𝑥𝑅 = 710 (= 01112 )
𝑥 = 2𝑛/2 𝑥𝐿 + 𝑥𝑅 = 28/2 × 810 + 710

Now, we get:
𝑥 × 𝑦 = (2𝑛/2 𝑥𝐿 + 𝑥𝑅 ) × (2𝑛/2 𝑦𝐿 + 𝑦𝑅 ) = 2𝑛 × 𝑥𝐿 𝑦𝐿 + 2𝑛/2 × (𝑥𝐿 𝑦𝑅 + 𝑥𝑅 𝑦𝐿 ) + 𝑥𝑅 𝑦𝑅
Algorithm
int multiply(x, y) {
n = max(|x| , |y| );
bit bit

if (n  ) return x × y;
xL = n / 2 leftmost bits of x;
xR = n / 2 rightmost bits of x;
yL = n / 2 leftmost bits of y;
yR = n / 2 rightmost bits of y;

r = multiply(xL + xR, yL + yR);

p = multiply(xL, yL);
q = multiply(xR, yR);

return p × 2n + (r - p - q) × 2n/2 + q;
}
 10

Strassen’s Matrix Multiplication

Suppose we want the product 𝐶 of two 2 × 2 matrices, 𝐴 and 𝐵. That is,

𝑐11 𝑐12 𝑎11 𝑎12 𝑏11 𝑏12
[𝑐
21 𝑐22 ] = [𝑎21 𝑎22 ] × [𝑏21 𝑏22 ]
𝑎 × 𝑏11 + 𝑎12 × 𝑏21 𝑎11 × 𝑏12 + 𝑎12 × 𝑏22
= [ 11 ]
𝑎21 × 𝑏11 + 𝑎22 × 𝑏21 𝑎21 × 𝑏12 + 𝑎22 × 𝑏22
Of course, the time complexity of this straightforward method is 𝑇(𝑛) = 𝑛3, where
𝑛 is the number of rows and columns in the matrices. To be specific, the above matrix
multiplication requires eight multiplications and four additions.
However, Strassen determined that if we let
𝑚1 = (𝑎11 + 𝑎22 ) × (𝑏11 + 𝑏22 )
𝑚2 = (𝑎21 + 𝑎22 ) × 𝑏11
𝑚3 = 𝑎11 × (𝑏12 − 𝑏22 )
𝑚4 = 𝑎22 × (𝑏21 − 𝑏11 )
𝑚5 = (𝑎11 + 𝑎12 ) × 𝑏22
𝑚6 = (𝑎21 − 𝑎11 ) × (𝑏11 + 𝑏12 )
𝑚7 = (𝑎12 − 𝑎22 ) × (𝑏21 + 𝑏22 )
the product 𝐶 is given by
𝑐11 𝑐12 𝑚1 + 𝑚4 − 𝑚5 + 𝑚7 𝑚3 + 𝑚5
[𝑐
21 𝑐22 ] = [ 𝑚2 + 𝑚4 𝑚1 + 𝑚3 − 𝑚2 + 𝑚6 ]
Strassen’s method requires seven multiplications and 18 additions/subtractions.
Thus, we have saved ourselves one multiplication at the expense of doing 14 additional
additions or subtractions.
Let 𝐴 and 𝐵 be matrices of size 𝑛 × 𝑛, where 𝑛 = 2𝑘 . Let 𝐶 be the product of 𝐴 and
𝐵. Each of these matrices is divided into four submatrices as follows:
𝐶 𝐶12 𝐴 𝐴12 𝐵 𝐵12
[ 11 ] = [ 11 ] × [ 11 ]
𝐶21 𝐶22 𝐴21 𝐴22 𝐵21 𝐵22
where
𝑐1,1 ⋯ 𝑐1,𝑛 𝑐1,𝑛+1 ⋯ 𝑐1,𝑛
2 2
𝐶11 =[ ⋮ ⋱ ⋮ ] 𝐶12 = [ ⋮ ⋱ ⋮ ]
𝑐𝑛,1 ⋯ 𝑐𝑛,𝑛 𝑐𝑛,𝑛+1 ⋯ 𝑐𝑛,𝑛
2 22 22 2
𝑐𝑛+1,1 ⋯ 𝑐𝑛+1,𝑛 𝑐𝑛+1,𝑛+1 ⋯ 𝑐𝑛+1,𝑛
2 2 2 2 2 2
𝐶21 =[ ⋮ ⋱ ⋮ ] 𝐶22 = [ ⋮ ⋱ ⋮ ]
𝑐𝑛,1 ⋯ 𝑐𝑛,𝑛 𝑐𝑛,𝑛+1 ⋯ 𝑐𝑛,𝑛
2 2
Using Strassen’s method, first we compute:
 11

𝑀1 = (𝐴11 + 𝐴22 ) × (𝐵11 + 𝐵22 )

where our operations are now matrix addition and multiplication. In the same way, we
compute 𝑀2 through 𝑀7 . Next we compute
𝐶11 = 𝑀1 + 𝑀4 − 𝑀5 + 𝑀7
and 𝐶12 , 𝐶21 , 𝐶22 . Finally, the product 𝐶 of 𝐴 and 𝐵 is obtained by combining the four
submatrices 𝐶𝑖𝑗 .
Algorithm
Strassen(n, A[1..n][1..n], B[1..n][1..n], C[1..n][1..n]) {
if (n  )
C = A × B;
else {
"Partition A into 4 submatrices A11, A12, A21, A22";
"Partition B into 4 submatrices B11, B12, B21, B22";

Strassen(n/2, A11 + A22, B11 + B22, M1);

…
Strassen(n/2, A12 – A22, B21 + B22, M7);

C11 = M1 + M4 – M5 + M7;
C12 = M3 + M5;
C21 = M2 + M4;
C22 = M1 + M3 – M2 + M6;

Combine C11, C12, C21, C22 into C;

}
}

Analysis of the algorithm

The divide-and-conquer recurrence is as follows:
𝑛 𝑛 2
𝑇(𝑛) = {7𝑇 ( ) + 18 ( ) 𝑛>𝛼
2 2
1 𝑛≤𝛼
The Master theorem implies that: 𝑇(𝑛) ∈ Θ(𝑛log2 7 ) ≈ Θ(𝑛2.81 )
 12

Find the substring with largest sum of elements in an array

 
Algorithm

sumMax(a[1..n], l, r) {
if (l == r) return max(a[l], 0);

c = (l + r) / 2;
maxLS = sumMax(a, l, c);
maxRS = sumMax(a, c + 1, r);

tmp = maxLpartS = 0;
for (i = c; i  l; i--) {
tmp += a[i];
if (tmp > maxLpartS) maxLpartS = tmp;
}
tmp = maxRpartS = 0;
for (i = c + 1; i  r; i++) {
tmp += a[i];
if (tmp > maxRpartS) maxRpartS = tmp;
}
tmp = maxLpartS + maxRpartS;
return max(tmp, maxLS, maxRS);
}
max = sumMax(a, 1, n);

Analysis of the algorithm

The divide-and-conquer recurrence is as follows:
𝑛 𝑛
𝑇 (⌊ ⌋) + 𝑇 (⌈ ⌉) + Θ(𝑛) 𝑛 > 1
𝑇(𝑛) = { 2 2
0 𝑛=1
Hint: 𝑇(𝑛) ∈ Θ(𝑛 log 𝑛)
 13

Closest-Pair Problem
Let 𝑃 be a list of 𝑛 > 1 points in the Cartesian plane: 𝑃 = {𝑝1 , 𝑝2 , … , 𝑝𝑛 }. Find
a pair of points with the smallest distance between them.
For the sake of simplicity and without loss of generality, we can assume that the
points in 𝑃 are ordered in nondecreasing order of their 𝑥 coordinate. In addition, let 𝑄 be a
list of all and only points in 𝑃 sorted in nondecreasing order of the 𝑦 coordinate.
If 2 ≤ 𝑛 ≤ 3, the problem can be solved by the obvious brute-force algorithm.
Besides, 𝑛 = 2,3 is also the stopping condition of the recursive process.
𝑛 𝑛
If 𝑛 > 3, we can divide the points into two subsets 𝑃𝐿 and 𝑃𝑅 of ⌈2 ⌉ and ⌊2 ⌋ points,
respectively, by drawing a vertical line through the median of their 𝑥 coordinates so that
𝑛 𝑛
⌈2 ⌉ points lie to the left of or on the line itself, and ⌊2 ⌋ points lie to the right of or on the
line . Then we can solve the closest-pair problem recursively for subsets 𝑃𝐿 and 𝑃𝑅 . Let
𝛿𝐿 and 𝛿𝑅 be the smallest distances between pairs of points in 𝑃𝐿 and 𝑃𝑅 , respectively, and
let 𝛿 = min{𝛿𝐿 , 𝛿𝑅 }.

S 

R

L

 

Note that 𝛿 is not necessarily the smallest distance between all the point pairs
because points of a closer pair can lie on the opposite sides of the separating line .
Therefore, we need to examine such points. Obviously, we can limit our attention to the
points inside the symmetric vertical strip of width 2𝛿 around the separating line , since
the distance between any other pair of points is at least 𝛿.
Algorithm
 14

ClosestPair(Point P[1..n], Point Q[1..n]) {

if (|P|  3)
return the minimal distance found by the brute-force
algorithm;

= P[n/2].x;

Copy the first n/2 points of P to PL;

Copy the same n/2 points from Q to QL;
Copy the remaining n/2 points of P to PR;
Copy the same n/2 points from Q to QR;

L = ClosestPair(PL, QL);
R = ClosestPair(PR, QR);
 = min(L, R);

Copy all the points p of Q for which |p.x - | <  into

S[1..k];
min = ;
for (i = 1; i < k; i++) {
j = i + 1;
while (j  k) && (|S[i].y – S[j].y| < min) {
min = min(√(𝑆[𝑖]. 𝑥 − 𝑆[𝑗]. 𝑥)2 + (𝑆[𝑖]. 𝑦 − 𝑆[𝑗]. 𝑦)2, min)
j++;
}
}

return min;
}

Analysis of the algorithm

The divide-and-conquer recurrence is as follows:
𝑛
𝑇(𝑛) = 2𝑇 ( ) + Θ(𝑛) ∈ Θ(𝑛 log 𝑛)
2
 15

The change-making problem

Given 𝑘 denominations: 𝑑1 < 𝑑2 < ⋯ < 𝑑𝑘 where 𝑑1 = 1 . Find the minimum

number of coins (of certain denominations) that add up to a given amount of money 𝑛.

Algorithm
moneyChange(d[1..k], money) {
for (i = 1; i  k; i++)
if (d[i] == money)
return 1;

minCoins = money;
for (i = 1; i  money / 2; i++) {
tmpSum = moneyChange(d, i) + moneyChange(d, money - i);
if (tmpSum < minCoins)
minCoins = tmpSum;
}
return minCoins;
}

Analysis of the algorithm

The divide-and-conquer recurrence is as follows:
⌊𝑛⁄2⌋
𝑛
∑ (𝑇(𝑖) + 𝑇(𝑛 − 𝑖)) + Θ ( ) 𝑛 > 2
𝑇(𝑛) = 2
𝑖=1
1 𝑛=2
{ 0 𝑛=1
Hint: 𝑇(𝑛) ∈ Ω(2𝑛 )
 16

Algorithm (upgraded version)

moneyChange(d[1..k], money) {
for (i = 1; i  k; i++)
if (d[i] == money)
return 1;

minCoins = money;
for (i = 1; i  k; i++)
if (money > d[i]) {
tmpSum = 1 + moneyChange(d, money - d[i]);
if (tmpSum < minCoins)
minCoins = tmpSum;
}
return minCoins;
}