Algorithm Analysis &

Types of Algorithms

Dr. Abdul Majid

[email protected]

1
 Data Structures & Algorithms
Writing algorithms

 There are no well-defined standards for writing

algorithms.
 It is problem and resource dependent.
Algorithms are never written to support a
particular programming code.
 As we know that all programming languages
share basic code constructs like loops (do, for,
while), flow-control (if-else), etc. These common
constructs can be used to write an algorithm.

2
 Data Structures & Algorithms
Writing Algorithms

 Write algorithms in a step-by-step manner, but it

is not always the case.

 Algorithm writing is a process and is executed

after the problem domain is well-defined.

 Algorithm is designed to get a solution of a given

problem. A problem can be solved in more than
one ways

3
 Data Structures & Algorithms
Writing Algorithms
 Problem − Design an algorithm to add two numbers and
display the result.

Step 1 − START
Step 2 − declare three integers a, b & c
Step 3 − define values of a & b
Step 4 − add values of a & b
Step 5 − store output of step 4 to c
Step 6 − print c
Step 7 − STOP

4
 Data Structures & Algorithms
Writing Algorithms
 Algorithms tell the programmers how to code the
program. Alternatively, the algorithm can be written as −

Step 1 − START ADD

Step 2 − get values of a & b
Step 3 − c ← a + b
Step 4 − display c
Step 5 − STOP

5
 Data Structures & Algorithms
Analysis of Algorithms

 Asymptotic Analysis
 Asymptotic analysis of an algorithm refers to
defining the mathematical framing of its run-time
performance.
 Asymptotic analysis is input bound i.e., if there's
no input to the algorithm, it is concluded to work
in a constant time. Other than the "input" all
other factors are considered constant.

6
 Data Structures & Algorithms
Analysis of Algorithms

 Asymptotic Analysis
 Asymptotic analysis refers to computing the
running time of any operation in mathematical
units of computation.
 For example, the running time of one operation is computed
as f(n) and may be for another operation it is computed as g(n2).
 This means that
 First operation running time will increase linearly with the
increase in n
 the running time of the second operation will increase
exponentially when n increases.

7
 Data Structures & Algorithms
Analysis of Algorithms

 Asymptotic Analysis
 Usually, the time required by an algorithm falls under
three types −
 Best Case − Minimum time required for program execution.
 Average Case − Average time required for program execution.
 Worst Case − Maximum time required for program execution.
 Asymptotic Notations
 Following are the commonly used asymptotic notations
to calculate the running time complexity of an algorithm.
 Ο Notation (big O notation)
 Ω Notation (Omega notation)
 θ Notation (Theta notation)

8
 Data Structures & Algorithms
Analysis of Algorithms
 Reading Assignment : Calculations of running time
complexity of algorithms

9
 Data Structures & Algorithms
Types of algorithms

 An algorithm is designed to achieve

optimum solution for a given problem.
 A basic solution that satisfies all the constraints
defining or in other words, one that lies within is called
a basic feasible solution.
 An optimal solution is a feasible solution where the
objective function reaches its maximum (or minimum)
value – for example, the most profit or the least cost.
 A globally optimal solution is one where there are
no other feasible solutions with better objective
function values.

10
 Data Structures & Algorithms
Types of algorithms

 Greedy Algorithms
 Divide and Conquer Algorithms
 Dynamic Programming

11
 Data Structures & Algorithms
Types of algorithms

 Greedy Algorithms
 In greedy algorithm approach:
 Decisions are made from the given solution domain.
As being greedy, the closest solution that seems to
provide an optimum solution is chosen.
 Greedy algorithms try to find a localized optimum
solution
 Which may eventually lead to globally optimized
solutions. However, generally greedy algorithms do
not provide globally optimized solutions.

12
 Data Structures & Algorithms
Types of algorithms

 Greedy Algorithms – Counting Currency

 This problem is to count to a desired value by choosing
the least possible coins and the greedy approach forces
the algorithm to pick the largest possible coin. If we are
provided coins of ₹ 1, 2, 5 and 10 and we are asked to
count ₹ 18 then the greedy procedure will be −

 1 − Select one ₹ 10 coin, the remaining count is 8

 2 − Then select one ₹ 5 coin, the remaining count is 3
 3 − Then select one ₹ 2 coin, the remaining count is 1
 4 − And finally, the selection of one ₹ 1 coins solves the problem

13
 Data Structures & Algorithms
Types of algorithms

 Greedy Algorithms – Counting Currency

 For the currency system, If we have coins of 1,

7, 10 value, counting coins for value 18 will be
absolutely optimum but for count like 15, it may
use more coins than necessary. For example,
the greedy approach will use 10 + 1 + 1 + 1 + 1
+ 1, total 6 coins. Whereas the same problem
could be solved by using only 3 coins (7 + 7 + 1)

14
 Data Structures & Algorithms
Types of algorithms

 Greedy Algorithms – Examples

 Most networking algorithms use the greedy approach.
Here is a list of few of them −
 Travelling Salesman Problem

 Prim's Minimal Spanning Tree Algorithm

 Kruskal's Minimal Spanning Tree Algorithm

 Dijkstra's Minimal Spanning Tree Algorithm

 Graph - Map Coloring

 Graph - Vertex Cover

 Knapsack Problem

 Job Scheduling Problem

15
 Data Structures & Algorithms
Types of algorithms

 Divide and Conquer Algorithms

 In divide and conquer approach, the problem in
hand, is divided into smaller sub-problems and
then each problem is solved independently.
 The solution of all sub-problems is finally merged
in order to obtain the solution of an original
problem.
 Examples
 Merge Sort, Quick Sort, Binary Search,
Strassen's Matrix Multiplication

16
 Data Structures & Algorithms
Types of algorithms

 Divide and Conquer Algorithms – Example

Merge Sort
 Merge sort is a sorting algorithm. The
algorithm roughly works as follows:
 Divide the sequence of n numbers into 2

halves
 Recursively sort the two halves

 Merge the two sorted halves into a

single sorted sequence

17
 Data Structures & Algorithms
Types of algorithms

 Divide and Conquer Algorithms – Example

Merge Sort

18
 Data Structures & Algorithms
Types of algorithms

 Divide and Conquer Algorithms – Example

Merge Sort

19
 Data Structures & Algorithms
Types of algorithms

 Dynamic Programming
 a method for solving a complex problem by breaking it
down into a collection of simpler subproblems, solving
each of those subproblems just once, and storing their
solutions
 the next time the same subproblem occurs, instead of
recomputing its solution, one simply looks up the
previously computed solution, thereby saving
computation time. This technique of storing solutions
to subproblems instead of recomputing them is called
memoization.

20
 Data Structures & Algorithms
Types of algorithms

 Dynamic Programming
 Here’s brilliant explanation on concept of

Dynamic Programming on Quora — Jonathan

Paulson’s answer to How should I explain
dynamic programming to a 4-year-old?
 *writes down "1+1+1+1+1+1+1+1 =" on a sheet of paper*
"What's that equal to?"
*counting* "Eight!"
*writes down another "1+" on the left*
"What about that?"
*quickly* "Nine!"
"How'd you know it was nine so fast?"
"You just added one more"
"So you didn't need to recount because you remembered there were
eight! Dynamic Programming is just a fancy way to say 'remembering
stuff to save time later'"

21
 Data Structures & Algorithms
Types of algorithms

 Dynamic Programming - Examples

 Fibonacci number series
 Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Project scheduling

22