Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Solution of Simultaneous Linear Equations

(i) Solving Small Numbers of Equations
Several methods can be used for solving small (n ≤ 3) sets of simultaneous equations
by hand (without need for a computer). These methods are:
1. Graphical method
2. Cramer’s rule
3. Inverse method for n  3
4. Elimination of unknowns

1. Graphical Method
 For two equations (n = 2):
a11 x1  a12 x2  b1
a21 x1  a22 x2  b2

 Solve both equations for x 2 : the intersection of the lines presents the solution.

a  b
x2   11  x1  1  x2  (slope)x1  intercept
 a12  a12
a  b
x2   21  x1  2
 a22  a22

 For n = 3, each equation will be a plane on a 3D coordinate system. Solution is the

point where these planes intersect.
 For n > 3, graphical solution is not practical.

Example (1): The Graphical Method for Two Equations

3x1  2x 2  18
 Solve:
- x1  2x 2  2

 Plot x2 vs. x1 , the intersection of the lines

presents the solution.

6
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

No solution Infinite solutions ill conditioned systems

(The lines are never cross) (The lines are coincident) (Slopes are too close, and sensitive
to round off errors).
Singular systems; in which the value of their determinants are zero.

2. Cramer's Rule (after Gabriel Cramer, 1750)

 This method is used to solve a system of equations in the form 𝑨 𝒙 = 𝒃 when

det.[A] ≠ 0.

 It is impractical for solution of large system of equations because, as the no. of

equations increases, the determinants are time-consuming to evaluate.

 In this method, the solution of the entire system needs:

2(𝑛 + 1)! no. of multiplications
(𝑛 + 1)! no. of additions or subtractions.
𝑛 no. of divisions

Now consider a system of (3) linear equations with (3) unknowns:

𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 = 𝑏2
𝑎31 𝑥1 + 𝑎32 𝑥2 + 𝑎33 𝑥3 = 𝑏3

In matrix form, Ax=b

where,
𝑎11 𝑎12 𝑎13 𝑥1 𝑏1
𝐴 = [𝑎21 𝑎22 𝑎23 ] , 𝑥 = {𝑥2 } , and 𝑏 = {𝑏2 }
𝑎31 𝑎32 𝑎33 𝑥3 𝑏3
7
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

 Steps of solution:
𝑎11 ⋯ 𝑎1𝑛
1. Form the coefficients matrix [𝑎𝑖𝑗 ] = [ ⋮ ⋱ ⋮ ] and the right-hand vector
𝑎𝑛1 ⋯ 𝑎𝑛𝑛
𝑏1
{𝑏𝑖 } = { ⋮ }
𝑏𝑛
2. Evaluate det.[A] using any method. If det.[A] = 0, then the set of equations has
no solution and the [A] is called singular; otherwise proceed to the next step.
3. When det.[A] ≠ 0, evaluate det.[𝐴𝑗 ] by replacing the jth. matrix with column
vector {𝑏𝑖 }.
4. Repeat step (3) for j = 1, 2, ……….., n.
5. Solve for unknowns 𝑥𝑗 by dividing det.[𝐴𝑗 ] by det.[A].

𝑑𝑒𝑡.[𝐴𝑗 ]
𝑥𝑗 = , j = 1, 2, …….…., n
𝑑𝑒𝑡.[𝐴]

Or
𝑏1 𝑎12 𝑎13 𝑎11 𝑏1 𝑎13 𝑎11 𝑎12 𝑏1
|𝑏2 𝑎22 𝑎23 | |𝑎21 𝑏2 𝑎23 | |𝑎21 𝑎22 𝑏2 |
𝑏3 𝑎32 𝑎33 𝑎31 𝑏3 𝑎33 𝑎31 𝑎32 𝑏3
𝑥1 = |𝐴|
, 𝑥2 = |𝐴|
, 𝑥3 = |𝐴|

Example (2): Use Cramer’s rule to solve:

8𝑥1 + 2𝑥2 + 3𝑥3 = 30
𝑥1 − 9𝑥2 + 2𝑥3 = 1
2𝑥1 + 3𝑥2 + 6𝑥3 = 31
8 2 3
det. [A] = |1 −9 2| = −421 ≠ 0 ∴ the system has a unique solution as:
2 3 6

30 2 3 8 30 3 8 2 30
1 9 2 1 1 2 1 9 1
31 3 6  842 2 31 6  421 2 3 31  1684
x1   2 x2   1 x3   4
 421  421  421  421  421  421

8
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Homework # 1: Use Cramer’s rule to solve:

(1) 𝑥1 + 3𝑥2 = 5
3𝑥1 − 4𝑥2 = 2 Ans. 𝑥1 = 2 and 𝑥2 = 1

(2) 𝑥 + 3𝑦 + 𝑧 = 5
2𝑥 + 𝑦 − 𝑧 = 3
𝑥 − 3𝑦 − 3𝑧 = 1 Ans. 𝑥 = −8 , 𝑦 = 8 and 𝑧 = −11
Expand determinants using pivotal condensation method.

3. Inverse Method
This method is also impractical for solution of large system of equations. First of
all, the equations to be solved must be arranged so that the diagonal coefficients are
dominant.

The matrix form is:

𝐴. 𝑥 = 𝑏…..…….………………….………………………(1)
Now if [A] is not singular [ i.e., det.[A] ≠ 0 ] , then 𝐴−1 is exist.
Pre-multiplication of Eq.(1) by 𝐴−1 :
𝐴−1 . 𝐴 𝑥 = 𝐴−1 . 𝑏
or 𝐼. 𝑥 = 𝐴−1 . 𝑏

or 𝑥 = 𝐴−1 . 𝑏 ………………………………………………(2)
where,
𝑐11 𝑐12 ⋯ 𝑐1𝑛 T
𝐴𝑑𝑗 [𝐴] 1 𝑐21 𝑐22 ⋯ 𝑐2𝑛
𝐴−1 = = [ ⋮ ⋮ ⋯ ⋮ ]
𝑑𝑒𝑡.[𝐴] 𝑑𝑒𝑡.[𝐴]
𝑐𝑛1 𝑐𝑛2 ⋯ 𝑐𝑛𝑛

and the cofactor matrix 𝐶𝑖𝑗 are given by:

𝑎11 𝑎12 ⋯ 𝑎1𝑗 ⋯ 𝑎1𝑛
𝑎21 𝑎22 ⋯ 𝑎2𝑗 ⋯ 𝑎2𝑛
| ⋮ ⋮ ⋮ ⋮ |
𝐶𝑖𝑗 = (−1)𝑖+𝑗 𝑑𝑒𝑡 ⋮ ⋮ ⋮
⋮ ⋮ ⋮ ⋮ ⋮
| |
𝑎𝑖1 𝑎𝑖2 ⋯ 𝑎𝑖𝑗 ⋯ 𝑎𝑖𝑛
𝑎𝑛1 𝑎𝑛2 ⋯ 𝑎𝑛𝑗 ⋯ 𝑎𝑛𝑛

9
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

After multiplying the minors by determinant of the elements (deleting the ith. row and jth.
column of matrix A) to obtain the cofactors. Then, transpose of cofactor matrix is called
𝑐11 𝑐21 𝑐31
Adjoint matrix: 𝐴𝑑𝑗 [𝐴] = [𝐶𝑖𝑗 ] = [𝑐12 𝑐22 𝑐32 ]
T

𝑐13 𝑐23 𝑐33

Example (3): Use matrix inversion method to solve:

𝑥1 − 9𝑥2 + 2𝑥3 = 1
8𝑥1 + 2𝑥2 + 3𝑥3 = 30
2𝑥1 + 3𝑥2 + 6𝑥3 = 31
Solution:
First, rearrange the equations so that the diagonal coefficients are dominant.
8𝑥1 + 2𝑥2 + 3𝑥3 = 30
𝑥1 − 9𝑥2 + 2𝑥3 = 1
2𝑥1 + 3𝑥2 + 6𝑥3 = 31
𝐴. 𝑥 = 𝑏
8 2 3 𝑥1 30
In matrix form: [1 −9 2] {𝑥2 } = { 1 }
2 3 6 𝑥3 31
𝐴𝑑𝑗 [𝐴]
𝐴−1 =
𝑑𝑒𝑡.[𝐴]
 First, find det.[A]:

|𝐴| = (−1)1+1 . 8 |−9 2

| + (−1)1+2 . 2 |
1 2
| + (−1)1+3 . 3 |
1 −9
| = − 421
3 6 2 6 2 3

 Find the cofactors:

−9 2 1 2 1 −9
𝑐11   | | = − 60 𝑐12   | | = −2 𝑐13   | | = 21
3 6 2 6 2 3
2 3 8 3 8 2
𝑐21   | |=−3 𝑐22   | | = 42 𝑐23   | | = − 20
3 6 2 6 2 3
2 3 8 3 8 2
𝑐31   | | = 31 𝑐32   | | = − 13 𝑐33   | | = − 74
−9 2 1 2 1 −9
−60 −2 21
∴ Matrix of cofactors [𝐶] = [ −3 42 −20]
31 −13 −74

10
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

 Find the 𝐴𝑑𝑗 [𝐴] = [𝐶]𝑇 :

−60 −3 31
𝑇
𝐴𝑑𝑗 [𝐴] = [𝐶] = [ −2 42 −13 ]
21 −20 −74
 Find 𝐴−1 :
−60 −3 31
𝐴𝑑𝑗 [𝐴] 1
𝐴−1 = = [ −2 42 −13]
𝑑𝑒𝑡.[𝐴] −421
21 −20 −74

 Check: A−1 . A = I or A. A−1 = I

 Find the values of x :

𝑥 = 𝐴−1 . 𝑏
𝑥1 −60 −3 31 30 2
1
𝑥
{ 2} = [ −2 42 −13] { 1 } = {1}
𝑥3 −421
21 −20 −74 31 4

−60(30)−3(1)+31(31)
For example: 𝑥1 = =2
−421

Homework # 2: Solve by inversion method:

(1) 𝑥1 + 𝑥2 + 𝑥3 = 0
2𝑥1 + 𝑥2 + 3𝑥3 = 1
4𝑥1 − 𝑥2 + 2𝑥3 = 5 Ans. 𝑥1 = 1 , 𝑥2 = −1 and 𝑥3 = 0

(2) 𝑥 − 2𝑦 + 3𝑧 = 2
2𝑥 − 3𝑧 = 3
𝑥+ 𝑦 + 𝑧 =0 Ans. 𝑥 = 21/19 , 𝑦 = −16/19 and 𝑧 = −5/19

4. Elimination of unknowns
 The basic strategy is to successively solve one of the equations of the set for one
of the unknowns and to eliminate that variable from the remaining equations by
substitution.
 The elimination of unknowns can be extended to large systems with more than
two or three equations. However, the method becomes extremely tedious to solve
by hand.

11
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Example (4): Use the elimination of unknowns method to solve:

2.5 x1  6.2 x 2  3.0
4.8 x1  8.6 x 2  5.5
Solution:
 Multiply the 1st eqn by 8.6 21.50 x1  53.32 x 2  25.8
and the 2nd eqn by 6.2  29.76 x1  53.32 x 2  34.1
 Add these equations  51.26 x1  0 x 2  59.9
 Solve for x1 : x1  59.9 / 51.26  1.168552478
 Using the 1st eqn solve for x 2 :
x 2  ( 3.0  2.5  1.168552478) / 6.2  0.01268045242
 Check if these satisfy the 2nd eqn:
4.8  1.168552478  8.6  0.01268045242  5.500000004
(The difference is due to the round-off errors).

(ii) Solving Large Numbers of Equations

(a) Direct Methods are those in which:
 Computations are completed in finite number of steps resulting in exact
solution,
 The method is independent of the accuracy desired.
1. Gauss−Elimination method,
2. Gauss−Jordan method,
3. LU – Decomposition or Crout's or Choleski's method,
4. Reducing matrix method.

(b) Indirect or Iterative Methods: used when direct methods are incapable or
may be slow:
 Begin with an approximate solution and obtain an improved one with each step
of the iterations,
 Require an infinite number of steps to obtain an exact solution in the absence
of round off errors,
 The accuracy of solution depends on number of iterations performed.
1. Jacobi's iteration method, and
2. Gauss−Seidel iteration method.
In some cases, direct and indirect methods are combined; first we may use a direct
method and then the solution may be improved by using indirect methods.

12
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

 Direct Methods:
1. Gauss − Elimination Method:
It is the most popular and efficient method for solving (n × n) system of equations. It consists
of a series of elementary row operations to transform the original matrix [A] into either:
 An upper triangular matrix, then by back substitution, the unknowns; 𝑥𝑛 , …., …𝑥2 ,
𝑥1 can be determined, or
 Lower triangular matrix, then by forward substitution, the unknowns; 𝑥1 , 𝑥2 , ………,
𝑥𝑛 are directly determined.
Notes:
1. Find the entry in the 1st. column with the largest absolute value. This entry is called the pivot.
2. Perform a row interchange, if necessary, so that the pivot is in the 1st. row.
3. Divide the 1st. row by the pivot. (This step is unnecessary if the pivot is 1.0)
4. Use elementary row operations to reduce the remaining entries in the 1st. column to zero’s.
5. Continue this process for other columns and @ each stage the remaining equations may
require rearranging to avoid a zero division in the 𝑎𝑖𝑗 position,
𝑛3
6. In this method, no. of multiplications and divisions = +𝑂(𝑛2 ). for example, for 10×10
3
10×10×10
eqs., no. of operations involved ≈ ≈ 333
3

Example (5): Solve by Gauss−Elimination method.

4𝑥1 − 2𝑥2 + 𝑥3 = 3
𝑥1 + 3𝑥2 − 2𝑥3 = 1
3𝑥1 − 2𝑥2 − 3𝑥3 = −10
Solution:
𝐴. 𝑥 = 𝑏
4 −2 1 3
𝑅 /4 →
[1 3 −2 1 ] 1
3 −2 −3 −10
1 −0.5 0.25 0.75
[1 3 −2 1 ] 𝑅1 − 𝑅2 →
3 −2 −3 −10 3𝑅1 − 𝑅3 →
1 −0.5 0.25 0.75
[0 −3.5 2.25 −0.25 ] 𝑅2 /−3.5 →
0 0.5 3.75 −10
1 −0.5 0.25 0.75
[0 1 −0.642 0.071 ]
0 0.5 3.75 12.25 0.5𝑅2 − 𝑅3 →

13
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

1 −0.5 0.25 0.75

[0 1 −0.642 0.071 ]
0 0 −4.071 −12.214 𝑅3 /−4.071 →
1 −0.5 0.25 0.75
[0 1 −0.642 0.071 ]
0 0 1 3

Now by back substitution, the unknowns (𝑥1 , 𝑥2 , and 𝑥3 ) can be determined as:
𝑥3 = 3

𝑥2 − 0.642(3) = 0.071 → 𝑥2 = 2
𝑥1 1
𝑥1 − 0.5(2) + 0.25(3) = 0.75 → 𝑥1 = 1 ∴ 𝑥
{ 2 } = {2}
𝑥3 3

2. Gauss – Jordan Elimination Method:

It is an extension of the Gauss method. It consists of a series of elementary row
operations to transform the original matrix [A] into identity matrix (by eliminating all the
coefficients below and above the leading diagonal), so that, the unknowns; 𝑥1 , 𝑥2 ,……,
𝑥𝑛 are readily determined without need for back substitution or forward substitution.

Thus, the given augmented matrix of the form:

𝑎11 𝑎12 ⋯ ⋯ 𝑎1𝑛 𝑏1
𝑎21 𝑎22 ⋯ ⋯ 𝑎2𝑛 𝑏2
⋮ ⋮ ⋮ ⋮
⋮ ⋮ ⋮ ⋮
[𝑎𝑛1 𝑎𝑛2 ⋯ ⋯ 𝑎𝑛𝑛 𝑏𝑛 ]
by using the Gauss−Jordan method, is reduced to that of an identity matrix as:
1 0 0 0 0 𝑏1∗
0 1 0 0 0 𝑏2∗
0 0 1 0 0 ⋮
0 0 0 1 0 ⋮
[0 0 0 0 1 𝑏𝑛∗ ]
and readily the unknowns are determined as: 𝑥1 = 𝑏1∗ , 𝑥2 = 𝑏2∗ , and …… 𝑥𝑛 = 𝑏𝑛∗
Notes:
1. This method is easier but requires more computational effort,
2. It provides a direct method for solving an inverse problems,
3. Rows may need to be switched to avoid division by zero after each operation.
14
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

𝑛3
4. The total no. of required operations by Gauss-Jordan method is = +𝑂(𝑛2 )
2
𝑛3
compared with +𝑂(𝑛2 ) operations required for Gauss-Elimination method.
3

Example (6): Solve the following system of linear equations

by Gauss−Jordan Elimination method.
2𝑥1 − 4𝑥2 + 6𝑥3 = 5
𝑥1 + 3𝑥2 − 7𝑥3 = 2
7𝑥1 + 5𝑥2 + 9𝑥3 = 4
Solution:
𝐴. 𝑥 = 𝑏

2 −4 6 5 𝑅1 /2 →
[1 3 −7 2 ]
7 5 9 4
1 −2 3 2.5
[1 3 −7 2 ] −𝑅1 + 𝑅2 →
7 5 9 4 −7𝑅1 + 𝑅3 →
1 −2 3 2.5
[0 5 −10 −0.5 ] 𝑅2 /5 →
0 19 −12 −13.5

1 −2 3 2.5 2𝑅2 + 𝑅1 →
[0 1 −2 −0.1 ]
0 19 −12 −13.5 −19𝑅2 + 𝑅3 →
1 0 −1 2.3
[0 1 −2 −0.1 ]
0 0 26 −11.6 𝑅3 /26 →

1 0 −1 2.3 𝑅3 + 𝑅1 →
[0 1 −2 −0.1 ] 2𝑅3 + 𝑅2 →
0 0 1 −0.45

1 0 0 1.85 𝑥1 1.85
[0 1 0 −0.99 ] ∴ {𝑥2 } = { −0.99 }
0 0 1 −0.45 𝑥3 −0.45

Note: To reduce round off errors take four or more numbers after decimals.

15
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Example (7): Find the inverse matrix 𝐀−𝟏 of the following system of linear
equations by Gauss−Jordan Elimination method.

2𝑥1 − 4𝑥2 + 6𝑥3 = 5

𝑥1 + 3𝑥2 − 7𝑥3 = 2
7𝑥1 + 5𝑥2 + 9𝑥3 = 4
Solution:

2 −4 6 1 0 0 𝑅1 /2 →
[1 3 −7 0 1 0 ]
7 5 9 0 0 1
1 −2 3 0.5 0 0
[1 3 −7 0 1 0 ] −𝑅1 + 𝑅2 →
7 5 9 0 0 1 −7𝑅1 + 𝑅3 →
1 −2 3 0.5 0 0
[0 5 −10 −0.5 1 0 ] 𝑅2 /5 →
0 19 −12 −3.5 0 1

1 −2 3 0.5 0 0 2𝑅2 + 𝑅1 →
[0 1 −2 −0.1 0.2 0 ]
0 19 −12 −3.5 0 1 −19𝑅2 + 𝑅3 →
1 0 −1 0.3 0.4 0
[0 1 −2 −0.1 0.2 0 ]
0 0 26 −1.6 −3.8 1 𝑅3 /26 →
1 0 −1 0.3 0.4 0 𝑅3 + 𝑅1 →
[0 1 −2 −0.1 0.2 0 ] 2𝑅3 + 𝑅2 →
0 0 1 −0.061 −0.146 0.038

1 0 0 0.239 0.254 0.038

[0 1 0 −0.222 −0.092 0.076 ]
0 0 1 −0.061 −0.146 0.038

0.239 0.254 0.038

−1
A = [ −0.222 −0.092 0.076 ]
−0.061 −0.146 0.038

16
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Pitfalls of Elimination Methods

 Division by zero. It is possible that during both elimination and back-substitution
phases a division by zero can occur.
2 x2  3x3  8
4 x1  6 x2  7 x3  3 a11  0 ( the pivot element)
2 x1  x2  6 x3  5

 Round-off errors. These errors occur due to that every result is dependent on previous
results. Consequently, an error in early steps will tend to propagate and cause errors
in subsequent steps. Therefore, a good idea to substitute your answers back into the
original equations and check whether a substantial error has occurred.

 Ill-conditioned systems are those where small changes in coefficients result in large
changes in solution. Also, it happens when two or more equations are nearly identical,
resulting a wide ranges of answers to satisfy the equations.

Surprisingly, substitution of the erroneous values, x1 = 8 and x2 = 1, into the original

equation will not reveal their incorrect nature clearly:
x1 + 2x2 = 10 8 + 2(1) = 10 (the same!)
1.1x1 + 2x2 = 10.4 1.1(8) + 2(1) = 10.8 (close!)

Important Observations
 An ill-conditioned system is one with a determinant close to zero
 If determinant D=0 then there are infinite solutions  singular system

 Scaling. (Multiplying or dividing each equation by the max. coefficient it contains)

does not change the equations but changes the value of the determinant in a significant
way. However, it does not change the ill-conditioned state of the equations!
DANGER! Scaling may hide the fact that the system is ill-conditioned!!

QUESTION: How can we find out whether a system is ill-conditioned or not?

Not easy! Luckily, most engineering systems yield well-conditioned results!
One way to find out: change the coefficients slightly and recompute & compare.

17
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Techniques for Improving Solutions

 Use of more significant numbers after decimals or double precision arithmetic will
minimize round-off errors.

 Pivoting. If a pivot element is zero, elimination step leads to division by zero. The
same problem may arise, when the pivot element is close to zero. This problem can be
avoided by:
 Partial pivoting. Switching the rows so that the largest element is the pivot element.
 Complete pivoting. Searching for the largest element in all rows and columns then
switching. But, this pivoting is rarely used because switching columns changes the
order of the x’s.

 Scaling. Minimize round-off errors and improve accuracy.

Example (8): Use of significant numbers to solve the round-off errors

Use Gauss elimination to solve the following equations

0.0003x1  3.0000 x2  2.0001

1.0000 x1  1.0000 x2  1.0000
Solution:

Dividing the 1st. equation by (0.0003) yields

x1  10000 x2  6667 which can be used to eliminate x1 from the 2nd equation:

 9999 x2   6666  x2  2 / 3

This result can be substituted back into the 1st. equation to evaluate x1 :
2.0001  3(2 / 3)
x1 
0.0003
However, the value for x1 is highly dependent on the number of significant digits
after decimals carried in the computation as shown in Table below:
Significant digits x2 x1 % relative error for x1
3 0.667 −3.33 1099
4 0.6667 0.0000 100
5 0.66667 0.30000 10
6 0.666667 0.330000 1
7 0.6666667 0.3330000 0.1

The exact solution is x2  2 / 3 & x1  1/ 3

18
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Example (9): Use of pivoting

Resolve Example (8) using the partial pivoting technique:

0.0003x1  3.0000 x2  2.0001

1.0000 x1  1.0000 x2  1.0000
Solution:

If the equations are solved in reverse order, the row with the large pivot element is
normalized. The equations are

1.0000 x1  1.0000 x2  1.0000

0.0003x1  3.0000 x2  2.0001

Elimination and substitution yield x2  2 / 3.

This result is substituted back into the 1st. equation to evaluate x1 :
1  (2 / 3)
x1 
1
and for different numbers of significant digits after decimals, the value for x1 can
be computed as shown in Table below:
Significant digits x2 x1 % relative error for x1
3 0.667 0.333 0.1
4 0.6667 0.3333 0.01
5 0.66667 0.33333 0.001
6 0.666667 0.333333 0.0001
7 0.6666667 0.3333333 0.00001

Comparing the results in this case with those of Example (8) shows that a pivot
strategy is much more satisfactory.

Homework # 3:

1. Solve by Gauss−Elimination method:

2 𝑥2 + 4𝑥3 = 6
4𝑥1 + 𝑥2 − 3𝑥3 = 1
3𝑥1 − 8 𝑥2 + 2𝑥3 = 2 Ans. 𝑥3 = 1.264 , 𝑥2 = 0.472 and 𝑥1 = 1.080

2. Solve by Gauss− Jordan method:

2𝑥 + 3𝑦 − 4𝑧 = −3
𝑥 + 2𝑦 + 3𝑧 = 3
3𝑥 − 𝑦 − 𝑧 = 6 Ans. 𝑥 = 2 , 𝑦 = −1 and 𝑧 = 1

19
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

3. LU − Decomposition Method:

To solve the equation of the form: 𝐴. 𝑥 = 𝑏

In this method, the matrix [A] is factorized as: 𝐴 = 𝐿 .𝑈

In which [A] is written as product of a lower triangular matrix and an upper

triangular matrix.

∴ 𝐴. 𝑥 = 𝐿𝑈𝑥 = 𝑏 ………………………………………………………(1)
In details:
𝑎11 𝑎12 𝑎13 𝑥1 𝑙11 0 0 𝑢11 𝑢12 𝑢13
[𝑎21 𝑎22 𝑎23 ] {𝑥2 } = [𝑙21 𝑙22 0 ][ 0 𝑢22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑥3 𝑙31 𝑙32 𝑙33 0 0 𝑢33
Total number of unknown = 𝑛2 + 𝑛

For simplifying, put 𝑙𝑖𝑖 = 1 in diagonal of L (Doolitle’s Method) or 𝑢𝑖𝑖 = 1 in

diagonal of U (Crout’s or Choleski’s Method).

To solve Eq.(1) for a given A and b, the solution is done in two logical steps:
1. First, let 𝑼. 𝒙 = 𝒚 and find y from the equation: L y = b for y.
(starting from top because L is a lower triangular matrix).
𝑙11 0 0 𝑦1 𝑏1
[𝑙21 𝑙22 0 ] {𝑦2 } = {𝑏2 }
𝑙31 𝑙32 𝑙33 𝑦3 𝑏3
2. Second, to find x use: Ux = y for x.
(starting from bottom because U is an upper triangular matrix).
𝑢11 𝑢12 𝑢13 𝑥1 𝑦1
[ 0 𝑢22 𝑢𝑎23 ] {𝑥2 } = {𝑦2 }
0 0 𝑎33 𝑥3 𝑦3

Example (10): Solve the following system of linear equations

by LU - decomposition method.
8𝑥1 + 2𝑥2 + 3𝑥3 = 30
𝑥1 − 9𝑥2 + 2𝑥3 = 1
2𝑥1 + 3𝑥2 + 6𝑥3 = 31

20
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Solution:

𝑨. 𝒙 = 𝒃

8 2 3 𝑥1 30
𝑥
[1 −9 2] { 2 } = { 1 }
2 3 6 𝑥3 31
Factorize [A] into L and U: 𝐴 = 𝐿 . 𝑈 and set 𝑙𝑖𝑖 or 𝑢𝑖𝑖 = 1 in diagonal of L
or in diagonal of U (Take 𝑙𝑖𝑖 = 1), then

8 2 3 1 0 0 𝑢11 𝑢12 𝑢13

[1 −9 2] = [𝑙21 1 0] [ 0 𝑢22 𝑢23 ]
2 3 6 𝑙31 𝑙32 1 0 0 𝑢33

for Row (1) of A:

8 = 1. 𝑢11 ; 𝑢11 = 8
2 = 1. 𝑢12 + (0)(𝑢22 ) + (0)(0) ; 𝑢12 = 2
3 = 1. 𝑢13 + (0)(𝑢23 ) + (0)(𝑢33 ) ; 𝑢13 = 3

for Row (2) of A:

1 = 𝑙21 . 𝑢11 + (1)(0) + (0)(0) ; 𝑙21 = 0.125
−9 = 𝑙21 . 𝑢12 + (1)(𝑢22 ) + (0)(0) ; 𝑢22 = 9.25
2 = 𝑙21 . 𝑢13 + (1)(𝑢23 ) + (0)(𝑢33 ) ; 𝑢23 = 1.625
for Row (3) of A:
2 = 𝑙31 . 𝑢11 + (𝑙32 )(0) + (1)(0) ; 𝑙31 = 0.25
3 = 𝑙31 . 𝑢12 + (𝑙32 )(𝑢22 ) + (1)(0) ; 𝑙32 = − 0.270
6 = 𝑙31 . 𝑢13 + (𝑙32 )(𝑢23 ) + (1)(𝑢33 ) ; 𝑢33 = 5.689

Now, 𝐴. 𝑥 = 𝐿𝑈𝑥 = 𝑏 is written as:

1 0 0 8 2 3 𝑥1 30
[0.125 1 𝑥
0] [0 −9.25 1.625] { 2 } = { 1 }
0.25 −0.27 1 0 0 5.689 𝑥3 31

Solve in two stages:

21
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

1. First, let 𝑼. 𝒙 = 𝒚 and find y from the equation: L y = b for y.

(starting from top because L is a lower triangular matrix).
1 0 0 𝑦1 30
[0.125 1 𝑦 =
0] { 2 } { 1 }
0.25 −0.27 1 𝑦3 31
∴ 𝑦1 = 30
𝑦2 = 1 − 0.125 (30) = −2.75
𝑦3 = 31− 0.25(30) + 0.27(−2.75) = 22.758

2. Second, to find x use: Ux = y for x.

(starting from bottom because U is an upper triangular matrix).
8 2 3 𝑥1 30
𝑥
[0 −9.25 1.625] { 2 } = { −2.75 }
0 0 5.689 𝑥3 22.758

5.689 𝑥3 = 22.758 ; 𝑥3 = 4.0

−9.25 𝑥2 + 1.625(4) = −2.75 ; 𝑥2 = 1.0
8 𝑥1 + 2(1) + 3(4) = 30 ; 𝑥1 = 2.0

 Important Notes on Choleski’s method

(1) Solution of A x = B when A is symmetric.

When A is symmetric, then AT = A, in this case factorize A into:

A  L LT or A  U UT

l11 0 0 0 u11 u12  unn 

 0 0  and  u22  u2 n 
where, L  l21 l22 U  0
         
l 0 lnn  0 0  unn 
 n1 ln 2 

Solution is also in two stages as before. But, there is a danger that complex
numbers will appear in (L or U) especially when the diagonal coefficients of A
are small or negative. Therefore, either continue the solution and get the results
or try to factorize A into:

22
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

A  L D LT or A  U D UT

1 0 0 d11 0 0  1 u12 u13 

where, L  l21 1 0 D   0 d 22 0  or U  0 1 u23 
l   0 d 33  0 0 1 
 31 l32 1   0 

(2) If A is not symmetric (AT ≠ A), then multiply both sides of A x = B by AT.

Thus A T A x  A TB

Let C  ATA ( symmetric) and F  A TB ( column)

Then Cx F

and use C  L LT or C  U UT

or C  L D LT or C  U D UT

Example (11): Solve Ax = B when A is symmetric by

(i) A  L LT and (ii) A  L D LT .
4 x1  x2  2
x1  3 x2  5
Solution:

(i) By using A  L LT : Write 4 1   l11 0  l11 l21 

1  3 l21 l22   0 l22 

Notice there are 3 unknowns ( l11 , l21 and l22 ).

By equating the corresponding elements from both sides:
l112  4  l11  2  Try l11  2
l11. l21  1  l21  0.5
l212  l222  3  l222  3  0.5 2  3.25
So l22   3.25 (complex number )
Continue the solution with this complex number, then the equations are recast as:
2 0  2 0.5   x1    2.0 
0.5  3.25  0  3.25   x2   5.0 

2 0.5   x1   y1 
Let 
0  3.25   x2   y2 

23
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Then
2 0   y1   2.0 

0.5  3.25   y2   5.0 
This gives
y1  1.0
0.5 y1   3.25 y2  5.0  y2  5.5 /  3.25
Then
2 0.5   x1   1.0 

0  3.25   x2   5.5 /  3.25 

Start from bottom, then

1.0  0.5(1.692)
x2  1.692 and x1   0.077
2
These values check but with truncation errors!

 Try again with l11  2

Then l21  0.5
and l22   3.25 (as before)

Then  2 0   2  0.5   x1    2.0

 0.5  3.25   0  3.25   x2    5.0 

Let
 2  0.5   x1   y1 

 0  3.25   x2   y2 
Then
 2 0   y1    2.0 
 0.5  3.25   y2   5.0 

This gives
y1  1.0
 0.5 y1   3.25 y2  5.0  y2  5.5 /  3.25

Then
 2  0.5   x1  

 1.0 
 0  3.25   x2   5.5 /  3.25 
Thus,
 1.0  0.5(1.692)
x2  1.692 and x1   0.077 (as before)
2
24
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Write 4 1    1 0  11
d 0  1 l21
(ii) By using A  L D LT :
1  3 l21 1  0 d 22  0 1 

There are 3 unknowns ( l21 , d11 and d 22 ). Work out D LT , then

4 1    1 0 d11 d11 l21

1  3 l21 1  0 d 22 

Equating the corresponding elements from both sides:

d11  4
d11. l21  1  l21  0.25
d11 l212  d 22  3  d 22  3.25
Then
1 0 4 0  1 0.25  x1    2.0
0.25 1 0  3.25 0 1   x2   5.0 
Let
4 0  1 0.25  x1    y1 
0  3.25 0 1   x2   y2 
Then
1 0  y1    2.0 This gives y1  2 and y2  5.5
0.25 1  y2    5.0 
Then
4 0  1 0.25  x1    2 
0  3.25 0 1   x2   5.5
Or
4 1   x1    2  Thus, x2  1.692 and x1  0.077
0  3.25  x2   5.5

Homework # 4: Solve By Crout’s method:

(1) 𝑥1 + 4 𝑥2 = 10
2𝑥1 + 10 𝑥2 − 4𝑥3 = 7
𝑥2 + 8𝑥3 − 𝑥4 = 6
𝑥3 − 6𝑥4 = 4
Ans. 𝑥4 = −0.466, 𝑥3 = 1.203, 𝑥2 = −4.093, and 𝑥1 = 26.373
(2) 2 𝑥1 + 𝑥2 + 𝑥3 = 7
𝑥1 + 2𝑥2 + 𝑥3 = 8
𝑥1 + 𝑥2 + 2𝑥3 = 9
Ans. 𝑥3 = 3 , 𝑥2 = 2 and 𝑥1 = 1

25
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

4. Reducing Matrix Method:

This method involves the formulation of reducing matrices to transform a given
matrix into upper-triangular form. Knowing that for n  n set  (n  1) reducing
matrices are needed.
Consider the following set of 3 x 3
 a11 a12 a13  b1 
A B  a21 a22 a23  b2 
a a33  b3 
 31 a32

First, to eliminate a21 and a31 from col.(1), pre-multiply the given matrix by the 1st.
reducing matrix [R1]:
1 0 0
R1    a 21/a11 1 0
 a /a 1 
 31 11 0
1 0 0  a11 a12 a13  b1  a11 a12 a13  b1 
Thus,  a 21/a11 1 0 a 21 a 22 a 23  b2    0 a 22 a 23  b2 
 a /a 1  a a 33  b3   0 a 32 a 33  b3 
 31 11 0  31 a 32
 from col.(2), pre-multiply the new matrix by the 2nd.
Second, to eliminate a32
reducing matrix [R2]:

1 0 0 a11 a12 a13 b1  a11 a12 a13  b1 

0 1 0  0 a 22 a 23 b2    0 a 22 a 23  b2 
0  a 32/a 22 1   0 a 32 a 33 b3   0 0   b3 
a 33
Note that the original matrix is now transformed into an upper triangular matrix.

Example (12): Solve by the reducing method.

𝑥1 + 𝑥2 + 𝑥3 = 6
−2𝑥1 − 𝑥2 + 2𝑥3 = 2
3𝑥1 − 2𝑥2 + 𝑥3 = 2
Solution:

 1 1 1  6  a11 a12 a13  b1 

 2  1 2  2  a 21 a 22 a 23  b2 
 3  2 1  2 a 31 a 32 a 33  b3 
To reduce this matrix into an upper-triangular form, first, eliminate a21 and a31
from col.(1), by pre-multiply the given matrix by the reducing matrix [R1]:
26
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

 1 0 0
R1    2 1 0
 3 0 1 
 1 0 0  1 1 1  6  1 1 1  6 
 2 1 0  2  1 2  2   0 1 4  14
 3 0 1   3  2 1  2  0  5  2   16

 from col.(2), pre-multiply the new matrix by the 2nd. reducing

Second, to eliminate a32
matrix [R2]:

1 0 0
R 2   0 1 0
0 5 1
1 0 0  1 1 1  6  1 1 1  6
0 1 0  0 1 4  14   0 1 4  14 
0 5 1  0  5  2   16  0 0 18  54

Solving by back substitutions gives

54
x3  3
18
x2  14  4(3)  2

x1  6  3  2  1

Homework # 5: Solve By reducing matrix method

2𝑥1 − 𝑥2 + 3𝑥3 − 2𝑥4 = 1

𝑥1 + 2𝑥2 − 𝑥3 + 𝑥4 = 6
3𝑥1 + 3𝑥2 + 4𝑥3 − 5𝑥4 = 1
5𝑥1 − 2𝑥2 + 7𝑥3 − 6𝑥4 = −2

Ans. 𝑥4 = 4, 𝑥3 = 3, 𝑥2 = 2, and 𝑥1 = 1

27
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

 Indirect Methods:
1. Jacobi's iteration method, and
2. Gauss−Seidel iteration method.

1. Jacobi's Iteration Method

The system of equations 𝐴. 𝑥 = 𝑏 for 3 × 3 system is:-

𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 = 𝑏2
𝑎31 𝑥1 + 𝑎32 𝑥2 + 𝑎33 𝑥3 = 𝑏3

 Rearrange the equations so that the diagonal coefficient is greater than other
coefficients in any equation (i.e., |𝑎𝑖𝑖 | > ∑|𝑎𝑖𝑗 | for i = 1, 2, ……..n
j = 1, 2, ……..n

 Check whether the iterative procedure is suitable or not. An iterative solution is

possible if the absolute value of the diagonal coefficient in any equation is
greater than the sum of the absolute values of all other coefficients.

 Express the set of equations in the form:

(𝑘+1) (𝑘) (𝑘)
𝑥1 = [𝑏1 − 𝑎12 𝑥2 − 𝑎13 𝑥3 ] / 𝑎11

(𝑘+1) (𝑘) (𝑘)

𝑥2 = [𝑏2 − 𝑎21 𝑥1 − 𝑎23 𝑥3 ] / 𝑎22

(𝑘+1) (𝑘) (𝑘)

𝑥3 = [𝑏3 − 𝑎31 𝑥1 − 𝑎32 𝑥2 ] / 𝑎33 for k = 1, 2, 3,………...

(0) (0) (0)

 Start with initial estimates (𝑥1 , 𝑥2 , 𝑥3 ) = 0, 0, 0 and compute
(1) (1) (1)
(𝑥1 , 𝑥2 , 𝑥3 ) which all in turn is used as new initial values to compute
(2) (2) (2)
(𝑥1 , 𝑥2 , 𝑥3 ) and so on.

 The iteration scheme is continued until convergence is reached, i.e., till

(𝑘+1) (𝑘) (𝑘+1) (𝑘) (𝑘+1) (𝑘)
(𝑥1 − 𝑥1 ), (𝑥2 − 𝑥2 ) and (𝑥3 − 𝑥3 ) is less than or equal to the
desired degree of accuracy specified.

28
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

Example (13): Solve the following equations by Jacobi’s method.

5𝑥1 − 2𝑥2 + 𝑥3 = 4
𝑥1 + 4𝑥2 − 2𝑥3 = 3
𝑥1 + 2𝑥2 + 4𝑥3 = 17
Solution:

 Check the diagonal coefficients and determine whether the iterative procedure is
suitable. Thus:-
|5| > |−2| + |1| → 5 > 3 (O.K.)

|4| > |1| + |−2| → 4 > 3 (O.K.) |𝑎𝑖𝑖 | > ∑|𝑎𝑖𝑗 |

|4| > |1| + |2| → 4 > 3 (O.K.)

∴ the iterative approach will converge.

 Express the set of equations as:

(𝑘+1) (𝑘) (𝑘)
𝑥1 = 0.8 + 0.4𝑥2 − 0.2𝑥3
(𝑘+1) (𝑘) (𝑘)
𝑥2 = 0.75 − 0.25𝑥1 + 0.5𝑥3
(𝑘+1) (𝑘) (𝑘)
𝑥3 = 4.25 − 0.25𝑥1 − 0.5𝑥2

(0) (0) (0)

 Start with initial estimates (𝑥1 , 𝑥2 , 𝑥3 ) = 0, 0, 0 and substituting into
(1) (1) (1)
equations yields (𝑥1 , 𝑥2 , 𝑥3 ) = 0.8, 0.75, 4.25 which all in turn is used as
(2) (2) (2)
new initial values to compute (𝑥1 , 𝑥2 , 𝑥3 ) and so on.
k
𝒙
0 1 2 3 4 5 6 7 8 9 10

𝒙𝟏 0 0.8 0.25 1.14 1.24 1.02 0.92 0.98 1.02 1.01 0.99
𝒙𝟐 0 0.75 2.68 2.53 1.89 1.79 1.99 2.07 2.02 1.98 1.99

𝒙𝟑 0 4.25 3.68 2.85 2.70 2.99 3.10 3.02 2.97 2.98 3.01

𝑥1 1
The exact solution for this system of equations is {𝑥2 } = {2}
𝑥3 3

29
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

2. Gauss – Seidel Iteration Method

This method converges faster than Jacobi’s method. In this method, start with initial
(0) (0) (0) (1) (1) (1)
estimates (𝑥1 , 𝑥2 , 𝑥3 ) = 0, 0, 0 to compute a new set (𝑥1 , 𝑥2 , 𝑥3 ). But, the new
computed value calculated first should replace the old one before computing all other
new values. Then, reuse the new estimated values as initial estimates to find the next
(1) (1) (1)
(𝑥1 , 𝑥2 , 𝑥3 ) and so on.

Example (14): Resolve Example (13) using Gauss−Seidel method.

5𝑥1 − 2𝑥2 + 𝑥3 = 4
𝑥1 + 4𝑥2 − 2𝑥3 = 3
𝑥1 + 2𝑥2 + 4𝑥3 = 17
Solution:

 After checking the diagonal coefficients and determining whether the iterative
procedure is suitable.

 Express the set of equations in the form:

(𝑘+1) (𝑘) (𝑘)
𝑥1 = 0.8 + 0.4𝑥2 − 0.2𝑥3

(𝑘+1) (𝑘+1) (𝑘)

𝑥2 = 0.75 − 0.25𝑥1 + 0.5𝑥3

(𝑘+1) (𝑘+1) (𝑘+1)

𝑥3 = 4.25 − 0.25𝑥1 − 0.5𝑥2

(0) (0) (0)

 Now starting with initial estimates (𝑥1 , 𝑥2 , 𝑥3 ) = 0, 0, 0 and substituting
into equations yields:-
(1) (0) (0)
𝑥1 = 0.8 + 0.4𝑥2 − 0.2𝑥3 = 0.8 + 0 + 0 = 0.8

(1) (1) (0)

𝑥2 = 0.75 − 0.25𝑥1 + 0.5𝑥3 = 0.75 – 0.25 (0.8) + 0 = 0.55

(1) (1) (1)

𝑥3 = 4.25 − 0.25𝑥1 − 0.5𝑥2 = 4.25 – 0.25(0.8) − 0.5(0.55) = 3.775

This process is continued until successive values of each vector are sufficiently
close in magnitude as shown in the following table:

30
Numerical Methods / Solution of Simultaneous Linear Equations Assist. Prof. Dr. Rafi' M. S.

k
𝒙
0 1 2 3 4 5 6 7

𝒙𝟏 0 0.8 0.265 1.249 0.956 1.002 1.001 0.999

𝒙𝟐 0 0.55 2.571 1.887 2.008 2.003 1.999 2.000

𝒙𝟑 0 3.775 2.898 2.994 3.007 2.998 3.000 3.000

Note that the results obtained in (5) iterations are more accurate than those
obtained in (10) iterations using Jacobi’s method.
𝑥1 1
The exact solution for this system of equations is {𝑥2 } = {2}
𝑥3 3

Homework # 6: Solve each of the following system of equations:

a. By Jacobi’s method.
b. By Gauss−Seidel method.

(1) 5𝑥1 − 𝑥2 − 𝑥3 = 2
𝑥1 + 𝑥2 + 4𝑥3 = 10
𝑥1 − 3𝑥2 + 𝑥3 = 0
𝑥1 1
Ans. {𝑥2 } = {1}
𝑥3 2

(2) 5𝑥 − 3𝑦 − 2𝑧 = 9
2𝑥 − 4𝑦 + 𝑧 = 8
𝑥 + 2𝑦 + 4𝑧 = 22

𝑥 4
Ans. {𝑦} = {1}
𝑧 4

(3) 8𝑥1 + 2𝑥2 + 3𝑥3 = 30

𝑥1 − 9𝑥2 + 2𝑥3 = 1
2𝑥1 + 3𝑥2 + 6𝑥3 = 31
𝑥1 2
Ans. {𝑥2 } = {1}
𝑥3 4

31