Proceedings of the 271h Conference

FA11 11:15 - on Decision and Control

Austin, Texas December 1988

A CONVERGENCE CONDITION FOR OPTIMAL NONLINEAR FILTERING

FOR SYSTEMS WITH UNKNOWN PARAMETERS

Francisco A. Casiello and Kenneth A. Loparo

Department of Systems Engineering

Case Western Reserve University
Cleveland, OH 44106

Abstract 2 Continuous T i m e Systems

This papers examines the problem of estimating the state of
a linear stochastic plant with unknown parameters taking values 2.1 Problem Formulation
in a tinite set.
Stochastic stability theory is used to establish conditions Consider the following linear stochastic system
under which the a-posteriori probabilities defined on a finite pa-
rameter set converge almost surely, both in continuous and dis- dx = A,xdt + B,udt + dv; ~ ( 0=)20 (2.1.1)
crete time. An example of what happens if the conditions are
d y = C,xdt 4-dw (2.1.2)
not satisfied is given.
where s ( t ) E R",u(t) E 'R",y(t) E RP,and A,,&, and
C, are matrices of appropiate dimensions, (A,, B,, C,) is a min-
1. Introduction imal triple for all T E F. Also x~ is normally distributed with
mean m(0) and covariance matrix P(O), and v(.) and w(.) are
Let T be an unknown parameter belonging to a finite set independent Brownian motion processes which are independent
F = {0,1, ...,f}. Let ( A p ,B,, C,) be a realization of a linear Of T .
stochastic system where T takes values in F. The additive ran- Let G(m,P ) denote a Gaussian density with mean m and
dom disturbances in the plant and observation equations are in- covariance P , then p(zlYt,r = i) = G(m,,P,) where Y* =
dependent of T . u { Y ( T ) , 0 5 T I tl, and
If no initial distribution is specified for T , then the estima-
tion problem can be solved by a maximun likelihood approach,
where a set of candidates probability distributions are tested by
means of a likelihood ratio, and the most likely parameter is se-
lected.
If an initial distribution for T is assumed, then T is consid-
(2.1.4)
ered as a random variable with an a-priori distribution, and the
a-posteriori distribution is calculated as the observations are with Pt(t) satisfying
made. This a-posteriori distribution can be written in terms
of the likelihood function, which satisfy a stochastic differential
equation.
A stochastic differential equation for the likelihood ratio with the initial condition P,(O)= P(0)
function is developed to find conditions for convergence of the Here Q" and Q" are the plant and observation noise covari-
likelihood ratio functions to zero using stochastic stability analy- ances matrices with Q" 2 0 and Q" > 0.
sis. Let Z,(t)denote the likelihood function that the system is in
These conditions are used to guarantee almost sure con- regime i given Y*. Then, irrespective of the initial distribution
vergence of the a-posteriori distribution to a distribution with Of T
all the probability mass concentrated at the correct parameter
value. We also develop these results in discrete time. dl, = I,C,m,dy; l,(O) = 1 (2.1.6)

If an a-priori distribution is specified for T , then the a-posteriori

distribution ?r,(t)can be written in terms of the likelihood func-
tions

(2.1.7)

1984
88CH2531-2/88/0000-1984$1
.OO @ 1988 IEEE
 2.3 Stability Theorems - Continuous time[4. Theorem
where r i ( t ) = Prob(r = ilyt), i E F.
5.71
Let p(zly') denote the conditional density of z ( t ) ,then by
the theorem of total probability, we have
Let V ( t ,z) be a continuous real valued function satisfying:
1)
4llzll) I V ( t , z )5 b(ll41)7
for llzll I K and some constant K, and where the functions
a and b are continuos and positive definite on R+.
2) V ( t , z )-+ w as llzll -+ w
3) The partial derivatives g ,E, and -exist and are
2.2 Stochastic Stability continuous.
Then if
We begin with some definitions and theorems on stochastic L V ( t , z ) I -r(llzll) for 0 < llzll I K
stability which are needed to establish our later results presented where the function y(.) is continuous and positive definite
in Sections 2.4 and 3.3. on R+,and also for all z # 0, a(.) -+ 03 as T -+ 00 where t
The set up is: is the Markov generator associated with the Markov diffusion
w(t) is a Wiener process defined on the probability space process defined by (2.2.2) and is given by:
(n,A,'P)and f ( t , z ) and g ( t , z ) are functions satisfying the fol-
lowing conditions:
1) f ( t , z), g(t,z) are continuous vector and matrix valued
functions in t
2) Ilf(t,z)112= lfi(t,z)I2I K(1 t 11z112), Vt and 2,
3) IIg(t,z)I12= Ci,j1gij(t,z)I2I K(1 t 11z1I2), v t and 2,
4) i l f ( t , z + a)- f(t,z)ll 5 Kllall, Vt and 2,
+
5 ) 11g(t, z a)- g(t,z)II I Kllall, Vt and z, K a nonnegative THEN z ( t ) converges to zero almost surely.
real number.
We consider a stochastic processes z ( t ) E R" which satisfies
2.4 Convergence theorems.
the Ito stochastic integral equation:
2.4.1 Convergence of the likelihood ratio function

z ( t ) = 20 t l
f ( ~z(T))dT
, t l g ( T , z(r))dw(T) 40) = zo
(2.2.1)
Let i denote the correct parameter value.
Define the likelihood ratio function &(t), j E F as
where w(t) is a Wiener process as defined above and the
I.
6. - 1
integral with respect to dw is defined in the It0 sense.
- li
The stochastic differential equation for the process z ( t ) is
given by: Theorem 1
d 4 t ) = f ( t ,z(t))dt4- g ( t , z ( t ) ) W t ) (2.2.2)
Let 2i = Cimi, i E F.
Definition 42, pp. 30, 311 E i i # i j , V i # j , i,jEF,THEN
The solution z ( t ) = 0 is said to be stochastically stable with
probabilty one iff for any p > 0 and e > 0 there is a 6 ( p , ~ >) 0
< j ( t ) -+ O a . ~ .V j #iEF
such that, if llzoll < 6 ( p , e), then

Proof:
We first derive a stochastic differential equation for the like-
lihood ratio.
Definition 2
Let the innovations sequence corresponding to T = i be
denoted dv, then
d y = dv + i i d t
The solution z ( t ) E 0 is said to be stochastically asymptoti-
cally stable in the large if further and the likelihood functions satisfy;

lim z ( t ) = 0) = 1,
P( t-CO Vzo E Rn (2.2.4) d l3. -- 1j.^Tdy
zj = IjiTdv + 1jiTiidt

here T denotes transpose and,

If z ( t ) = 0 is a stochastically asymptotically stable in the
large solution of (2.2.2), then z ( t ) is said to converge to zero al-
moat surely.
or

Applying Ito's differentiation rule to (j yields;

1985
 In some sense the noise satisfies a persistent excitation pro-
perty, which guarantees convergence if each triple is minimal and
the convergence condition is satisfied.

Remark
= &(Zj - i;)Tdv
In this section we shall deal with only two parameters and a
The stochastic stability theorem of Section 2.3, can be ap- scalar system; the results can readily be translated t o the more
plied t o the stochastic differential equation for t, as follows general situation with multiple parameters and vector- valued
Select V(&) = &j', n > 0, then we have systems.
Assume then that F = (0, l}, then the a-posteriori proba-
bilities satisfy:

dno = nonl(C0mo - Clml)(dy- ydt)

LV = n(n - 1)&,(2,- i : ) T ( i , - 2:)
dnl = non1(Clml - Como)(dy- ydt)
For 0 < n < 1 and Z, # Zl we have
We shall work with only one of the equation since they are
y(tj) = -n(n - 1)<,(i3 - i l ) T ( i j - 2:) > o +
linked by the algebraic constrain no n1 = 1.
+
Here 0 = noComo s1Clml
and & + 0 almost surely for all j # i E F. We see that no = 0 and no = 1 are equilibrium points of the
Remark: system of differential equations.
For 0 < n < 1 the Lyapunov function tndoes not satisfy
the condition 3 of the theorem, however V(&) = &; is in the We maintain that if T = 0 is the correct parameter then
domain of the differential operator L = t;(i,- 2t)T(2, - i,)% no = 1 is a locally stable equilibrium point. Furthermore AO = 0
which is all what is required for the conclusion of the theorem to is a locally unstable equilibrium point.
be valid. This follows from the following argument:
We first note that if T = 0 this implies that there exists an
2.4.2 Convergence of t h e a-posteriori probabilities. innovations representation of y as d y = dv t Comodt where v
is a Brownian motion process with respect to the sigma alge-
Corolay 1. Under the conditions of theorem 1 bra generated by the observations and the probability measure
P(ylr = 0).
s i ( t )+ 1 a s . as t -+ 00 Now, we can write
This result follows from the representation formula

1
Tj(t)=
1+ C , E F , , P * W ) $ $ or
Clearly r l ( t )-+ 1 a.s. iff & ( t )-+ 0 a.s. V j E F , j # i.
dno = noa:(Como - Clml)2dtfnon1(Como -Clml)dv (2.4.4.1)

2.4.3 Persistent excitation of t h e noise We now linearize (2.4.4.1) about no" with nd = (1 - ni),to
obtain:
We have established that Z, # 2,, V Z , J E F, 2 # J is a
sufficient condition for almost sure convergence of n,(t) -+ 1, as d*o = (1 - 4 4 t 3(T:)2)K2(Ao- *:)dt + (1- 2s;)K(~o- n;)dv
t 4 w, where z is the correct parameter.
We now show that i j , J E F can not be identically zero Here K = (Como - C l m l )
for aJl t. Select n; = 1 then
In effect:
d r o = -(no - 1)Kdv
2, = C,m,
A candidate Lyapunov function is V(n0)= for n > 0, then
dZ3 = C,dm, = C,A,m,dt + C,B,udt + C,K,(dy - 2,dt)
= C,(A,m, -I-B,u + K&)dt - C,K,i,dt + C,K,dv 1
LV = -n(n-l)n,n-2(a~-1)2K2 < 0 if n < 1 and 0 < r0 < 1
with dv = d y - i,dt. 2
Since (A,, B,, C,) is a minimal triple, and the innovation But this implies local stability about TO =1
process dv which is a Brownian motion takes arbritary values Select now no" = 0 then
in RP with positive probability, 2, can not be identically zero
unless C3K3 0. dao = noK2dt + noKdv
But C,K, = C,P,CF(Qw)-'. Since both P, and Q" are
of fd rank C3K, = 0 only if C, = 0, but this is not possible and for the same candidate Lyapunov function we obtain
because the triple (A,, B,, C,) is minimal.

1986
 Let G(Cimi(k),CiMiCF t Q"')= p(y(k)lyk-', r = i) denote the
likelihood that the system is in regime i given the observations
Y(9, 0 51 I IC.
Then for 0 < TO < 1with n > 0 as required LV is positive, If an a-priori distributions is specified for r , then the a-
which implies that TO = 0 is locally unstable. posteriori distribution can be written in terms of the likelihood
Since the stochastic differential equation has only two equi- functions as
librium points, one of them locally stable and the other locally
unstable, all trajectories will converge to the stable equlibrium
point. Note that Theorem 1 guarantees global convergence, and
ensures that the stable equilibrium point is actually globally
atractive. and then the conditional distribution of z(k) is

p(z(k)IYk) = C p ( z ( k ) l Y k , .= i)*i(k)
iEF
3 Discrete time

3.1 Problem formulation

Consider a linear discrete time stochastic system of the form 3.2 Stability Theorem - Discrete time

Discrete time Supermartingales

Consider a sequence of random variables {z(k)}k=O,l,...oo

defined on a probability space (R,A,P).
Let Y k be a current of sigma algebras satisfying

where z(k) E R",u(k) E R",y(k) E RP,A,., Br and C, are yk-' c yk c A, 0 I 25 k

matrices of appropriate dimensions, ( A p ,B,., CT)is a minimal
triple for all r E F. Also r o , { v ( k ) }and { w ( k ) } are mutually such that the conditional expectation
independent white Gaussian random sequences with
E{z(k)[Yk-'} 5 ~ ( -k I), and E{(z(O)(}< 00
20 G(m(O),P(0))
The sequence is then called a Yksupemartingale .
4k) G(0,Q")
- G(0,&"'I
Definitions 1 and 2 of section 2.2 can be transcribed for the
process z(k) defined above.
The stability theorem of section 2.3 also holds if condition 3
r is an unknown parameter taking values in a finite set F =
tO,L...,f}. is replaced by the following condition [3, pp. 841:
3) E { v ( k , z ( k ) ) I Y k - ' } - V ( k - 1,z(k- 1)) I -7(IIz(k- 1)II),
Let Y k denote the sigma algebra generated by the observa- where $,.) is a positive function, i.e. V ( k , z ( k ) )is a super-
tions y(l), 0 5 1 5 k. Let r take values in F = {0,1, ...,f}. martingale.
Let the conditional density of the state at time k assuming
model i be given by
3.3 Convergence Theorems - Discrete time
p(z(k)lYk?r = i) = G(?i(k),Pi(k))
3.3.1 Convergence of the likelihood ratio function
where i.i(k) and Pi(k) are the conditional mean and covariance
matrix assuming model i and are given by: Let i denote the correct parameter value. Define the likeli-
hood ratio function t j ( k ) , j E F as
? i ( k ) = mi(k) t K i ( k ) [ y ( k -
) Cimi(k)] (3.1.2)
G(Cjmj(k),CjMj(k)CjTt Q"')
Pi(k) = ( I - Ki(k)Cj)Mi(k) (3.1.3)
"(') = G(Cimi(k),CiMi(k)CTt Q w )
and the Kalman gain
Theorem 2
Ki(k) = Mi(k)CF[CiMi(k)C? + &"'I- (3.1.4)

here mi(k) and Mi(k) are the one step ahead mean and covari- Let & ( k ) = Cimi(k), Z i ( k ) = CiMi(k)CFt Q w , i E F
ance matrix assuming model i and If G ( & ( k ) ,Z j ( k ) ) # G(2j(k),Zj(k)), V i ,j E F,i # j THEN
p(r(k)lYk-',r = i ) = G(mi(k),Mi(k)) t ( k ) -+ 0 a.$. V j # i E F
with mj(k) and Mi(k) computed recursively by Proof: Write

mi(k) = AjZi(k - 1) + B+(k - 1) (3.1.5)

M i ( k ) = AiPi(k)AT t Q"; Mi(0) = P(0) (3.1.6)

1987
 We conclude that if (A0 - A l ) z ( k ) is in the range space of
(Bo - B1) then it is possible to design a control that inhibits the
ability of the estimator to identify the correct parameter.

4. Conclusions

The problem of convergence of the a-posteriori probabili-

ties for systems with unknown parameters is studied. If an a-
priori distribution for the unknown parameter is specified, the
unknown parameter is treated as a random variable, then the
From Jensen's inequality results are valid along the sample path T = i .
Stochastic stability theorems are used to establish condi-
tions for convergence, and a counterexample is formulated to see
that the conditions are indeed necessary and sufficient.

iff References

1) Hijab, 0. "The Adaptive LQG Problem - Part I", IEEE

TAC, Vol. AC-28, No. 2, Feb. 1981, pp. 171-178.
2) Kushner, H. Stochastic Stability and Control, Academic
3.3.2 Convergence of the a-posteriori probabilities Press, New York, 1967.
3) Bucy, R. and Joseph, P. Filtering for Stochastic Processes
Corolary 2. Under the conditions of Theorem 2., with Applications to Guidance , Interscience Publishers, Wi-
ley, New York, 1968.
4 ) Gaxd., T. Introduction to Stochastic Differential Equations,
M. Dekker, 1988.

3.3.3 An example

In this section we design a control sequence such that the

conditions for convergence established before are not satisfied.
We consider a discrete time problem with perfect state ob-
servations.
For this problem the a-posteriori probabilities can be writ-
ten as:

p(z(k)lr= j , z(k - l))Tj(k - 1)

Tj(k) = "j (0) = "30
E, p ( z ( k ) l=~ n, z(k - l ) ) ~ , ( k- 1)
(3.3.3.1)
where
p ( z ( k ) l r = j , z ( k - 1)) =

1
(2~)-%(detQ')-$ exp --ll(Ajz(k - 1)
2
+ Bju(k - l ) ) l l ~ v - ~
Let F = (0,l}, and make at any time k

+ = A l z ( k )t B I U ( ~ )
Aoz(k) Bo~(k)

that is

(Bo- Bl)u(k) = -(A0 - A l ) z ( k ) (3.3.3.2)

then G(mo(k),Q') = G ( m l ( k ) , Q u )and the conditions for the

convergence are not satisfied. From equation (3.3.3.1) we see
that ~ j ( k )= T ~ O ,Vk.

1988