376 MA THEMA TICS: J. L. DOOB PROC. N. A. S.

STOCHASTIC PROCESSES AND STATISTICS

BY J. L. DoOB*
DEPARTMENT OF MATHEMATICS, COLUMBIA UNIVERSITY
Communicated May 5, 1934
A stochastic process is defined by Khintchinel to be a one parameter
set of chance variables: x(t), - o < t < co. It is supposed that if ti,
..., th is any finite set of values of t, and aj < x < by, j = 1, . . .,n any set
of intervals, the probability that
aj < x(tj) < bj, j-= 1, .. ., n, 1
is defined. If the probability that (1) is true is independent of transla-
tions of the t-axis, the process is called stationary.
Let Q be any space in which a probability measure2 is defined. Let
{ TJ, - c < t < , be a set of one to one transformations of Q into itself,
such that Tt, + t2 = TI,Tt2. If X is an element of Q, the set of elements
Downloaded from https://www.pnas.org by 196.200.146.5 on June 12, 2024 from IP address 196.200.146.5.

{ Ttw }, - Co<. t < o is called a path curve. These path curves have been
studied, and many results, such as the ergodic theorem, obtained. It was
shown by Khintchinel that there is a formal analogy between this study
and that of stochastic processes. It will be shown in this paper that the
two theories are abstractly identical.
An example of a stochastic process can be obtained as follows. Let
Q, 'Tt be defined as in the preceding paragraph, and let sp(w) be a measurable
function on U. A chance variable is, by definition, a measurable function
on a space on which a probability measure is defined. We define x(t) as
the function qo(Tiw). The probability that (1) is true is then the measure
of the set of elements w such that
aj < p(Ts3w) < bj, j =1, ..., n. (2)
The following theorem shows that every stochastic process can be obtained
in this way. If. the stochastic process is stationary, the transformations
{ T } are measure preserving, and conversely.
THXEOREM 1. A stochastic process can always be considered as a set of
measurable functionals (pt(w), -0 < t < co on the function space a offunctions
x(t) = w defined for - c< t < co, on which a probability measure is defined.
If T is the transformation of Q taking x(t) into x(t + r), 1t(o) =-o(TOw).
Let { x(t) } be the chance variables of the given stochastic process.
There corresponds to every value of t a space Ot of elements w, on which a
probability measure is defined, and x(t) is a measurable function, (pt(w),
on 91. Transform (t into the x-axis by the transformation St which
takes the set of points w, at which (pl(wt) - a into the point x = a for each
value of a, Then (i(Cot) = x if x = S,w$, and the probability measure on
 VOL. 20, 1934 MA THEMA TICS: J. L. DOOB 377
(lt induces one on the x-axis. The chance variable x(t) becomes the
chance variable x *(t) = x. We can thus suppose that x2(t) is a chance
variable corresponding to a distribution on the x-axis, and that #pt(wj) =
,po(x) = x(t), i.e., that pt(ct) is the function x-where measure is defined
in a way depending on t. By hypothesis, probability is assigned to events
determined by conditions of the form
aj < x (tj) < by, j = I1, ...,p n, W1)
i.e., a probability measure is defined on the function space Q, the measure
being determined by its values on sets determined by conditions of the
form (1').3 The chance variable x(to) can be considered as the function
defined on Q which takes on the value x(to) at the element x(t) of U. This
functional is a measurable function on Q. The first part of the theorem
is thus proved. The second part is obvious.
Since x(t + r) = TT [x(t)], Theorem 1 shows that if the process is sta-
tionary, and if simple continuity conditions are satisfied,4 the ergodic
theorem of Birkhoff and similar theorems can be applied. For instance,
Downloaded from https://www.pnas.org by 196.200.146.5 on June 12, 2024 from IP address 196.200.146.5.

by the ergodic theorem, if x(O) (considered as a function on Q) is integrable

over Q (i.e., if the expectation of the chance variable x(O) exists) there is a
chance variable x such that
lim 1- rTxx(t)dt = x (3)
T-c TJ O
with probability 1.5
If we suppose that the chance variable x(t) of a stochastic process is
independent of t for n . t < n + 1 for every integer n, the specialization of
the above has important applications. The process then becomes es-
sentially a sequence ..., x--1, xo, xl, ... of chance variables, or, as in
Theorem 1, the stochastic process becomes a set of measurable functions
defined on the space Qs of all sequences (. . ., x-1, xO, xl, ...) on which a
probability measure is defined.6 Here (3) becomes
n
i
lim - xEic = X7 (3 )

true with probability 1, if the process is stationary, which means here

that the probability measure on %, is to be invariant under the trans-
formation T taking (. ..,x-, xo, xl, ....) into (..., xo, xi, x2, .. .), and if
the expectation of x1 exists.
The special case in which the chance variables xo, x1 ... are inde-
pendent and all have the same distributions is of importance. This
process is stationary, so (3') holds. It can be proved that in this case
the transformation T is metrically transitive. This means that x is a
constant8-in fact that x is the expectation of xi. Conversely the following
theorem can be proved.
 378 MA THEMA TICi: J. L. DOOB PROC. N. A. S.

THEOREM 2. Let xl, x2, ... be a sequence of independent ciance variables

with the same distributions. If there is a sequence of constants cl, c2,
such that the probability that
1 n
lim sup
n_az-
c
Zxj -c < o
n j_ =
(4)
is greater than zero, then the expectation E of xj exists, and

lim -, Ex2 = E (3")

n_-.c n j =
with probability 1.
This theorem depends on the fact that if xl, x2, ... is a sequence of
independent chance variables with the same distributions, a necessary
and sufficient condition that their expectations exist is that the probability
I
that lim sup xM/n < co is greater than 0.
This work can be considered as a generalization of the following fact.
Downloaded from https://www.pnas.org by 196.200.146.5 on June 12, 2024 from IP address 196.200.146.5.

n
If f(x) is the probability density9 of a chance variable x, II f(xj) is the
j=1
density for the distribution of the results of n independent trials. What
has been done above can be interpreted as letting n become infinite, ob-
taining a probability set-up which is suitable for any finite number of trials.
Using the results described above, a rigorous proof can be obtained of
the validity of the method of maximum likelihood of R. A. Fisher, which
has supplanted the use of Bayes' Theorem.'0 Let f(x, p) for each value of
p in a neighborhood of a value po be a probability density. Let x be a
chance variable whose distribution has density f(x, Po). The problem is
to estimate po by means of samples of values of x. The method of Fisher
n
is to choose the value p (xi, ..., x.) of p which maximizes nlf(xo, p) for
j-1
fixed xi, .. ., x". Then it can be shown using the above results, that under
suitable restrictions on the continuity of f(x, p) in p, pn(X1, . . ., X") ap-
proaches po with probability 1, and that for large n the distribution of
PM (XlX.. X.) is nearly normal, with mean Po and variance 1/no2, where
_Jf f(x, po) ap2 [(log f(x, po)] dx.
'Mathemat. Ann., 109, 604-615 (1934); this paper should be compared with a paper
by the same author in these PROCEEDINGS, 19, 567-573 (1933);
2 We shall mean by this that there is a non-negative completely additive set function,
defined on a collection of sets of elements of the space (called measurable sets). If
A,, A2, ... are measurable sets, we suppose that their complements and their sum are
measurable. We suppose that the set of all elements in the space has measure 1.
Measurable functions and Lebesgue integration can be defined in the usual way.
 VOL. 20, 1934 PHYSICS: R. C. TOLMAN 379
3This way of defining measure in function space was discussed by Kolmogoroff,
Ergebnisse Mathematik, 2, No. 3, Grundbegriffe der Wahrscheinlichkeitsrechnung, § 4.
4 It is sufficient that if E is any set in O determined by conditions of the form (1'),
and if E is transformed into Et by Tt, the measure of E Et should be continuous in
tat t = 0.
c For a simple proof of the ergodic theorem, following the lines of the first proof, given
by Birkhoff, cf. A. Khintchine, Mathemat. Ann., 107, 485-488 (1933).
B This situation was discussed by Khintchine, Zeit. Angewandte Mathemat. Mechanik,
13, 101-103 (1933), who treated the particular case of chance variables taking on only
the values 1 or 0. The general case was discussed by E. Hopf, Journal of Mathematics
and Physics, M. I. T., 13, 51-102 (1934), who obtained (3') but not Theorem 2.
7 Kolmogoroff, loc. cit.,3 p. 59, announced this result in the special case of independent
chance variables, and announced also Theorem 2, under the assumption that the prob-
ability is 1 that the upper limit in (4) is 0.
8 Loc. cit.,6 p. 488.
9 If f(x) is defined for - X < x < co except possibly for a set of points of Lebesgue
measure 0, is Lebesgue measurable, not negative, and integrable over - X < x < co,
co

f f(x)dx
- 00
= 1, f(x) will be called a probability density.
10 This method was discussed (unrigorously) by Fisher in the Phil. Trans. Roy. Soc.
Downloaded from https://www.pnas.org by 196.200.146.5 on June 12, 2024 from IP address 196.200.146.5.

London, Series A, 222 (1921). The treatment of H. Hotelling, Trans. Amer. Math.
Soc., 32, 847-859 (1930), holds only in certain special cases.
* NATIONAL RESEARCH FELLOW.

REMARKS ON THE POSSIBLE FAILURE OF ENERGY CON-

SER VA TION
BY RicHARD C. TOLMAN
NoRMAN BRIDGE LABORATORY OF PHYSICS, CALIFORMA INSTITUTE OF TECHNOLOGY
Communicated April 17, 1934
1. Possibility of Failure in the Case of Elementary Processes.-The
possible failure of the principle of the conservation of energy, in the case of
the continuous j#-ray spectrum accompanying radioactive decompositions,
and perhaps also in the case of processes occurring in the interior of stars,
has several times been suggested by Bohr.' From a theoretical point of
view such a failure might be due to a breakdown in the applicability of
ordinary mechanical notions under circumstances where the electron would
have to be regarded as localizable within regions small compared with its
classical dimensions.
In the case of the stars there is at present no definite observational evi-
dence which would lead us to abandon the principle of the conservation of
energy, beyond the removal of limitations on our attempts to explain the
continued luminosity of those objects and to account, in general, for the
existence of a supply of available energy in the universe.