Lecture 5. The Daniell-Kolmogorov existence theorem

The Daniell-Kolmogorov extension theorem is one of the first deep theorems of the theory of stochastic processes. It provides existence results for nice probability measures on path (function) spaces. It is however non-constructive and relies on the axiom of choice. In what follows, in order to avoid heavy notations we restrict to the one dimensional case $d=1$ . The multidimensional extension is straightforward and let to the reader.

Definition. Let $(X_t)_{t \ge 0}$ be a stochastic process. For $t_1 , ... , t_n \in \mathbb{R}_{\ge 0}$ we denote by $\mu_{t_1,...,t_n}$ the probability distribution of the random variable $(X_{t_1},...,X_{t_n}).$ It is therefore a probability measure on $\mathbb{R}^n$ . This probability measure is called a finite dimensional distribution of the process $(X_t)_{t \ge 0}$ .

If two processes have the same finite dimensional distributions, then it is clear that the two processes induce the same distribution on the path space $\mathcal{A} (\mathbb{R}_{\ge 0}, \mathbb{R})$ because cylinders generate the $\sigma$ -algebra $\mathcal{T} (\mathbb{R}_{\ge 0}, \mathbb{R})$ (see Lecture 2).

The finite dimensional distributions of a given process satisfy the two following properties: If $t_1,...,t_n \in \mathbb{R}_{\ge 0}$ and if $\tau$ is a permutation of the set $\{1,...,n\}$ , then:

$\mu_{t_1,...,t_n} (A_1 \times ... \times A_n)=\mu_{t_{\tau (1)},...,t_{\tau (n)}} (A_{\tau(1)} \times ... \times A_{\tau(n)}),\quad A_i \in \mathcal{B}(\mathbb{R}).$
$\mu_{t_1,...,t_n} (A_1 \times ... \times A_{n-1} \times \mathbb{R})=\mu_{t_1,...,t_{n-1}} (A_1 \times ... \times A_{n-1}),\quad A_i \in \mathcal{B}(\mathbb{R}).$

Conversely,

Theorem (Daniell-Kolmogorov theorem). Assume that we are given for every $t_1,...,t_n \in \mathbb{R}_{\ge 0}$ a probability measure $\mu_{t_1,...,t_n}$ on $\mathbb{R}^n$ . Let us assume that these probability measures satisfy:

$\mu_{t_1,...,t_n} (A_1 \times ... \times A_n)=\mu_{t_{\tau (1)},...,t_{\tau (n)}} (A_{\tau(1)} \times ... \times A_{\tau(n)}),\quad A_i \in \mathcal{B}(\mathbb{R}).$

$\mu_{t_1,...,t_n} (A_1 \times ... \times A_{n-1} \times \mathbb{R})=\mu_{t_1,...,t_{n-1}} (A_1 \times ... \times A_{n-1}),\quad A_i \in \mathcal{B}(\mathbb{R}).$

Then, there is a unique probability measure $\mu$ on $(\mathcal{A}(\mathbb{R}_+, \mathbb{R}), \mathcal{T}(\mathbb{R}_{\ge 0},\mathbb{R}))$ such that for $t_1,...,t_n \in \mathbb{R}_{\ge 0}$ , $A_1,...,A_n \in \mathcal{B}(\mathbb{R})$ :

$\mu (\pi_{t_1} \in A_1,...,\pi_{t_n} \in A_n)=\mu_{t_1,...,t_n}(A_1 \times ... \times A_n).$

The Daniell-Kolmogorov theorem is often used to construct processes thanks to the following corollary:

Corollary. Assume given for every $t_1,...,t_n \in \mathbb{R}_{\ge 0}$ a probability measure $\mu_{t_1,...,t_n}$ on $\mathbb{R}^n$ . Let us further assume that these measures satisfy the assumptions of the Daniell-Kolmogorov theorem. Then, there exists a probability space $\left( \Omega , \mathcal{F}, \mathbb{P} \right)$ as well as a process $(X_t)_{t \ge 0}$ defined on this space such that the finite dimensional distributions of $(X_t)_{t \ge 0}$ are given by the $\mu_{t_1,...,t_n}$ ‘s.

Proof of the corollary:

As a probability space we chose

$\left( \Omega , \mathcal{F}, \mathbb{P} \right)=\left( \mathcal{A} ( \mathbb{R}_{\ge 0} ,\mathbb{R}), \mathcal{T}(\mathbb{R}_{\ge 0},\mathbb{R}), \mu \right)$

where $\mu$ is the probability measure given by the Daniell-Kolmogorov theorem. The canonical coordinate process $(\pi_t)_{t \ge 0}$ defined on $\mathcal{A}( \mathbb{R}_{\ge 0} ,\mathbb{R})$ by $\pi_t (f)=f(t)$ satisfies the required property. $\square$

We now turn to the proof of the Daniell-Kolmogorov theorem. This proof proceeds in several steps.

As a first step, let us recall the Caratheodory extension theorem that is often useful for the effective construction of measures (for instance the construction of the Lebesgue measure on $\mathbb{R}$ ):

Theorem (Caratheodory theorem). Let $\Omega$ be a non-empty set and let $\mathcal{A}$ be a family of subsets that satisfy:

$\Omega \in \mathcal{A}$ ;
If $A,B \in \mathcal{A}$ , $A \cup B \in \mathcal{A}$ ;

If $A \in \mathcal{A}$ , $\Omega \backslash A \in \mathcal{A}$ .

Let $\sigma ( \mathcal{A} )$ be the $\sigma$ -algebra generated by $\mathcal{A}$ . If $\mu_0$ is $\sigma$ -additive measure on $( \Omega, \mathcal{A} )$ which is $\sigma$ -finite, then there exists a unique $\sigma$ -additive measure $\mu$ on $( \Omega, \sigma(\mathcal{A}) )$ such that for $A \in \mathcal{A}$ , $\mu_0 (A)=\mu(A).$

As a first step, we prove the following fact:

Lemma. Let $B_n \subset \mathbb{R}^n$ , $n \in \mathbb{N}$ be a sequence of Borel sets that satisfy $B_{n+1} \subset B_n \times \mathbb{R}.$ Let us assume that for every $n \in \mathbb{N}$ a probability measure $\mu_n$ is given on $(\mathbb{R}^n, \mathcal{B} (\mathbb{R}^n))$ and that these probability measures are compatible in the sense that

$\mu_{n} (A_1 \times ... \times A_{n-1} \times \mathbb{R})=\mu_{n-1} (A_1 \times ... \times A_{n-1}),\quad A_i \in \mathcal{B}(\mathbb{R})$

and satisfy:

$\mu_n (B_n) > \varepsilon,$

where $0<\varepsilon<1$ . There exists a sequence of compact sets $K_n \subset \mathbb{R}^n$ , $n \in \mathbb{N}$ , such that:

$K_n \subset B_n$

$K_{n+1} \subset K_n \times \mathbb{R}$ .

$\mu_n (K_n) \ge \frac{\varepsilon}{2}$ .

Proof of the lemma:

For every $n$ , we can find a compact set $K_n^* \subset \mathbb{R}^n$ such that

$K^*_n \subset B_n$

and

$\mu_n (B_n \backslash K^*_n )\le \frac{\varepsilon}{2^{n+1}}.$

Let now

$K_n=(K^*_1 \times \mathbb{R}^{n-1} ) \cap ... \cap (K^*_{n-1} \times \mathbb{R}) \cap K^*_n.$

It is easily checked that:

$K_n \subset B_n$
$K_{n+1} \subset K_n \times \mathbb{R}$

Moreover,

$\mu_n (K_n)$
$=\mu_n (B_n) - \mu_n (B_n \backslash K_n)$
$=\mu_n (B_n)-\mu_n \left( B_n \backslash \left( (K^*_1 \times \mathbb{R}^{n-1} ) \cap ... \cap (K^*_{n-1} \times\mathbb{R}) \cap K^*_n \right) \right)$
$\ge \mu_n (B_n)-\mu_n \left( B_n \backslash \left( (K^*_1 \times \mathbb{R}^{n-1} )\right) \right)-...-\mu_n \left( B_n \backslash \left( K^*_{n-1} \times \mathbb{R} \right) \right)-\mu_n (B_n \backslash K_n^*)$
$\ge \mu_n (B_n)-\mu_1 (B_1 \backslash K_1^*)-...-\mu_n (B_n \backslash K_n^*)$
$\ge \varepsilon-\frac{\varepsilon}{4}-...-\frac{\varepsilon}{2^{n+1}}$
$\ge \frac{\varepsilon}{2}.$

$\square$

With this in hands, we can now turn to the proof of the Daniell-Kolmogorov theorem.

Proof of the Daniell-Kolmogorov theorem:

For the cylinder

$\mathcal{C}_{t_1,...,t_n} (B)= \{ f \in \mathcal{A}(\mathbb{R}_+,\mathbb{R}), (f(t_1) ,...,f(t_n) )\in B \}$

where $t_1,...,t_n \in \mathbb{R}_{\ge 0}c$ and where $B$ is a Borel subset of $\mathbb{R}^n$ , we define

$\mu \left(\mathcal{C}_{t_1,...,t_n} (B)\right)=\mu_{t_1,...,t_n} (B).$

Thanks to the assumptions on the $\mu_{t_1,...,t_n}$ ‘s, it is seen that such a $\mu$ is well defined and satisfies:

$\mu \left( \mathcal{A}( \mathbb{R}_{\ge 0}, \mathbb{R}) \right)=1.$

The set $\mathcal{A}$ of all the possible cylinders $\mathcal{C}_{t_1,...,t_n} (B)$ satisfies the assumption of Caratheodory’s theorem. Therefore, in order to conclude, we have to show that $\mu$ is $\sigma$ -additive, that is, if $\left( C_n \right)_{n \in \mathbb{N}}$ is a sequence of pairwise disjoint cylinders and if $C=\cup_{n \in \mathbb{N}} C_n$ is a cylinder then

$\mu \left( C \right)=\sum_{n=0}^{+\infty} \mu (C_n).$

This is the difficult part of the theorem. Since for $N\in \mathbb{N}$ ,

$\mu (C)=\mu \left( C \backslash \cup_{n =0}^N C_n \right)+\mu \left( \cup_{n =0}^N C_n \right),$

we just have to show that

$\lim_{N \rightarrow + \infty} \mu \left( D_N \right)=0.$

where $D_N=C \backslash \cup_{n =0}^N C_n$ .

The sequence $(\mu(D_N))_{N\in \mathbb{N}}$ is positive decreasing and therefore converges. Let assume that it converges toward $\varepsilon >0$ . We shall prove that in that case

$\cap_{N \in \mathbb{N}} D_N \neq \emptyset,$

which is clearly absurd.

Since $D_N$ is a cylinder, the event $\cup_{N \in \mathbb{N}} D_N$ only involves a coutable sequence of times $t_1<...<t_n<...$ and we may assume (otherwise we can add convenient other sets in the sequence of the $D_N$ 's) that every $D_N$ can be described as follows

$D_N =\{ f \in \mathcal{A} (\mathbb{R}_{\ge 0},\mathbb{R}), (f(t_1),...,f(t_N)) \in B_N \}$

where $B_n \subset \mathbb{R}^n$ , $n \in \mathbb{N}$ , is a sequence of Borel sets such that

$B_{n+1} \subset B_n \times \mathbb{R}.$

Since we assumed $\mu (D_N) \ge \varepsilon$ , we can use the previous lemma to construct a sequence of compact sets $K_n \subset \mathbb{R}^n$ , $n \in \mathbb{N}$ , such that:

$K_n \subset B_n$
$K_{n+1} \subset K_n \times \mathbb{R}$
$\mu_{t_1,...,t_n} (K_n) \ge \frac{\varepsilon}{2}$

Since $K_n$ is non-empty, we pick $(x_1^n,...,x_n^n) \in K_n.$

The sequence $(x_1^n)_{n\in \mathbb{N}}$ has a convergent subsequence $(x_1^{j_1(n)})_{n\in \mathbb{N}}$ that converges toward $x_1 \in K_1$ . The sequence $((x_1^{j_1(n)},x_2^{j_1 (n)})_{n\in \mathbb{N}}$ has a convergent subsequence that converges toward $(x_1,x_2) \in K_2$ . By pursuing this process, we obtain a sequence $(x_n)_{n \in \mathbb{N}}$ such that for every $n$ , $(x_1,...,x_n) \in K_n.$

The event

$\{ f \in \mathcal{A} (\mathbb{R}_+,\mathbb{R}), (f(t_1),...,f(t_N)) =(x_1,...,x_N) \}$

is in $D_N$ , this leads to the expected contradiction. Therefore, the sequence $(\mu(D_N))_{N\in \mathbb{N}}$ converges toward $0$ , which implies the $\sigma$ -additivity of $\mu$ $\square$

As it has been stressed, the Daniell-Kolmogorov theorem is the basic tool to prove the existence of a stochastic process with given finite dimensional distributions. As an example, let us illustrate how it may be used to prove the existence of the so-called Gaussian processes.

Definition. A real-valued stochastic process $(X_t)_{t \ge 0}$ defined on $(\Omega , \mathcal{F}, \mathbb{P})$ is said to be a Gaussian process if all the finite dimensional distributions of $X$ are Gaussian random variables.

If $(X_t)_{t \ge 0}$ is a Gaussian process, its finite dimensional distributions can be characterized, through Fourier transform, by its mean function

$m(t)=\mathbb{E} (X_t)$

and its covariance function

$R(s,t)=\mathbb{E} \left( (X_t -m(t)) (X_s -m(s)) \right).$

We can observe that the covariance function $R(s,t)$ is symmetric $(R(s,t)=R(t,s))$ and positive, that is for $a_1,...,a_n \in \mathbb{R}$ and $t_1,...,t_n \in \mathbb{R}_{\ge 0}$ ,

$\sum_{1 \le i,j \le n} a_i a_j R(t_i,t_j)$
$=\sum_{1 \le i,j \le n} a_i a_j \mathbb{E} \left( (X_{t_i} -m(t_i)) (X_{t_j}-m(t_j)) \right)$
$= \mathbb{E} \left( \left( \sum_{i=1}^n a_i (X_{t_i}-m(t_i)) \right)^2 \right) \ge 0$

Conversely, as an application of the Daniell-Kolmogorov theorem, we let the reader prove as an exercise the following proposition.

Proposition. Let $m:\mathbb{R}_{\ge 0} \rightarrow \mathbb{R}$ and let $R: \mathbb{R}_{\ge 0} \times \mathbb{R}_{\ge 0} \rightarrow \mathbb{R}$ be a symmetric and positive function. There exists a probability space $\left( \Omega , \mathcal{F}, \mathbb{P} \right)$ and a Gaussian process $(X_t)_{t \ge 0}$ defined on it, whose mean function is $m$ and whose covariance function is $R$ .

2 Responses to Lecture 5. The Daniell-Kolmogorov existence theorem

luc says:

November 6, 2014 at 3:44 pm

Hi,
in the statement of Caratheodory’s theorem, for the uniqueness of extension, do we need the sigma-finiteness of $\mu_0$ ?

If $\mu_0(\Omega)$ is finite, then by an application of monotone class theorem, we get the uniqueness. If sigma finite, we can consider a countable partition with each part of finite measure, then use previous result.

best.

- Fabrice Baudoin says:
  
  November 6, 2014 at 10:55 pm
  
  Yes, we need $\mu_0$ to be $\sigma$-finite. I corrected, thanks.