Stochastic Processes, (Advanced Probability II)

Lecture Notes in PDF

Notes will be posted here before each lecture. The notes from last year are available. However, I will be revising them as we go along, and the division into lectures will differ somewhat, so you shouldn't completely rely on them rather than coming to class!

Contents

Including a running list of all definitions, lemmas, theorems, corollaries, examples and exercises to date.

Chapter 1: Stochastic Processes and Random Functions (16 January)

Stochastic processes as indexed collections of random variables and as random functions. Sample paths and constraints on them. Examples.

Chapter 2: Building Infinite Processes from Finite-Dimensional Distributions (18 January)

Finite-dimensional distributions of a process. Theorems of Daniell and Kolmogorov on extending finite-dimensional distributions to infinite-dimensional ones.

Chapter 3: Building Infinite Processes by Recursive Conditioning (23 January)

Probability kernels and regular conditional probabilities. Theorem of Ionescu Tuclea on constructing processes from regular conditional distributions.

Chapter 4: One-Parameter Processes (23 January)

One-parameter processes; examples thereof. Representation of one-parameter processes in terms of shift operators.

Chapter 5: Stationarity and Dynamics (25 January)

Strong, weak, and conditional stationarity. Stationarity as shift-invariance. Measure-preserving transformations and stationary processes.

Chapter 6: Random Times and Recurrence (30 January)

Reminders about filtrations and stopping times. Waiting times of various sorts, especially recurrence times. Poincaré and Kac recurrence theorems. "The eternal return of the same" and its statistical applications.

Chapter 7: Continuity (1 February)

Kinds of continuity for stochastic processes. Versions and modifications of stochastic processes. Benefits of continuous sample paths, and an example of the impossibility of deducing them from the finite dimensional distributions alone. Separable random functions.

Chapter 8: More on Continuity (1 February)

Existence of separable modifications of a stochastic process (in detail). Idea of measurable modifications. Conditions for the existence of measurable, cadlag and continuous modifications.

Chapter 9: Markov Processes (6 February)

Definition and meaning of the Markov property. Transition probability kernels. Existence of Markov processes with specified transitions. Invariant distributions. Dependence of the Markov property on filtrations.

Chapter 10: Two Views of Markov Processes (8 February)

Markov sequences as transformations of noise; transducers. Markov processes as collections of operators: Markov operator semi-groups evolve the distribution of states, and their adjoint operators evolve the "observables", the bounded measurable functions of the state. Some functional-analytic facts about conjugate spaces and adjoint operators.

Chapter 11: Examples of Markov Processes (13 February)

The logistic map as an example of turning nonlinear, deterministic dynamical systems into linear Markov operators. The Wiener process as an example of finding the transition kernels and time-evolution operators. Generalization of the Wiener process example to other processes with stationary and independent increments, and connections to limiting distributions of sums of IID random variables.

Chapter 12: Generators (15 February)

The generators of the semi-groups associated with Markov processes: analogy with exponential functions, how to find generators starting from semi-groups, some uses of generators for solving differential equations, Laplace transforms and resolvents. Hille-Yosida theorem on building semi-groups from generators.

Chapter 13: Strongly Markovian Processes and Martingale Problems (20 February)

The strong Markov property is being Markovian even at random times. An example of how a Markov process can fail to be strongly Markovian. The concept of a "martingale problem". Relationship between solutions of martingale problems and strong Markov processes.

Chapter 14: Feller Processes (22 February)

Clarificiations on initial states and distributions of Markov processes; Markov families, the probability kernel from initial states to paths. Definition of Feller processes and its physical motivations; reformulation in terms of semi-groups; unique correspondence between Feller processes and their generators. Attributes of Feller processes: cadlag sample paths, strong Markov property, Dynkin's formula.

Chapter 15: Convergence of Feller Processes (27 February)

Weak convergence of stochastic processes; hints as to the Skorokhod topology on cadlag functions; necessary and sufficient, and merely sufficient, conditions for convergence in distribution of cadlag processes. Convergence in distribution of Feller processes. Convergence of discrete-time Markov processes on Feller processes. Convergence of Markov processes on ordinary differential equations.

Chapter 16: Convergence of Random Walks (27 February and 1 March)

The Wiener process as a Feller process. Continuous-time random walks. Convergence of random walks to the Wiener process via the Feller-process machinery; via direct use of the theorems on weak convergence.

Chapter 17: Diffusions and the Wiener Process (6 March)

Definition of diffusions. The Wiener process as the prototypical diffusion. Resume of the Wiener process's properties. Wiener processes with respect to arbitrary filtrations. Gaussian processes. Wiener measure. Non-dfferentiability of almost-all continuous functions.

Chapter 18: Preview of Stochastic Integrals (8 March)

Why we want stochastic integrals. A heuristic approach via Euler's method (the Euler-Bernstein scheme).

Chapter 19: Stochastc Integrals and Stochastic Differential Equations (20--29 March)

Rigorous approach to stochastic integrals, after Ito. Ito integrals of "elementary" processes; extension to a wider class of integrands via approximation. Ito's isometry. Some simple but instructive examples. Ito processes. Ito's formula for change of variables. Stratonovich integrals (briefly). Representation of nice martingales as Ito integrals. Stochastic differential equations: existence and uniqueness of solutions. A more realistic model of Brownian motion, leading to a stochastic differential equation (the Langevin equation) and Ornstein-Uhlenbeck processes.

Chatper 20: More on Stochastic Differential Equations (29 March, 3 and 5 April)

Solutions of SDEs are Feller diffusions (as they solve martingale problems). The Kolmogorov "forward" and "backward" equations, for the evolution of the probability density and the observables, respectively. Examples of the forward or Fokker-Planck equation and its solution.

Chapter 21: A First Look at Small-Noise Limits of Stochastic Differential Equations (10 April)

SDEs defined by adding a small amount of white noise to ODEs. Solutions of the SDEs converge in distribution on the solutions of the ODE as the noise goes to zero (via Feller properties). An exponential upper bound on the probability of given magnitude of deviation between the solutions. Preview of coming attractions in large deviations theory.

Chapter 22: Spectral Analysis and Mean-Square Ergodicity (12--24 April)

"White noise" as a linear functional; its description as a generalized stochastic process (mean zero, uncorrelated, Gaussian) and equivalence to the Ito integral. Spectral representation of stochastic processes, especially weakly stationary ones, or, random Fourier transforms. The Wiener-Khinchin theorem linking the spectrum to the autocorrelation function. How the white noise lost its color. Our first ergodic theorem: convergence of time averages in mean square. A first proof assuming rapid decay of correlations. A stronger second proof based on the spectral representation.

Chapter 23: Ergodic Properties (26 April and 1 May)

Ideological remarks on ergodic theory. Dynamical systems and their invariants; connections to Markov processes. Ergodic properties of functions and ergodic limits. Asymptotic mean stationarity.

Chapter 24: Birkhoff's Ergodic Theorem (4 May)

The almost-sure or individual ergodic theorem of Birkhoff: in asymptotically mean-stationary systems, the time average of a well-behaved observable converges to its expectation, with probability 1.

References

Confined to works explicitly cited to date.

The entire set of notes, including future chapters.