Contents

Preface                                                                   x
CHAPTER 1 Introduction                                                    1
1.1. Randomness                                                           1
1.2. Stochastic processes                                                 5
1.3. Purposes of stochastic models                                        9
1.4. Overview                                                            12
1.5. Bibliographic remarks                                               13
1.6. Exercises                                                           14
CHAPTER 2 Discrete time Markov chains                                    16
2.1. Precipitation at Snoqualmie Falls                                   16
2.2. The marginal distribution                                           21
2.3. Classification of states                                            23
2.4. Stationary distribution                                             35
2.5. Long term behavior                                                  43
2.6. Markov chain Monte Carlo methods                                    52
2.7. Likelihood theory for Markov chains                                 58
2.8. Higher order chains                                                 70
2.9. Chain-dependent models                                              74
2.10. Random walks and harmonic analysis                                 82
2.11. Bienaym-Galton-Watson branching processes                          90
2.12. Hidden Markov models                                              103
2.13. Bibliographic remarks                                             112
2.14. Exercises                                                         114
CHAPTER 3 Continuous time Markov chains                                 125
3.1. The soft component of cosmic radiation                             125
3.2. The pure birth process                                             128
3.3. The Kolmogorov equations                                           133
3.4. A general construction                                             140
3.5. Queueing systems                                                   147
3.6. An improved model for cosmic radiation                             151
3.7. Statistical inference for continuous time Markov chains            153
3.8. Modeling neural activity                                           164
3.9. Blood formation in cats                                            172
3.10. Bibliographic remarks                                             181
3.11. Exercises                                                         181
CHAPTER 4. Markov random fields                                         189
4.1. The Ising model of ferromagnetism                                  189
4.2. Markov random fields                                               191
4.3. Phase transitions in Markov random fields                          196
4.4. Likelihood analysis of the Ising model                             200
4.5. Reconstruction of astronomical images                              203
4.6. Image analysis and pedigrees                                       209
4.7. Bibliographic remarks                                              219
4.8. Exercises                                                          219
CHAPTER 5. Point processes                                              227
5.1. A model of traffic patterns                                        227
5.2. General concepts                                                   230
5.3. Estimating second-order parameters for stationary point processes  238
5.4. Relationships between processes                                    241
5.5. Modeling the complete intensity                                    245
5.6. Marked point processes                                             250
5.7. Spatial point processes                                            260
5.8. Bibliographic remarks                                              268
5.9. Exercises                                                          270
CHAPTER 6. Brownian motion and diffusion                                276
6.1. Brownian motion                                                    276
6.2. Second-order processes                                             280
6.3. The Brownian motion process                                        283
6.4. A more realistic model of Brownian motion                          289
6.5. Diffusion equations                                                294
6.6. Likelihood inference for stochastic differential equations         301
6.7. The Wright-Fisher model of diploid populations                     305
6.8. Bibliographic remarks                                              311
6.9. Exercises                                                          311
APPENDIX A. Some statistical theory                                     318
A.1. Multinomial likelihood                                             318
A.2. The parametric case                                                319
A.3. Likelihood ratio tests                                             320
A.4. Sufficiency                                                        322
APPENDIX B. Linear difference equations with constant coefficients      325
B.1. The forward shift operator                                         325
B.2. Homogeneous difference equations                                   325
B.3. Non-homogeneous difference equations                               327
APPENDIX C. Some theory of partial differential equations               329
C.1. The method of auxiliary equations                                  329
C.2. Some applications                                                  330
References                                                              332
Index of results                                                        349
Applications and examples                                               351
Index of notation                                                       354
Index of terms                                                          359
Data sets                                                               371