Library of Congress Cataloging-in-Publication Data

Andrews, Larry C.

Field guide to probability, random processes, and random data analysis / Larry C. Andrews, Ronald L. Phillips.

p. cm. – (Field guide series)

Includes bibliographical references and index.

ISBN 978-0-8194-8701-8

1. Mathematical analysis. 2. Probabilities. 3. Random data (Statistics) I. Phillips, Ronald L. II. Title.

QA300.A5583 2012

519.2—dc23

2011051386

Published by

SPIE

P.O. Box 10

Bellingham, Washington 98227-0010 USA

Phone: +1.360.676.3290

Fax: +1.360.647.1445

Email: books@spie.org

Web: http://spie.org

The content of this book reflects the work and thoughts of the author. Every effort has been made to publish reliable and accurate information herein, but the publisher is not responsible for the validity of the information or for any outcomes resulting from reliance thereon.

Printed in the United States of America.

First printing

Introduction to the Series

Welcome to the SPIE Field Guides—a series of publications written directly for the practicing engineer or scientist. Many textbooks and professional reference books cover optical principles and techniques in depth. The aim of the SPIE Field Guides is to distill this information, providing readers with a handy desk or briefcase reference that provides basic, essential information about optical principles, techniques, or phenomena, including definitions and descriptions, key equations, illustrations, application examples, design considerations, and additional resources. A significant effort will be made to provide a consistent notation and style between volumes in the series.

Each SPIE Field Guide addresses a major field of optical science and technology. The concept of these Field Guides is a format-intensive presentation based on figures and equations supplemented by concise explanations. In most cases, this modular approach places a single topic on a page, and provides full coverage of that topic on that page. Highlights, insights, and rules of thumb are displayed in sidebars to the main text. The appendices at the end of each Field Guide provide additional information such as related material outside the main scope of the volume, key mathematical relationships, and alternative methods. While complete in their coverage, the concise presentation may not be appropriate for those new to the field.

The SPIE Field Guides are intended to be living documents. The modular page-based presentation format allows them to be easily updated and expanded. We are interested in your suggestions for new Field Guide topics as well as what material should be added to an individual volume to make these Field Guides more useful to you. Please contact us at fieldguides@SPIE.org.

John E. Greivenkamp, Series Editor

College of Optical Sciences

The University of Arizona

The Field Guide Series

Keep information at your fingertips with all of the titles in the Field Guide Series:

Field Guide to

Adaptive Optics, Robert Tyson & Benjamin Frazier

Atmospheric Optics, Larry Andrews

Binoculars and Scopes, Paul Yoder, Jr. & Daniel Vukobratovich

Diffractive Optics, Yakov G. Soskind

Geometrical Optics, John Greivenkamp

Illumination, Angelo Arecchi, Tahar Messadi, & John Koshel

Infrared Systems, Detectors, and FPAs, Second Edition, Arnold Daniels

Interferometric Optical Testing, Eric Goodwin & Jim Wyant

Laser Pulse Generation, Rüdiger Paschotta

Lasers, Rüdiger Paschotta

Microscopy, Tomasz Tkaczyk

Optical Fabrication, Ray Williamson

Optical Fiber Technology, Rüdiger Paschotta

Optical Lithography, Chris Mack

Optical Thin Films, Ronald Willey

Polarization, Edward Collett

Radiometry, Barbara Grant

Special Functions for Engineers, Larry Andrews

Spectroscopy, David Ball

Visual and Ophthalmic Optics, Jim Schwiegerling

Field Guide to Probability, Random Processes, and Random Data Analysis

Developed in basic courses in engineering and science, mathematical theory usually involves deterministic phenomena. Such is the case for solving a differential equation that describes a linear system where both input and output are deterministic quantities. In practice, however, the input to a linear system, such as imaging or radar systems, can contain a “random” quantity that yields uncertainty about the output. Such systems must be treated by probabilistic rather than deterministic methods. For this reason, probability theory and random-process theory have become indispensable tools in the mathematical analysis of these kinds of engineering systems.

Topics included in this Field Guide are basic probability theory, random processes, random fields, and random data analysis. The analysis of random data is less well known than the other topics, particularly some of the tests for stationarity, periodicity, and normality.

Much of the material is condensed from the authors’ earlier text Mathematical Techniques for Engineers and Scientists (SPIE Press, 2003). As is the case for other volumes in this series, it is assumed that the reader has some basic knowledge of the subject.

Larry C. Andrews

Professor Emeritus

Townes Laser Institute

CREOL College of Optics

University of Central Florida

Ronald L. Phillips

Professor Emeritus

Townes Laser Institute

CREOL College of Optics

University of Central Florida

Table of Contents

Glossary of Symbols and Notation x

Probability: One Random Variable 1

Terms and Axioms 2

Random Variables and Cumulative Distribution 3

Probability Density Function 4

Expected Value: Moments 5

Example: Expected Value 6

Expected Value: Characteristic Function 7

Gaussian or Normal Distribution 8

Other Examples of PDFs: Continuous RV 9

Other Examples of PDFs: Discrete RV 12

Chebyshev Inequality 13

Law of Large Numbers 14

Functions of One RV 15

Example: Square-Law Device 16

Example: Half-Wave Rectifier 17

Conditional Probabilities 18

Conditional Probability: Independent Events 19

Conditional CDF and PDF 20

Conditional Expected Values 21

Example: Conditional Expected Value 22

Probability: Two Random Variables 23

Joint and Marginal Cumulative Distributions 24

Joint and Marginal Density Functions 25

Conditional Distributions and Density Functions 26

Example: Conditional PDF 27

Principle of Maximum Likelihood 28

Independent RVs 29

Expected Value: Moments 30

Example: Expected Value 31

Bivariate Gaussian Distribution 32

Example: Rician Distribution 33

Functions of Two RVs 34

Sum of Two RVs 35

Product and Quotient of Two RVs 36

Conditional Expectations and Mean-Square Estimation 37

Sums of N Complex Random Variables 38

Central Limit Theorem 39

Example: Central Limit Theorem 40

Phases Uniformly Distributed on (−π, π) 41

Phases Not Uniformly Distributed on (−π, π) 42

Example: Phases Uniformly Distributed on (− α, α) 43

Central Limit Theorem Does Not Apply 45

Example: Non-Gaussian Limit 46

Random Processes 48

Random Processes Terminology 49

First- and Second-Order Statistics 50

Stationary Random Processes 51

Autocorrelation and Autocovariance Functions 52

Wide-Sense Stationary Process 53

Example: Correlation and PDF 54

Time Averages and Ergodicity 55

Structure Functions 56

Cross-Correlation and Cross-Covariance Functions 57

Power Spectral Density 58

Example: PSD 59

PSD Estimation 60

Bivariate Gaussian Processes 61

Multivariate Gaussian Processes 62

Examples of Covariance Function and PSD 63

Interpretations of Statistical Averages 64

Random Fields 65

Random Fields Terminology 66

Mean and Spatial Covariance Functions 67

1D and 3D Spatial Power Spectrums 68

2D Spatial Power Spectrum 69

Structure Functions 70

Example: PSD 71

Transformations of Random Processes 72

Memoryless Nonlinear Transformations 73

Linear Systems 74

Expected Values of a Linear System 75

Example: White Noise 76

Detection Devices 77

Zero-Crossing Problem 78

Random Data Analysis 79

Tests for Stationarity, Periodicity, and Normality 80

Nonstationary Data Analysis for Mean 81

Analysis for Single Time Record 82

Runs Test for Stationarity 83

Equation Summary 85

Biography 90

Index 91