\documentclass{article}
\usepackage{amssymb}
\usepackage{amsmath}
%\usepackage{slide-article}
\usepackage{slide-article-tom}
\ifx\pdfoutput\undefined
     \usepackage[dvips]{graphicx}
\else
     \usepackage[pdftex]{graphicx}
%%     \usepackage{type1cm}
%%     \usepackage{color}
     \pdfcompresslevel9
\fi
%  \usepackage{epsfig}
%\usepackage{graphics}
\usepackage{hyperref}

%\definecolor{Emerald}{cmyk}{1,0,0.50,0}
\hypersetup{colorlinks,
            linkcolor=blue,
            %pdfpagemode=FullScreen
            pdfpagemode=None
            }


%\usepackage{hyper}
%\usepackage{hthtml}
%\def\hyperref#1#2#3#4{\hturl{#1}}

\def\pagedone{\newpage}

\def\tthdump#1{#1}

\tthdump{\def\sectionhead#1{\begin{center}{\LARGE\hypertarget{#1}
     {#1}\hyperlink{Our general topics:}{\hfil$\leftarrow$}}\end{center}}}
     
%%tth:\def\sectionhead#1{{\LARGE#1\hypertarget{#1}{#1}}
%%tth:     \special{html: <A NAME="#1"></A><a href="\#Top of file">       Top</a>}}

\tthdump{\def\quotesection#1{\begin{center}{\LARGE\hypertarget{#1}
     {#1}\hyperlink{The quotes}{\hfil$\twoheadleftarrow$}}\end{center}}}
     
%%tth:\def\quotesection#1{{\LARGE#1\hypertarget{#1}{#1}}
%%tth:     \special{html: <A NAME="#1"></A><a href="\#The quotes">       <-</a>}}

%%tth:\def\makehyperlink#1{\special{html: <a href="\##1">}{\large#1}\special{html: </a>}}

%%tth:\def\binom#1#2{\left(\begin{array}{c}#1\\#2\end{array}\right)}

%\def\sectionhead#1{\begin{center}{\LARGE #1}\end{center}}
%\def\sectionhead#1{\section{#1}}

% defines a 2 element column vector.
\def\col#1#2{\left(\begin{array}{c}#1\\#2\end{array}\right)}
\def\tcol#1#2{(#1, #2)^T}

% defines the expectation operator.
\def\expect#1{\langle #1 \rangle}

%%%\def\bigep{{\Large{$\varepsilon$}}}

\begin{document}
\raggedright
%%tth:\special{html: <A NAME="Top of file"></A>}

\pagestyle{myfooters}
%\pagestyle{plain}

\thispagestyle{empty}


%%tth:\special{html:<title> Peripatetic</title>}

%Slide 1
\title{{\LARGE\bf Peripatetic}\newline (A random walk\newline in random walks)\newline \newline}
\author{Tom Carter
\newline
\newline
\newline
\tthdump{\href{http://astarte.csustan.edu/\~tom/SFI-CSSS}{http://astarte.csustan.edu/\~\ tom/SFI-CSSS}}
%%tth:\href{http://astarte.csustan.edu/~tom/SFI-CSSS}{http://astarte.csustan.edu/\~tom/SFI-CSSS}
\vfill
Complex Systems Summer School
\newline
}
\date{June, 2005}
\maketitle

%Slide 2
\sectionhead{Our general topics:}
%%tth:\begin{itemize}
%%tth:\item
\tthdump{\hyperlink{Some random (variable) background}
               {\ $\circledcirc$ Some random (variable) background\newline}}
%%tth:\makehyperlink{Some random (variable) background}
%%tth:\item
\tthdump{\hyperlink{What is a random walk?}
               {$\circledcirc$ What is a random walk?\newline}}
%%tth:\makehyperlink{What is a random walk?}
%%tth:\item
\tthdump{\hyperlink{Some Intuitive Derivations}
               {$\circledcirc$ Some Intuitive Derivations\newline}}
%%tth:\makehyperlink{Some Intuitive Derivations}

%%tth:\end{itemize}

\pagedone



\sectionhead{Some random (variable) background}

    This is a brief tour (peripatetic -- wandering about) of some topics in random
    walks.  It mostly consists of a few interesting ideas to start exploration
    of the general topic.  We'll start with a bit on random variables.  
    
\begin{itemize}

        \item A random variable is a view into a
           set of possible values.  Associated with each possible value is a probability.
           For a discrete random variable, there is a finite or countably infinite set
           of possible values and probabilities $\{(a_i, p_i)\}$, with the condition
           that $\sum_i p_i = 1$.  Thus, for example,
           we could talk about the random variable $\xi$ drawing values from the
           set of possible values $\{(1, 1/2), (-1, 1/2)\}$.  When we evaluate the
           random variable $\xi$, we get either 1 or -1, each with probability
           1/2.
           
\pagedone

        \item We can build new random variables.  For example, if $\xi_1$ and $\xi_2$
            are random variables over $\{(1, 1/2), (-1, 1/2)\}$, then $\xi_1 + \xi_2$
            is a random variable (but over the set $\{(2, 1/4), (0, 1/2), (-2, 1/4)\}$).
            We must be a bit careful sometimes -- for example, $\xi_1 + \xi_1$ is
            also a random variable (but this time over $\{(2, 1/2), (-2, 1/2)\}$).
            
        \item Given a random variable $X$ over $\{(a_i, p_i)\}$, we define the
            {\em expectation} (or {\em expected value}) of $X$ by:
            \[ \expect{X} = \sum_ip_ia_i. \]
            
            Note that the expectation is a linear operator:
            \[ \expect{\alpha X + \beta Y} = \alpha\expect{X} + \beta\expect{Y} \]
            for $\alpha, \beta$ real numbers, and $X, Y$ random variables.
            
\pagedone

        \item Note that the expectation of a constant $\alpha$ is that constant
            (a constant can be thought of as a random variable over $\{(\alpha, 1)\}$):
            \[ \expect{\alpha} = \alpha. \]

        \item Example: if $\xi_1$ and $\xi_2$ are random variables over
            $\{(1, 1/2), (-1, 1/2)\}$, then
            \begin{eqnarray*}
                \expect{\xi_1} & = & \frac{1}{2} * 1 + \frac{1}{2} * (-1)
                     = 0 = \expect{\xi_2}, \\
                \expect{\xi_1 + \xi_2} & = & \expect{\xi_1} + \expect{\xi_2} = 0 + 0 = 0, \\
                \expect{\xi_1\xi_2} & = & \frac{1}{2} * 1 + \frac{1}{2} * (-1) = 0,
            \end{eqnarray*}
            but
            \[\expect{\xi_1^{\ 2}} = \frac{1}{2} * 1^2 + \frac{1}{2} * (-1)^2 = 1, \]
            and
            \begin{eqnarray*}
                \expect{(\xi_1 + \xi_2)^2} & = & \expect{\xi_1^{\ 2} + 2 * \xi_1\xi_2
                    + \xi_2^{\ 2}} \\
                    & = & \expect{\xi_1^{\ 2}} + 2 * \expect{\xi_1\xi_2}
                    + \expect{\xi_2^{\ 2}} \\
                    & = & 1 + 0 + 1 \\
                    & = & 2.
            \end{eqnarray*}
            
\pagedone

        \item Given a random variable $X$, we define the {\em variance} of $X$ by:
            \[ V(X) = \expect{(X - \expect{X})^2}. \]
            We can also calculate this as:
            \begin{eqnarray*}
                V(X) & = & \expect{(X - \expect{X})^2} \\
                     & = & \expect{X^2 - 2 X \expect{X} + \expect{X}^2} \\
                     & = & \expect{X^2} - 2 \expect{X\expect{X}} + \expect{\expect{X}^2} \\
                     & = & \expect{X^2} - 2 \expect{X}^2 + \expect{X}^2 \\
                     & = & \expect{X^2} - \expect{X}^2.
             \end{eqnarray*}
             
         \item The {\em standard deviation} of a random variable $X$ is given by:
             \[ \sigma(X) = V(X)^{1/2}. \]
    \end{itemize}
\pagedone

\sectionhead{What is a random walk?}
     
    There are a variety of ways to define a random walk.  Here we
    start with a relatively simple version, which will allow us to develop some
    classical results on random walks.  Later, we can generalize.

    \begin{itemize}
    
        \item Let $\{\xi_i | i = 1, 2, 3, \ldots\}$ be a set of (independent) random variables
            over $\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$ (in particular, `observing' one
            of the random variables has no effect on observations of any of the rest of them).
            Then a {\em simple random walk} is a sequence $(S_n)$ where
            \begin{eqnarray*}
                S_0 & = & 0, \\
                S_n & = & \xi_1 + \xi_2 + \ldots + \xi_n.
            \end{eqnarray*}
            
        \item It is easy to see that
            \[ -n \le S_n \le n. \]
            
\pagedone
            
        \item  We have that
            \[ \expect{S_0} = 0, \]
            and thus
            \begin{eqnarray*}
                \expect{S_n} & = & \expect{S_{n-1} + \xi_n} \\
                             & = & \expect{S_{n-1}} + \expect{\xi_n} \\
                             & = & \expect{S_{n-1}} + 0 \\
                             & = & \expect{S_{n-1}} \\
                             & = & \expect{S_{n-2}} \\
                             & \vdots & \\
                             & = & \expect{S_0} \\
                             & = & 0.
            \end{eqnarray*}
            
        \item We also have
            \begin{eqnarray*}
                \expect{S_{n}^{\ 2}} & = & \expect{(S_{n-1} + \xi_n)^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2} + 2 * S_{n-1}\xi_n + \xi_n^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \expect{S_{n-1}\xi_n}
                              + \expect{\xi_n^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}\expect{\xi_i\xi_n}
                              + 1 \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}0
                              + 1 \\
                       & = & \expect{S_{n-1}^{\ 2}} + 1 \\
                       & = & \expect{S_{n-2}^{\ 2}} + 2 \\
                       & \vdots & \\
                       & = & \expect{S_0^{\ 2}} + n \\
                       & = & n,
            \end{eqnarray*}
            and thus
            \begin{eqnarray*}
                V(S_n) & = & \expect{S_{n}^{\ 2}} - \expect{S_{n}}^2 \\
                       & = & n - 0 \\
                       & = &  n.
            \end{eqnarray*}
            
            In other words, the variance of $S_n$ is $n$, and the standard deviation
            of $S_n$ is $\sqrt{n}$.
            
\pagedone

        \item  We know that $S_n$ ranges between $-n$ and $n$.  But how is it distributed
            across that range?  In other words, if $-n \le k \le n$, what is the probability
            that $S_n = k$ (i.e., $P(S_n = k)$)?  We can make a couple of observations.  First,
            by symmetry, $P(S_n = k) = P(S_n = -k)$, since each of the $\xi_i$ is over
            $\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$.  Second, by a parity argument,
            if $n$ is even and $k$ is odd, or if $n$ is odd and $k$ is even,
            then $P(S_n = k) = 0$.  Let us look, then, at the case $n$ is even and
            $k \ge 0$ is also even.
            
            We know that $S_n = \sum_i\xi_i$, and that each of the $\xi_i$ is either -1
            or 1.  Thus, $S_n = k$ when exactly ($\frac{n}{2} + \frac{k}{2}$) of the
            $\xi_i$ are +1 and the rest (i.e., $n - (\frac{n}{2} + \frac{k}{2})
            = (\frac{n}{2} - \frac{k}{2})$) are -1.  This can happen in
            $\binom{n}{\frac{n}{2} + \frac{k}{2}}$ different ways.  Each of these
            is equally likely, and there are $2^n$ total possibilities.
            
\pagedone

            Thus, we have that
            \begin{eqnarray*}
                P(S_n = k) & = & \binom{n}{\frac{n}{2} + \frac{k}{2}}\frac{1}{2^n} \\
                           & = & \frac{n!}{(\frac{n}{2} + \frac{k}{2})!
                                   \ (n - (\frac{n}{2} + \frac{k}{2}))!\ 2^n} \\
                           \\
                           & = & \frac{n!}{(\frac{n + k}{2})!
                                   \ (\frac{n - k}{2})!\ 2^n}
            \end{eqnarray*}
            
            We can also do the quick consistency check:
            $$\sum_{k=-n}^{n}P(S_n = k)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 
             \ \ \ \ $$
            \begin{eqnarray*}
                & = & P(S_n = 0) + 2 \sum_{k=1}^{n}P(S_n = k) \\
                      & = & P(S_n = 0) + 2 \sum_{k=1}^{n/2}P(S_n = 2k) \\
                      & = & P(S_n = 0) + 2 \sum_{k=1}^{n/2}
                              \binom{n}{\frac{n}{2} + \frac{2k}{2}}\frac{1}{2^n}
            \end{eqnarray*}
            \begin{eqnarray*}
                      & = & P(S_n = 0) + \frac{1}{2^n}\ 2 \sum_{k=1}^{n/2}
                              \binom{n}{\frac{n}{2} + k} \\
                      & = & \frac{1}{2^n}\binom{n}{\frac{n}{2}}
                              + \frac{1}{2^n}\ 2 \sum_{k=1}^{n/2}
                              \binom{n}{\frac{n}{2} + k} \\
                      & = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
                              + \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}
                              + \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}\right) \\
                      & = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
                              + \sum_{k=1}^{n/2} \binom{n}{n - (\frac{n}{2} + k)}
                              + \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}\right) \\
                      & = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
                              + \sum_{k=0}^{n/2 - 1} \binom{n}{k}
                              + \sum_{k=n/2 + 1}^{n} \binom{n}{k}\right) \\
                      & = & \frac{1}{2^n}\left(\sum_{k=0}^{n} \binom{n}{k}\right) \\
                      & = & \frac{1}{2^n} * 2^n \\
                      \\
                      & = & 1.
            \end{eqnarray*}
            (ugh . . . :-)
            
\pagedone

        \item Let's generalize slightly, and suppose that our random walk may have
            unequal probabilities of moving in the two directions.  In other words,
            suppose our random variables $\xi_i$ are over
            $\{(1, p), (-1, q)\}$, with $q = 1 - p$.
            
            In this case, given two such random variables $\xi_1$ and $\xi_2$, we have:
            \begin{eqnarray*}
                \expect{\xi_1} & = & p * 1 + q * (-1) = p - q \\
                      & = & 2p -1 = \expect{\xi_2}, \\
                \expect{\xi_1 + \xi_2} & = & \expect{\xi_1} + \expect{\xi_2} = 2 * (2p - 1) \\
                                      & = & 4p - 2, \\
                \expect{\xi_1\xi_2} & = & (p^2 + q^2) * 1 + 2pq * (-1) \\
                          & = & p^2 - 2pq + q^2 = (p - q)^2 \\
                          & = & (2p -1)^2 = 4p^2 - 4p + 1.
            \end{eqnarray*}
\pagedone
            We also have:
            \[\expect{\xi_1^{\ 2}} = p * 1^2 + q * (-1)^2 = p + q = 1, \]
            and
            \begin{eqnarray*}
                \expect{(\xi_1 + \xi_2)^2} & = & \expect{\xi_1^{\ 2} + 2 * \xi_1\xi_2
                    + \xi_2^{\ 2}} \\
                    & = & \expect{\xi_1^{\ 2}} + 2 * \expect{\xi_1\xi_2}
                    + \expect{\xi_2^{\ 2}} \\
                    & = & 1 + 2 * (p - q)^2 + 1 \\
                    & = & 2 + 2 * (p - q)^2 \\
                    & = & 2 + 2 * (4p^2 - 4p + 1) \\
                    & = & 2 + 8 p^2 - 8p + 2 \\
                    & = & 4 + 8 p^2 - 8p.
            \end{eqnarray*}

 
\pagedone

        \item  If we again let $S_0 = 0$ and $S_n = S_0 + \xi_1 + \ldots \xi_n$,
            we have that
            \[ \expect{S_0} = 0, \]
            and in general
            \begin{eqnarray*}
                \expect{S_n} & = & \expect{S_{n-1} + \xi_n} \\
                             & = & \expect{S_{n-1}} + \expect{\xi_n} \\
                             & = & \expect{S_{n-1}} + p - q \\
                             & = & \expect{S_{n-2}} + \expect{\xi_{n-1}} + p - q \\
                             & = & \expect{S_{n-2}} + 2 * (p - q)\\
                             & \vdots & \\
                             & = & \expect{S_0} + n * (p - q)\\
                             & = & n * (p - q).
            \end{eqnarray*}
            
        \item We also have
            \begin{eqnarray*}
                \expect{S_{n}^{\ 2}} & = & \expect{(S_{n-1} + \xi_n)^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2} + 2 * S_{n-1}\xi_n + \xi_n^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \expect{S_{n-1}\xi_n}
                              + \expect{\xi_n^{\ 2}} \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}\expect{\xi_i\xi_n}
                              + 1 \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}(p - q)^2
                              + 1 \\
                       & = & \expect{S_{n-1}^{\ 2}} + 2(n - 1)(p - q)^2 + 1 \\
                       & = & \expect{S_{n-2}^{\ 2}} + 2((n - 1) + (n - 2))(p - q)^2 + 2\\
                       & \vdots & \\
                       & = & \expect{S_0^{\ 2}} + 2\left(\sum_{i = 1}^{n - 1}i\right)(p - q)^2 + n \\
                       & = & 0 + n(n - 1)(p - q)^2 + n \\
                       & = & n + n(n - 1)(p - q)^2,
            \end{eqnarray*}
\pagedone
            and thus
            \begin{eqnarray*}
                V(S_n) & = & \expect{S_{n}^{\ 2}} - \expect{S_{n}}^2 \\
                       & = & n + n(n - 1)(p - q)^2 - n^2 * (p - q)^2 \\
                       & = & n - n(p - q)^2 \\
                       & = & n(p + q)^2 - n(p - q)^2 \\
                       & = & n((p + q)^2 - (p - q)^2) \\
                       & = & n(4pq) \\
                       & = & 4npq.
            \end{eqnarray*}
            
            In other words, the variance of $S_n$ is $4npq$, and the standard deviation
            of $S_n$ is $2\sqrt{npq}$.
     \end{itemize}
     
 \pagedone
 
 \sectionhead{Some Intuitive Derivations}
 
     Every so often, I like to be a physicist (or biologist) and cavalier about
     error bounds.
     \begin{itemize}
         \item Suppose $S_n = \sum_i\xi_i$ is a random walk, with $\xi_i$ random variables
             over $\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$.  Let us write
             $P(S_n = k)$ as $P(k, n)$.
             
             We can observe that
             $$P(k, n + 1) = \frac{1}{2} P(k-1, n) + \frac{1}{2} P(k+1, n).$$
             Now assume that $n$ and $k$ are large, let $\delta$ and $\tau$ be (small)
             real numbers, and then let $x = \delta k$ and $t = \tau n$.
             
             We then have $P(x, t) = P(\delta k, \tau n)$, and so
             
             $P(x, t + \tau) = P(\delta k, \tau(n + 1)) $
             \begin{eqnarray*}
                 & = & \frac{1}{2} P(\delta(k - 1), \tau n) + \frac{1}{2}P(\delta(k + 1), \tau n) \\
                 \\
                 & = & \frac{1}{2} P(x - \delta, t) + \frac{1}{2}P(x + \delta, t).
             \end{eqnarray*}
 \pagedone
         From this, we get
         
         $P(x, t + \tau) - P(x, t)$
         $$ = \frac{1}{2}(P(x - \delta, t) + P(x + \delta, t) - 2 P(x, t)).$$
         
         Now consider two approximations.  First, for small (infinitesimal) $\tau$,
         we have
         $$P(x, t + \tau) = P(x, t) + \tau * \frac{\partial P(x, t)}{\partial t},$$
         and for small (infinitesimal) $\delta$, we have
         
         $P(x + \delta, t) + P(x - \delta, t) $
         $$ = 2P(x, t) + \delta^2 * \frac{\partial^2P(x, t)} {\partial x^2}.$$
         Putting pieces together, we have:
         $$\frac{\partial P(x, t)}{\partial t} = \frac{\delta^2}{2\tau}
                 \frac{\partial^2P(x, t)} {\partial x^2}$$
         (i.e., the diffusion equation \ldots).
         
 \pagedone
 
         More generally, if we have a biased random walk (over $\{(1, p), (-1, q)\}$),
         then
             $$ P(x, t + \tau) = p * P(x - \delta, t) + q * P(x + \delta, t),$$
         and, using the approximations, we have
             \begin{eqnarray*}
                 &   &\tau * \frac{\partial P(x, t)}{\partial t} \\
                 \\
                 & = & P(x, t + \tau) - P(x, t) \\
                 \\
                 & = & p * P(x - \delta, t) + q * P(x + \delta, t) - P(x, t) \\
                 \\
                 & = & p * \left(P(x, t) - \delta * \frac{\partial P(x, t)}{\partial x}
                        + \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}\right) \\
                 \\
                 &   &  + q * \left(P(x, t) + \delta * \frac{\partial P(x, t)}{\partial x}
                        + \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}\right) \\
                 \\
                 &   &  - P(x, t) \\
                 & = & (p + q - 1) P(x, t) + (q - p) * \delta
                         * \frac{\partial P(x, t)}{\partial x} \\
                 &   &  + \frac{(p + q)\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2} \\
                 \\
                 & = & (q - p) * \delta * \frac{\partial P(x, t)}{\partial x}
                        + \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}.
             \end{eqnarray*}
 \pagedone
         Writing this in a slightly different form, 
         we have
             $$\frac{\partial P(x, t)}{\partial t}
                  = D * \frac{\partial^2P(x, t)} {\partial x^2}
                    + D * \beta * \frac{\partial P(x, t)}{\partial x},$$
                    
         where
             $$D = \frac{\delta^2}{2 \tau}\ \mathrm{and}\ \beta = \frac{2(1 - 2p)}{\delta}$$
         
         (i.e., diffusion with drift \ldots).
             
\pagedone
        \item Let's look at another approach to continuous versions of these issues.
            Instead of looking at random variables over a discrete set, let the
            random variables draw their values from a probability distribution.
            In particular, let
                $$w: \Re \to [0, 1]$$
            be an integrable (measurable)
            function, with
                $$\int_{-\infty}^\infty w(s) ds = 1.$$
            
            Then a random variable $\xi$ over $w(s)$ gives the value $s$ with
            probability $w(s)$ (i.e., $P(\xi = s) = w(s)$).
            
            Note that we can use the Dirac delta function $\delta(x - x_0)$ to recover
            the discrete examples if we want to.
\pagedone
            Recall that the Dirac delta function has the
            properties
            \begin{eqnarray*}
                \delta(x - x_0) dx & = & 0\ \ \mathrm{if}\ |x - x_0| > \frac{dx}{2} \\
                \delta(x - x_0) dx & = & 1\ \ \mathrm{if}\ |x - x_0| \le \frac{dx}{2}
            \end{eqnarray*}
            and
            $$ \int_{-\infty}^\infty \delta(x - x_0) dx = 1.$$
            
            Then, if we let
            $$ w(x) = \sum_i p_i * \delta(x - a_i),$$
            we are back in the discrete case.
            
            Let's mention here also that the delta function has the property:
            $$\delta(x - x_0) = \frac{1}{2 \pi} \int_{-\infty}^\infty
                e^{it(x_0 - x)} dt.$$
\pagedone
  
            Given a random variable $\xi$ over a probability distribution $w(s)$,
            we can look at the expected value of the random variable
            $$\expect{\xi} = \int_{-\infty}^\infty s * w(s) ds,$$
            the mean square
            $$\expect{\xi^2} = \int_{-\infty}^\infty s^2 * w(s) ds,$$
            the variance
            $$V(\xi) = \expect{\xi^2} - \expect{\xi}^2,$$
            and so on.
            
            Now suppose we have a probability distribution $w(s)$ with mean $\mu$
            and standard deviation $\sigma$ (i.e., if $\xi$ is a random variable
            over $w(s)$, then $\mu = \expect{\xi}$, and $\sigma^2 = V(\xi)$).  Note that
            there is no guarantee for a given distribution $w(s)$ that either the mean
            $\mu$ or standard deviation $\sigma$ exist -- the integrals could diverge.  In
            this example, we are assuming they do exist.
            
\pagedone
   
            Let $\{\xi_i\}$ be a collection of (independent) random variables over
            the distribution $w(s)$, and let $S_n = S_0 + \sum_1^n\xi_i$ (with $S_0 = 0$)
            be a random walk.  What can we say about the distribution of the values of
            $\frac{1}{n}S_n$?  In other words, what can we say about the distribution
            of the average of $n$ (identically distributed) random variables?
            
            We want to find the probability that $\frac{1}{n}S_n = x$ (let's write this as
            $P_n(x)$).  We will have $\frac{1}{n}S_n = x$ if $\xi_i = s_i$ and
            $\frac{1}{n}\sum_is_i = x$.
            The probability of this happening is $\prod_iw(s_i)$, since the $\xi_i$
            are independent of each other.  We need to add up the probabilities over
            all possible ways that $\frac{1}{n}\sum_is_i = x$ (as we did in the discrete case).
            In other words, we will have
            $$P_n(x) = \underset{\frac{1}{n}\sum_is_i = x}{\int\int\cdots\int}w(s_1)\cdots w(s_n)
                        ds_1\cdots ds_n.$$
            The limits of integration are fairly messy, so we will use the Dirac
            delta function.
            
\pagedone

            We will then have
            
            $P_n(x)$
            $$ = \int_{-\infty}^\infty\cdots\int_{-\infty}^\infty\delta(x - \frac{1}{n}\sum_js_j)
                    w(s_1)\cdots w(s_n)ds_1\cdots ds_n.$$
            Using the (fourier transform) property of the delta function, we have
            
            $2\pi P_n(x)$
            \begin{eqnarray*}
                &= & \int_{-\infty}^\infty...\int_{-\infty}^\infty
                    e^{it(\frac{1}{n}\sum_js_j - x)}
                    w(s_1)...w(s_n)dtds_1...ds_n \\
                \\
                &= & \int_{-\infty}^\infty\cdots\int_{-\infty}^\infty
                    e^{-itx}
                    \prod_j(e^{it\frac{s_j}{n}}w(s_j))dtds_1\cdots ds_n \\
                &= & \int_{-\infty}^\infty e^{-itx}
                    \left(\prod_j\int_{-\infty}^\infty(e^{it\frac{s_j}{n}}w(s_j))ds_j\right)dt.
            \end{eqnarray*}
            If we now let $Q(t) = \int_{-\infty}^\infty e^{\frac{its}{n}}w(s)ds$, we have
            $$P_n(x) = \frac{1}{2\pi}\int_{-\infty}^\infty e^{-itx}Q^n(t) dt.$$
            
\pagedone

            Now let's look at $Q(t)$.  We can expand the exponential to get
            \begin{eqnarray*}
                Q(t) & = & \int_{-\infty}^\infty e^{\frac{its}{n}}w(s)ds \\
                     & = & \int_{-\infty}^\infty w(s)(1 + \frac{its}{n}
                            - \frac{1}{2}\frac{t^2s^2}{n^2} \cdots)ds \\
                     & = & \int_{-\infty}^\infty w(s)ds
                            + \frac{it}{n}\int_{-\infty}^\infty s * w(s)ds \\
                     &   & - \frac{t^2}{2n^2} \int_{-\infty}^\infty s^2w(s)ds + \cdots \\
                     & = & 1 + \frac{it}{n}\expect{s} - \frac{1}{2n^2}t^2\expect{s^2} + \cdots
            \end{eqnarray*}
            Now we take the log, and use the expansion $\ln(1 + y) = y - \frac{1}{2}y^2 + \cdots$
            to get
            \begin{eqnarray*}
                \ln(Q^n(t)) & = & n \ln(1 + \frac{it}{n}\expect{s}
                            - \frac{t^2}{2n^2}\expect{s^2} + \cdots) \\
                     & = & n * (\frac{it}{n}\expect{s} - \frac{1}{2n^2}t^2\expect{s^2}
                         - \frac{1}{2}(\frac{it}{n}\expect{s})^2 + \cdots)\\
                     & = & (it\expect{s}
                         - \frac{1}{2n}t^2(\expect{s^2} - \expect{s}^2) + \cdots) \\
                     & = & (it\mu - \frac{1}{2n}t^2\sigma^2 + \cdots)
            \end{eqnarray*}
            where $\mu$ and $\sigma$ are the mean and standard deviation of the distribution
            $w(s)$.
  
\pagedone
 
            Discarding all the higher order terms, and taking antilogs, we get
            $$Q^n(t) = e^{it\mu - \frac{1}{2n}t^2\sigma^2},$$
            and then that
            $$P_n(x) = \frac{1}{2\pi}\int_{-\infty}^\infty
                    e^{it(\mu -x) - \frac{1}{2n}t^2\sigma^2}dt.$$
            Now we use the formula
            $$\int_{-\infty}^\infty e^{at - bt^2}dt = \sqrt{\frac{\pi}{b}}
                    * e^{\left(\frac{a^2}{4b}\right)}$$
            with $a = i(\mu - x)$, $b = \frac{1}{2n}\sigma^2$, to finally get 
            \begin{eqnarray*}
                P_n(x) & = & \frac{1}{2\pi}\int_{-\infty}^\infty
                              e^{it(\mu -x) - \frac{1}{2n}t^2\sigma^2}dt \\
                       & = & \frac{1}{2\pi}\sqrt{\frac{\pi}{\frac{1}{2n}\sigma^2}}
                              * e^{\left(-\frac{(\mu - x)^2}
                                  {4 * \frac{1}{2n}\sigma^2}\right)} \\
                       & = & \frac{1}{\frac{\sigma}{\sqrt{n}}\sqrt{2 \pi}}
                              * e^{\left(-\frac{(x - \mu)^2}
                                 {2\left(\frac{\sigma^2}{n}\right)}\right)}.
            \end{eqnarray*}
            In other words, it is a normal distribution with mean $\mu$ and standard deviation
            $\frac{\sigma}{\sqrt{n}}$.
            
\pagedone

            Note that we made no assumptions about the distribution $w(s)$ except that
            it actually has a mean ($\mu$) and a standard deviation ($\sigma$) (and,
            of course, that it goes to zero fast enough for large $|s|$ that the
            approximations work out right \ldots).
            
            What this says is that if we average a bunch of identically distributed
            independent random variables, the result is a normal distribution, whether
            or not the original distribution was normal.
            
            This is usually called the {\em Central Limit Theorem}.
     \end{itemize}

 
 \pagedone
 
 \footnotesize
 \bibliographystyle{plain}
 \tthdump{\hypertarget{References}{}\hyperlink{Our general topics:}{\hfil \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 
   \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ To top $\leftarrow$}}
%%tth:{\special{html: <A NAME="References"></A><a href="\#Top of file">       Top</a>}}
\vspace{-1.0 in}

\begin{thebibliography}{12}
%%tth:{\special{html: <font size="+0">}}


 \bibitem{spitzer}
 Spitzer, Frank,
 {\em Principles of Random Walk}, 2nd ed,
Springer, New York, 1976.

\bibitem{denny}
Denny, Mark and Gaines, Steven,
{\em Chance in Biology} - Using Probability to Explore Nature,
Princeton University Press, Princeton, 2000.



\end{thebibliography}

\tthdump{\hyperlink{Our general topics:}{\hfil To top $\leftarrow$}}
%%tth:{\special{html: <a href="\#Top of file">       Back to top of file</a>}}

% 
% \sectionhead{On-line references}
% 
% 
% Some of the references listed above are available on line.  They are listed again here for easy access:
% 
% %\bibitem{abrams2}
% Abrams D S and Lloyd S, 
% Non-Linear Quantum Mechanics implies Polynomial Time 
% solution for NP-complete and $\#$P problems,
% %in {\it LANL e-print} quant-ph/9801041, http://xxx.lanl.gov (1998)
% \hyperref{http://xxx.lanl.gov/abs/quant-ph/9801041}{}{}
% %\hyperURL{http}{xxx.lanl.gov/abs/quant-ph}{9801041}
% {http://xxx.lanl.gov/abs/quant-ph/9801041}
% 
% 


\end{document}
%%         \item Oh, all right.  Since we're here, let's use Stirling's approximation
%%             $$n! \approx   \sqrt{2\pi}\ n^ne^{-n}\sqrt{n}.$$
%%             to calculate:
%%             
%%             $$P(S_n = k) = \frac{n!}{(\frac{n + k}{2})!
%%                                    \ (\frac{n - k}{2})!\ 2^n} $$
%%                                    
%%             \begin{eqnarray*} & \approx & \frac{\sqrt{2\pi}\ n^ne^{-n}\sqrt{n}}
%%                                  {\sqrt{2\pi}\ (\frac{n + k}{2})^{\frac{n + k}{2}}
%%                                       e^{-\frac{n + k}{2}}\sqrt{\frac{n + k}{2}}
%%                                    \ \sqrt{2\pi}\ (\frac{n - k}{2})^{\frac{n - k}{2}}
%%                                    e^{-\frac{n - k}{2}}\sqrt{\frac{n - k}{2}}\ 2^n} \\
%%                               \\
%%                               & = & \frac{n^n\sqrt{n}}
%%                                  {\sqrt{2\pi}\ (\frac{n + k}{2})^{\frac{n + k}{2}}
%%                                       \sqrt{\frac{n + k}{2}}
%%                                    \ \ (\frac{n - k}{2})^{\frac{n - k}{2}}
%%                                    \sqrt{\frac{n - k}{2}}\ 2^n} \\
%%                               \\
%%                               & = & \frac{n^n\sqrt{n}}
%%                                  {\sqrt{2\pi}\ (n + k)^{\frac{n + k}{2}}
%%                                       \sqrt{\frac{n + k}{2}}
%%                                    \ \ (n - k)^{\frac{n - k}{2}}
%%                                    \sqrt{\frac{n - k}{2}}} \\
%%                               \\
%%                               & = & \frac{n^n\sqrt{\frac{4n}{(n+k)(n-k)}}}
%%                                  {\sqrt{2\pi}\ (n + k)^{\frac{n + k}{2}}
%%                                    \ (n - k)^{\frac{n - k}{2}}}
%%             \end{eqnarray*}
%%             
%% \pagedone
%% 
%%             \begin{eqnarray*}
%%                               & = & \frac{e^{n \ln(n)}\sqrt{\frac{4n}{(n + k)(n - k)}}}
%%                                  {\sqrt{2\pi}\ e^{\frac{n + k}{2}\ln(n + k)}
%%                                    \ e^{\frac{n - k}{2}\ln(n - k)}} \\
%%                               \\    
%%                               & = & \frac{1}{\sqrt{2\pi}}e^{n \ln(n)-\frac{n + k}{2}\ln(n + k)
%%                                     - \frac{n - k}{2}\ln(n - k)} \\
%%                               &   & * \sqrt{\frac{4n}{(n + k)(n - k)}}                                
%%             \end{eqnarray*}
%%             so
%%             
%%             $\ln(P(s_n = k)) $
%%             \begin{eqnarray*}
%%                               & \approx & \ln(\frac{1}{\sqrt{2\pi}}) 
%%                                      + n \ln(n)-\frac{n + k}{2}\ln(n + k)\\
%%                               &   & - \frac{n - k}{2}\ln(n - k)
%%                                     + \frac{1}{2}\ln\left(\frac{4n}{(n + k) (n - k)}\right) \\
%%                               \\
%%                               & = & \ln(\frac{1}{\sqrt{2\pi}}) 
%%                                      + n \ln(n)-\frac{n + k}{2}\ln(n + k)\\
%%                               &   & - \frac{n - k}{2}\ln(n - k)
%%                                     + \frac{1}{2}(\ln(4)) + \frac{1}{2}\ln(n)\\
%%                               &   & - \frac{1}{2}\ln((n + k)(n - k))                               
%%             \end{eqnarray*}
%%             
%% \pagedone
%%             \begin{eqnarray*}
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                      + \frac{2n + 1}{2} \ln(n)-\frac{n + k + 1}{2}\ln(n + k)\\
%%                               &   & - \frac{n - k + 1}{2}\ln(n - k) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                      + \frac{2n + 1}{2}(\ln(n)
%%                                      - \frac{n+k+1}{2n + 1}\ln(n + k) \\
%%                               &   &  - \frac{n-k+1}{2n + 1}\ln(n - k)) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                      + \frac{1}{\frac{2}{2n + 1}}(\ln(n)
%%                                      - \frac{n+k+1}{2n + 1}\ln(n + k) \\
%%                               &   &  - \frac{n-k+1}{2n + 1}\ln(n - k)) \\                           
%%             \end{eqnarray*}
%%             
%%             so $ \lim_{n \to \infty} \ln(P(S_n = k)) $
%%             \begin{eqnarray*}
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     - \lim_{n \to \infty}\frac{1}{\frac{4}{(2n+1)^2}}
%%                                          (\frac{1}{n} \\
%%                               &   &  + \frac{2k + 1}{(2n + 1)^2}\ln(n+k)
%%                                      - \frac{n + k + 1}{(2n + 1)(n + k)} \\
%%                               &   &  - \frac{2k - 1}{(2n + 1)^2}\ln(n-k)
%%                                      - \frac{n - k + 1}{(2n + 1)(n - k)})
%%             \end{eqnarray*}
%%             
%% \pagedone
%% 
%%             \begin{eqnarray*}
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{1}{4}
%%                                          (\frac{(2n+1)^2}{n} \\
%%                               &   &  + (2k + 1)\ln(n+k)
%%                                      - \frac{(n + k + 1)(2n + 1)}{(n + k)} \\
%%                               &   &  - (2k - 1)\ln(n-k)
%%                                      - \frac{(n - k + 1)(2n + 1)}{(n - k)}) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{1}{4}
%%                                          (\frac{(2n+1)^2}{n} \\
%%                               &   &  - \frac{(n - k + 1)(2n + 1)}{(n - k)}
%%                                      - \frac{(n + k + 1)(2n + 1)}{(n + k)} \\
%%                               &   &  + (2k + 1)\ln(n+k) - (2k - 1)\ln(n-k)) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{2n+1}{4}
%%                                          (\frac{(2n+1)}{n} \\
%%                               &   &  - \frac{n - k + 1}{(n - k)}
%%                                      - \frac{n + k + 1}{(n + k)}) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{2n+1}{4}
%%                                       (\frac{(2n+1)(n-k)(n+k)}{n(n-k)(n+k)} \\
%%                               &   & - \frac{n(n-k+1)(n+k) + n(n-k)(n+k+1)}{n(n-k)(n+k)})
%%             \end{eqnarray*}
%%             
%% \pagedone
%% 
%%             \begin{eqnarray*}
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{2n+1}{4}
%%                                       (\frac{(2n+1)(n^2 - k^2)}{n^3 - nk^2} \\
%%                               &   & - \frac{(n-k+1)(n^2+nk) + (n^2-nk)(n+k+1)}{n^3 - nk^2}) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{2n+1}{4} \\
%%                               &   &   (\frac{2n^3 - 2nk^2 + n^2 - k^2}{n^3 - nk^2} \\
%%                               &   & - \frac{n^3 +n^2k - kn^2 -nk^2 +n^2 +nk}{n^3 - nk^2} \\
%%                               &   & - \frac{n^3 +n^2k + n^2 -n^2k -nk^2 -nk}{n^3 - nk^2}) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}})
%%                                     + \lim_{n \to \infty}\frac{2n+1}{4}
%%                                     (\frac{- n^2 - k^2 -2nk}{n^3 - nk^2}) \\
%%                               \\
%%                               & = & \ln(\frac{2}{\sqrt{2\pi}}) - \frac{1}{2}
%%             \end{eqnarray*}
%%             
%%