\documentclass{article}
\usepackage{amssymb}
\usepackage{amsmath}
%\usepackage{slide-article}
\usepackage{slide-article-tom}
\ifx\pdfoutput\undefined
\usepackage[dvips]{graphicx}
\else
\usepackage[pdftex]{graphicx}
%% \usepackage{type1cm}
%% \usepackage{color}
\pdfcompresslevel9
\fi
% \usepackage{epsfig}
%\usepackage{graphics}
\usepackage{hyperref}
%\definecolor{Emerald}{cmyk}{1,0,0.50,0}
\hypersetup{colorlinks,
linkcolor=blue,
%pdfpagemode=FullScreen
pdfpagemode=None
}
%\usepackage{hyper}
%\usepackage{hthtml}
%\def\hyperref#1#2#3#4{\hturl{#1}}
\def\pagedone{\newpage}
\def\tthdump#1{#1}
\tthdump{\def\sectionhead#1{\begin{center}{\LARGE\hypertarget{#1}
{#1}\hyperlink{Our general topics:}{\hfil$\leftarrow$}}\end{center}}}
%%tth:\def\sectionhead#1{{\LARGE#1\hypertarget{#1}{#1}}
%%tth: \special{html: Top}}
\tthdump{\def\quotesection#1{\begin{center}{\LARGE\hypertarget{#1}
{#1}\hyperlink{The quotes}{\hfil$\twoheadleftarrow$}}\end{center}}}
%%tth:\def\quotesection#1{{\LARGE#1\hypertarget{#1}{#1}}
%%tth: \special{html: <-}}
%%tth:\def\makehyperlink#1{\special{html: }{\large#1}\special{html: }}
%%tth:\def\binom#1#2{\left(\begin{array}{c}#1\\#2\end{array}\right)}
%\def\sectionhead#1{\begin{center}{\LARGE #1}\end{center}}
%\def\sectionhead#1{\section{#1}}
% defines a 2 element column vector.
\def\col#1#2{\left(\begin{array}{c}#1\\#2\end{array}\right)}
\def\tcol#1#2{(#1, #2)^T}
% defines the expectation operator.
\def\expect#1{\langle #1 \rangle}
%%%\def\bigep{{\Large{$\varepsilon$}}}
\begin{document}
\raggedright
%%tth:\special{html: }
\pagestyle{myfooters}
%\pagestyle{plain}
\thispagestyle{empty}
%%tth:\special{html: Peripatetic}
%Slide 1
\title{{\LARGE\bf Peripatetic}\newline (A random walk\newline in random walks)\newline \newline}
\author{Tom Carter
\newline
\newline
\newline
\tthdump{\href{http://astarte.csustan.edu/\~tom/SFI-CSSS}{http://astarte.csustan.edu/\~\ tom/SFI-CSSS}}
%%tth:\href{http://astarte.csustan.edu/~tom/SFI-CSSS}{http://astarte.csustan.edu/\~tom/SFI-CSSS}
\vfill
Complex Systems Summer School
\newline
}
\date{Fall, 2008}
\maketitle
%Slide 2
\sectionhead{Our general topics:}
%%tth:\begin{itemize}
%%tth:\item
\tthdump{\hyperlink{Some random (variable) background}
{\ $\circledcirc$ Some random (variable) background\newline}}
%%tth:\makehyperlink{Some random (variable) background}
%%tth:\item
\tthdump{\hyperlink{What is a random walk?}
{$\circledcirc$ What is a random walk?\newline}}
%%tth:\makehyperlink{What is a random walk?}
%%tth:\item
\tthdump{\hyperlink{Some Intuitive Derivations}
{$\circledcirc$ Some Intuitive Derivations\newline}}
%%tth:\makehyperlink{Some Intuitive Derivations}
%%tth:\item
\tthdump{\hyperlink{Financial Modeling}
{$\circledcirc$ Financial Modeling\newline}}
%%tth:\makehyperlink{Financial Modeling}
%%tth:\end{itemize}
\pagedone
\sectionhead{Some random (variable) background}
This is a brief tour (peripatetic -- wandering about) of some topics in random
walks. It mostly consists of a few interesting ideas to start exploration
of the general topic. We'll start with a bit on random variables.
\begin{itemize}
\item A random variable is a view into a
set of possible values. Associated with each possible value is a probability.
For a discrete random variable, there is a finite or countably infinite set
of possible values and probabilities $\{(a_i, p_i)\}$, with the condition
that $\sum_i p_i = 1$. Thus, for example,
we could talk about the random variable $\xi$ drawing values from the
set of possible values $\{(1, 1/2), (-1, 1/2)\}$. When we evaluate the
random variable $\xi$, we get either 1 or -1, each with probability
1/2.
\pagedone
\item We can build new random variables. For example, if $\xi_1$ and $\xi_2$
are random variables over $\{(1, 1/2), (-1, 1/2)\}$, then $\xi_1 + \xi_2$
is a random variable (but over the set $\{(2, 1/4), (0, 1/2), (-2, 1/4)\}$).
We must be a bit careful sometimes -- for example, $\xi_1 + \xi_1$ is
also a random variable (but this time over $\{(2, 1/2), (-2, 1/2)\}$).
\item Given a random variable $X$ over $\{(a_i, p_i)\}$, we define the
{\em expectation} (or {\em expected value}) of $X$ by:
\[ \expect{X} = \sum_ip_ia_i. \]
Note that the expectation is a linear operator:
\[ \expect{\alpha X + \beta Y} = \alpha\expect{X} + \beta\expect{Y} \]
for $\alpha, \beta$ real numbers, and $X, Y$ random variables.
\pagedone
\item Note that the expectation of a constant $\alpha$ is that constant
(a constant can be thought of as a random variable over $\{(\alpha, 1)\}$):
\[ \expect{\alpha} = \alpha. \]
\item Example: if $\xi_1$ and $\xi_2$ are random variables over
$\{(1, 1/2), (-1, 1/2)\}$, then
\begin{eqnarray*}
\expect{\xi_1} & = & \frac{1}{2} * 1 + \frac{1}{2} * (-1)
= 0 = \expect{\xi_2}, \\
\expect{\xi_1 + \xi_2} & = & \expect{\xi_1} + \expect{\xi_2} = 0 + 0 = 0, \\
\expect{\xi_1\xi_2} & = & \frac{1}{2} * 1 + \frac{1}{2} * (-1) = 0,
\end{eqnarray*}
but
\[\expect{\xi_1^{\ 2}} = \frac{1}{2} * 1^2 + \frac{1}{2} * (-1)^2 = 1, \]
and
\begin{eqnarray*}
\expect{(\xi_1 + \xi_2)^2} & = & \expect{\xi_1^{\ 2} + 2 * \xi_1\xi_2
+ \xi_2^{\ 2}} \\
& = & \expect{\xi_1^{\ 2}} + 2 * \expect{\xi_1\xi_2}
+ \expect{\xi_2^{\ 2}} \\
& = & 1 + 0 + 1 \\
& = & 2.
\end{eqnarray*}
\pagedone
\item Given a random variable $X$, we define the {\em variance} of $X$ by:
\[ V(X) = \expect{(X - \expect{X})^2}. \]
We can also calculate this as:
\begin{eqnarray*}
V(X) & = & \expect{(X - \expect{X})^2} \\
& = & \expect{X^2 - 2 X \expect{X} + \expect{X}^2} \\
& = & \expect{X^2} - 2 \expect{X\expect{X}} + \expect{\expect{X}^2} \\
& = & \expect{X^2} - 2 \expect{X}^2 + \expect{X}^2 \\
& = & \expect{X^2} - \expect{X}^2.
\end{eqnarray*}
\item The {\em standard deviation} of a random variable $X$ is given by:
\[ \sigma(X) = V(X)^{1/2}. \]
\end{itemize}
\pagedone
\sectionhead{What is a random walk?}
There are a variety of ways to define a random walk. Here we
start with a relatively simple version, which will allow us to develop some
classical results on random walks. Later, we can generalize.
\begin{itemize}
\item Let $\{\xi_i | i = 1, 2, 3, \ldots\}$ be a set of (independent) random variables
over $\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$ (in particular, `observing' one
of the random variables has no effect on observations of any of the rest of them).
Then a {\em simple random walk} is a sequence $(S_n)$ where
\begin{eqnarray*}
S_0 & = & 0, \\
S_n & = & \xi_1 + \xi_2 + \ldots + \xi_n.
\end{eqnarray*}
\item It is easy to see that
\[ -n \le S_n \le n. \]
\pagedone
\item We have that
\[ \expect{S_0} = 0, \]
and thus
\begin{eqnarray*}
\expect{S_n} & = & \expect{S_{n-1} + \xi_n} \\
& = & \expect{S_{n-1}} + \expect{\xi_n} \\
& = & \expect{S_{n-1}} + 0 \\
& = & \expect{S_{n-1}} \\
& = & \expect{S_{n-2}} \\
& \vdots & \\
& = & \expect{S_0} \\
& = & 0.
\end{eqnarray*}
\item We also have
\begin{eqnarray*}
\expect{S_{n}^{\ 2}} & = & \expect{(S_{n-1} + \xi_n)^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2} + 2 * S_{n-1}\xi_n + \xi_n^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \expect{S_{n-1}\xi_n}
+ \expect{\xi_n^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}\expect{\xi_i\xi_n}
+ 1 \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}0
+ 1 \\
& = & \expect{S_{n-1}^{\ 2}} + 1 \\
& = & \expect{S_{n-2}^{\ 2}} + 2 \\
& \vdots & \\
& = & \expect{S_0^{\ 2}} + n \\
& = & n,
\end{eqnarray*}
and thus
\begin{eqnarray*}
V(S_n) & = & \expect{S_{n}^{\ 2}} - \expect{S_{n}}^2 \\
& = & n - 0 \\
& = & n.
\end{eqnarray*}
In other words, the variance of $S_n$ is $n$, and the standard deviation
of $S_n$ is $\sqrt{n}$.
\pagedone
\item We know that $S_n$ ranges between $-n$ and $n$. But how is it distributed
across that range? In other words, if $-n \le k \le n$, what is the probability
that $S_n = k$ (i.e., $P(S_n = k)$)? We can make a couple of observations. First,
by symmetry, $P(S_n = k) = P(S_n = -k)$, since each of the $\xi_i$ is over
$\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$. Second, by a parity argument,
if $n$ is even and $k$ is odd, or if $n$ is odd and $k$ is even,
then $P(S_n = k) = 0$. Let us look, then, at the case $n$ is even and
$k \ge 0$ is also even.
We know that $S_n = \sum_i\xi_i$, and that each of the $\xi_i$ is either -1
or 1. Thus, $S_n = k$ when exactly ($\frac{n}{2} + \frac{k}{2}$) of the
$\xi_i$ are +1 and the rest (i.e., $n - (\frac{n}{2} + \frac{k}{2})
= (\frac{n}{2} - \frac{k}{2})$) are -1. This can happen in
$\binom{n}{\frac{n}{2} + \frac{k}{2}}$ different ways. Each of these
is equally likely, and there are $2^n$ total possibilities.
\pagedone
Thus, we have that
\begin{eqnarray*}
P(S_n = k) & = & \binom{n}{\frac{n}{2} + \frac{k}{2}}\frac{1}{2^n} \\
& = & \frac{n!}{(\frac{n}{2} + \frac{k}{2})!
\ (n - (\frac{n}{2} + \frac{k}{2}))!\ 2^n} \\
\\
& = & \frac{n!}{(\frac{n + k}{2})!
\ (\frac{n - k}{2})!\ 2^n}
\end{eqnarray*}
We can also do the quick consistency check:
$$\sum_{k=-n}^{n}P(S_n = k)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
\ \ \ \ $$
\begin{eqnarray*}
& = & P(S_n = 0) + 2 \sum_{k=1}^{n}P(S_n = k) \\
& = & P(S_n = 0) + 2 \sum_{k=1}^{n/2}P(S_n = 2k) \\
& = & P(S_n = 0) + 2 \sum_{k=1}^{n/2}
\binom{n}{\frac{n}{2} + \frac{2k}{2}}\frac{1}{2^n}
\end{eqnarray*}
\begin{eqnarray*}
& = & P(S_n = 0) + \frac{1}{2^n}\ 2 \sum_{k=1}^{n/2}
\binom{n}{\frac{n}{2} + k} \\
& = & \frac{1}{2^n}\binom{n}{\frac{n}{2}}
+ \frac{1}{2^n}\ 2 \sum_{k=1}^{n/2}
\binom{n}{\frac{n}{2} + k} \\
& = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
+ \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}
+ \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}\right) \\
& = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
+ \sum_{k=1}^{n/2} \binom{n}{n - (\frac{n}{2} + k)}
+ \sum_{k=1}^{n/2} \binom{n}{\frac{n}{2} + k}\right) \\
& = & \frac{1}{2^n}\left(\binom{n}{\frac{n}{2}}
+ \sum_{k=0}^{n/2 - 1} \binom{n}{k}
+ \sum_{k=n/2 + 1}^{n} \binom{n}{k}\right) \\
& = & \frac{1}{2^n}\left(\sum_{k=0}^{n} \binom{n}{k}\right) \\
& = & \frac{1}{2^n} * 2^n \\
\\
& = & 1.
\end{eqnarray*}
(ugh . . . :-)
\pagedone
\item Let's generalize slightly, and suppose that our random walk may have
unequal probabilities of moving in the two directions. In other words,
suppose our random variables $\xi_i$ are over
$\{(1, p), (-1, q)\}$, with $q = 1 - p$.
In this case, given two such random variables $\xi_1$ and $\xi_2$, we have:
\begin{eqnarray*}
\expect{\xi_1} & = & p * 1 + q * (-1) = p - q \\
& = & 2p -1 = \expect{\xi_2}, \\
\expect{\xi_1 + \xi_2} & = & \expect{\xi_1} + \expect{\xi_2} = 2 * (2p - 1) \\
& = & 4p - 2, \\
\expect{\xi_1\xi_2} & = & (p^2 + q^2) * 1 + 2pq * (-1) \\
& = & p^2 - 2pq + q^2 = (p - q)^2 \\
& = & (2p -1)^2 = 4p^2 - 4p + 1.
\end{eqnarray*}
\pagedone
We also have:
\[\expect{\xi_1^{\ 2}} = p * 1^2 + q * (-1)^2 = p + q = 1, \]
and
\begin{eqnarray*}
\expect{(\xi_1 + \xi_2)^2} & = & \expect{\xi_1^{\ 2} + 2 * \xi_1\xi_2
+ \xi_2^{\ 2}} \\
& = & \expect{\xi_1^{\ 2}} + 2 * \expect{\xi_1\xi_2}
+ \expect{\xi_2^{\ 2}} \\
& = & 1 + 2 * (p - q)^2 + 1 \\
& = & 2 + 2 * (p - q)^2 \\
& = & 2 + 2 * (4p^2 - 4p + 1) \\
& = & 2 + 8 p^2 - 8p + 2 \\
& = & 4 + 8 p^2 - 8p.
\end{eqnarray*}
\pagedone
\item If we again let $S_0 = 0$ and $S_n = S_0 + \xi_1 + \ldots \xi_n$,
we have that
\[ \expect{S_0} = 0, \]
and in general
\begin{eqnarray*}
\expect{S_n} & = & \expect{S_{n-1} + \xi_n} \\
& = & \expect{S_{n-1}} + \expect{\xi_n} \\
& = & \expect{S_{n-1}} + p - q \\
& = & \expect{S_{n-2}} + \expect{\xi_{n-1}} + p - q \\
& = & \expect{S_{n-2}} + 2 * (p - q)\\
& \vdots & \\
& = & \expect{S_0} + n * (p - q)\\
& = & n * (p - q).
\end{eqnarray*}
\item We also have
\begin{eqnarray*}
\expect{S_{n}^{\ 2}} & = & \expect{(S_{n-1} + \xi_n)^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2} + 2 * S_{n-1}\xi_n + \xi_n^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \expect{S_{n-1}\xi_n}
+ \expect{\xi_n^{\ 2}} \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}\expect{\xi_i\xi_n}
+ 1 \\
& = & \expect{S_{n-1}^{\ 2}} + 2 * \sum_{i=1}^{n-1}(p - q)^2
+ 1 \\
& = & \expect{S_{n-1}^{\ 2}} + 2(n - 1)(p - q)^2 + 1 \\
& = & \expect{S_{n-2}^{\ 2}} + 2((n - 1) + (n - 2))(p - q)^2 + 2\\
& \vdots & \\
& = & \expect{S_0^{\ 2}} + 2\left(\sum_{i = 1}^{n - 1}i\right)(p - q)^2 + n \\
& = & 0 + n(n - 1)(p - q)^2 + n \\
& = & n + n(n - 1)(p - q)^2,
\end{eqnarray*}
\pagedone
and thus
\begin{eqnarray*}
V(S_n) & = & \expect{S_{n}^{\ 2}} - \expect{S_{n}}^2 \\
& = & n + n(n - 1)(p - q)^2 - n^2 * (p - q)^2 \\
& = & n - n(p - q)^2 \\
& = & n(p + q)^2 - n(p - q)^2 \\
& = & n((p + q)^2 - (p - q)^2) \\
& = & n(4pq) \\
& = & 4npq.
\end{eqnarray*}
In other words, the variance of $S_n$ is $4npq$, and the standard deviation
of $S_n$ is $2\sqrt{npq}$.
\end{itemize}
\pagedone
\sectionhead{Some Intuitive Derivations}
Every so often, I like to be a physicist (or biologist) and cavalier about
error bounds.
\begin{itemize}
\item Suppose $S_n = \sum_i\xi_i$ is a random walk, with $\xi_i$ random variables
over $\{(1, \frac{1}{2}), (-1, \frac{1}{2})\}$. Let us write
$P(S_n = k)$ as $P(k, n)$.
We can observe that
$$P(k, n + 1) = \frac{1}{2} P(k-1, n) + \frac{1}{2} P(k+1, n).$$
Now assume that $n$ and $k$ are large, let $\delta$ and $\tau$ be (small)
real numbers, and then let $x = \delta k$ and $t = \tau n$.
We then have $P(x, t) = P(\delta k, \tau n)$, and so
$P(x, t + \tau) = P(\delta k, \tau(n + 1)) $
\begin{eqnarray*}
& = & \frac{1}{2} P(\delta(k - 1), \tau n) + \frac{1}{2}P(\delta(k + 1), \tau n) \\
\\
& = & \frac{1}{2} P(x - \delta, t) + \frac{1}{2}P(x + \delta, t).
\end{eqnarray*}
\pagedone
From this, we get
$P(x, t + \tau) - P(x, t)$
$$ = \frac{1}{2}(P(x - \delta, t) + P(x + \delta, t) - 2 P(x, t)).$$
Now consider two approximations. First, for small (infinitesimal) $\tau$,
we have
$$P(x, t + \tau) = P(x, t) + \tau * \frac{\partial P(x, t)}{\partial t},$$
and for small (infinitesimal) $\delta$, we have
$P(x + \delta, t) + P(x - \delta, t) $
$$ = 2P(x, t) + \delta^2 * \frac{\partial^2P(x, t)} {\partial x^2}.$$
Putting pieces together, we have:
$$\frac{\partial P(x, t)}{\partial t} = \frac{\delta^2}{2\tau}
\frac{\partial^2P(x, t)} {\partial x^2}$$
(i.e., the diffusion equation \ldots).
\pagedone
More generally, if we have a biased random walk (over $\{(1, p), (-1, q)\}$),
then
$$ P(x, t + \tau) = p * P(x - \delta, t) + q * P(x + \delta, t),$$
and, using the approximations, we have
\begin{eqnarray*}
& &\tau * \frac{\partial P(x, t)}{\partial t} \\
\\
& = & P(x, t + \tau) - P(x, t) \\
\\
& = & p * P(x - \delta, t) + q * P(x + \delta, t) - P(x, t) \\
\\
& = & p * \left(P(x, t) - \delta * \frac{\partial P(x, t)}{\partial x}
+ \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}\right) \\
\\
& & + q * \left(P(x, t) + \delta * \frac{\partial P(x, t)}{\partial x}
+ \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}\right) \\
\\
& & - P(x, t) \\
& = & (p + q - 1) P(x, t) + (q - p) * \delta
* \frac{\partial P(x, t)}{\partial x} \\
& & + \frac{(p + q)\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2} \\
\\
& = & (q - p) * \delta * \frac{\partial P(x, t)}{\partial x}
+ \frac{\delta^2}{2} * \frac{\partial^2 P(x, t)}{\partial x^2}.
\end{eqnarray*}
\pagedone
Writing this in a slightly different form,
we have
$$\frac{\partial P(x, t)}{\partial t}
= D * \frac{\partial^2P(x, t)} {\partial x^2}
+ D * \beta * \frac{\partial P(x, t)}{\partial x},$$
where
$$D = \frac{\delta^2}{2 \tau}\ \mathrm{and}\ \beta = \frac{2(1 - 2p)}{\delta}$$
(i.e., diffusion with drift \ldots).
\pagedone
\item Let's look at another approach to continuous versions of these issues.
Instead of looking at random variables over a discrete set, let the
random variables draw their values from a probability distribution.
In particular, let
$$w: \Re \to [0, 1]$$
be an integrable (measurable)
function, with
$$\int_{-\infty}^\infty w(s) ds = 1.$$
Then a random variable $\xi$ over $w(s)$ gives the value $s$ with
probability $w(s)$ (i.e., $P(\xi = s) = w(s)$).
Note that we can use the Dirac delta function $\delta(x - x_0)$ to recover
the discrete examples if we want to.
\pagedone
Recall that the Dirac delta function has the
properties
\begin{eqnarray*}
\delta(x - x_0) dx & = & 0\ \ \mathrm{if}\ |x - x_0| > \frac{dx}{2} \\
\delta(x - x_0) dx & = & 1\ \ \mathrm{if}\ |x - x_0| \le \frac{dx}{2}
\end{eqnarray*}
and
$$ \int_{-\infty}^\infty \delta(x - x_0) dx = 1.$$
Then, if we let
$$ w(x) = \sum_i p_i * \delta(x - a_i),$$
we are back in the discrete case.
Let's mention here also that the delta function has the property:
$$\delta(x - x_0) = \frac{1}{2 \pi} \int_{-\infty}^\infty
e^{it(x_0 - x)} dt.$$
\pagedone
Given a random variable $\xi$ over a probability distribution $w(s)$,
we can look at the expected value of the random variable
$$\expect{\xi} = \int_{-\infty}^\infty s * w(s) ds,$$
the mean square
$$\expect{\xi^2} = \int_{-\infty}^\infty s^2 * w(s) ds,$$
the variance
$$V(\xi) = \expect{\xi^2} - \expect{\xi}^2,$$
and so on.
Now suppose we have a probability distribution $w(s)$ with mean $\mu$
and standard deviation $\sigma$ (i.e., if $\xi$ is a random variable
over $w(s)$, then $\mu = \expect{\xi}$, and $\sigma^2 = V(\xi)$). Note that
there is no guarantee for a given distribution $w(s)$ that either the mean
$\mu$ or standard deviation $\sigma$ exist -- the integrals could diverge. In
this example, we are assuming they do exist.
\pagedone
Let $\{\xi_i\}$ be a collection of (independent) random variables over
the distribution $w(s)$, and let $S_n = S_0 + \sum_1^n\xi_i$ (with $S_0 = 0$)
be a random walk. What can we say about the distribution of the values of
$\frac{1}{n}S_n$? In other words, what can we say about the distribution
of the average of $n$ (identically distributed) random variables?
We want to find the probability that $\frac{1}{n}S_n = x$ (let's write this as
$P_n(x)$). We will have $\frac{1}{n}S_n = x$ if $\xi_i = s_i$ and
$\frac{1}{n}\sum_is_i = x$.
The probability of this happening is $\prod_iw(s_i)$, since the $\xi_i$
are independent of each other. We need to add up the probabilities over
all possible ways that $\frac{1}{n}\sum_is_i = x$ (as we did in the discrete case).
In other words, we will have
$$P_n(x) = \underset{\frac{1}{n}\sum_is_i = x}{\int\int\cdots\int}w(s_1)\cdots w(s_n)
ds_1\cdots ds_n.$$
The limits of integration are fairly messy, so we will use the Dirac
delta function.
\pagedone
We will then have
$P_n(x)$
$$ = \int_{-\infty}^\infty\cdots\int_{-\infty}^\infty\delta(x - \frac{1}{n}\sum_js_j)
w(s_1)\cdots w(s_n)ds_1\cdots ds_n.$$
Using the (fourier transform) property of the delta function, we have
$2\pi P_n(x)$
\begin{eqnarray*}
&= & \int_{-\infty}^\infty...\int_{-\infty}^\infty
e^{it(\frac{1}{n}\sum_js_j - x)}
w(s_1)...w(s_n)dtds_1...ds_n \\
\\
&= & \int_{-\infty}^\infty\cdots\int_{-\infty}^\infty
e^{-itx}
\prod_j(e^{it\frac{s_j}{n}}w(s_j))dtds_1\cdots ds_n \\
&= & \int_{-\infty}^\infty e^{-itx}
\left(\prod_j\int_{-\infty}^\infty(e^{it\frac{s_j}{n}}w(s_j))ds_j\right)dt.
\end{eqnarray*}
If we now let $Q(t) = \int_{-\infty}^\infty e^{\frac{its}{n}}w(s)ds$, we have
$$P_n(x) = \frac{1}{2\pi}\int_{-\infty}^\infty e^{-itx}Q^n(t) dt.$$
\pagedone
Now let's look at $Q(t)$. We can expand the exponential to get
\begin{eqnarray*}
Q(t) & = & \int_{-\infty}^\infty e^{\frac{its}{n}}w(s)ds \\
& = & \int_{-\infty}^\infty w(s)(1 + \frac{its}{n}
- \frac{1}{2}\frac{t^2s^2}{n^2} \cdots)ds \\
& = & \int_{-\infty}^\infty w(s)ds
+ \frac{it}{n}\int_{-\infty}^\infty s * w(s)ds \\
& & - \frac{t^2}{2n^2} \int_{-\infty}^\infty s^2w(s)ds + \cdots \\
& = & 1 + \frac{it}{n}\expect{s} - \frac{1}{2n^2}t^2\expect{s^2} + \cdots
\end{eqnarray*}
Now we take the log, and use the expansion $\ln(1 + y) = y - \frac{1}{2}y^2 + \cdots$
to get
\begin{eqnarray*}
\ln(Q^n(t)) & = & n \ln(1 + \frac{it}{n}\expect{s}
- \frac{t^2}{2n^2}\expect{s^2} + \cdots) \\
& = & n * (\frac{it}{n}\expect{s} - \frac{1}{2n^2}t^2\expect{s^2}
- \frac{1}{2}(\frac{it}{n}\expect{s})^2 + \cdots)\\
& = & (it\expect{s}
- \frac{1}{2n}t^2(\expect{s^2} - \expect{s}^2) + \cdots) \\
& = & (it\mu - \frac{1}{2n}t^2\sigma^2 + \cdots)
\end{eqnarray*}
where $\mu$ and $\sigma$ are the mean and standard deviation of the distribution
$w(s)$.
\pagedone
Discarding all the higher order terms, and taking antilogs, we get
$$Q^n(t) = e^{it\mu - \frac{1}{2n}t^2\sigma^2},$$
and then that
$$P_n(x) = \frac{1}{2\pi}\int_{-\infty}^\infty
e^{it(\mu -x) - \frac{1}{2n}t^2\sigma^2}dt.$$
Now we use the formula
$$\int_{-\infty}^\infty e^{at - bt^2}dt = \sqrt{\frac{\pi}{b}}
* e^{\left(\frac{a^2}{4b}\right)}$$
with $a = i(\mu - x)$, $b = \frac{1}{2n}\sigma^2$, to finally get
\begin{eqnarray*}
P_n(x) & = & \frac{1}{2\pi}\int_{-\infty}^\infty
e^{it(\mu -x) - \frac{1}{2n}t^2\sigma^2}dt \\
& = & \frac{1}{2\pi}\sqrt{\frac{\pi}{\frac{1}{2n}\sigma^2}}
* e^{\left(-\frac{(\mu - x)^2}
{4 * \frac{1}{2n}\sigma^2}\right)} \\
& = & \frac{1}{\frac{\sigma}{\sqrt{n}}\sqrt{2 \pi}}
* e^{\left(-\frac{(x - \mu)^2}
{2\left(\frac{\sigma^2}{n}\right)}\right)}.
\end{eqnarray*}
In other words, it is a normal distribution with mean $\mu$ and standard deviation
$\frac{\sigma}{\sqrt{n}}$.
\pagedone
Note that we made no assumptions about the distribution $w(s)$ except that
it actually has a mean ($\mu$) and a standard deviation ($\sigma$) (and,
of course, that it goes to zero fast enough for large $|s|$ that the
approximations work out right \ldots).
What this says is that if we average a bunch of identically distributed
independent random variables, the result is a normal distribution, whether
or not the original distribution was normal.
This is usually called the {\em Central Limit Theorem}.
\end{itemize}
\pagedone
\sectionhead{Financial Modeling}
Let's use some of these ideas to do some financial modeling. As an example,
let's develop the (in)famous Black-Scholes model for options pricing.
\begin{itemize}
\item We can start with the simplest financial instrument, the fixed rate bond.
If we ``buy" amount $V_0$ of a rate $r$ bond at time $t=0$, then at time $t=1$
we can redeem the for value $V_0(1 + r)$. If we wait longer to redeem the bond,
then at some time in the future we can redeem the bond for
$$V(n, r) = V_0(1 + r)^n$$
One can also think of this as a ``savings account" with interest rate $r$. In this
case, we can ask the more general question, what is $V(t, r)$ for real values of
$t$, rather than just integral values of $t$?
This will depend on the specifics of the bond (savings account). In its simplest form,
the bond will have ``coupons" that can be redeemed at specific times in the future, or
in the case of a savings account, interest will be ``compounded" on specific dates.
Let's look at various possibilities for compounding. Suppose the bond has (annual)
interest rate $r$. If interest is compounded $k$ times during the year ($k$ would be
4 for quarterly compounding, 12 for monthly compounding, etc.), then the value
at time $t$ would be
$$V(t, r, k) = V_0(1 + \frac{r}{k})^{kt}$$
If we smooth this out, and let $k$ go to infinity (i.e., ``continuous compounding"),
then we will have
$$V(t, r) = V_0 \lim_{k \to \infty}((1 + \frac{r}{k})^{kt}) = V_0 e^{rt}.$$
We thus know how to set a price for a bond to be redeemed at some time $t$ in
the future.
\end{itemize}
\pagedone
\begin{itemize}
\item Now let's generalize. Suppose that the interest rate $r$, instead of being fixed,
varied over time. What price should we be willing to pay for such a financial
instrument? We can think of this financial instrument as a stock (share) in a corporation.
Our ``return on investment" will be uncertain, and will depend on the performance
of the corporation (and also on the change in price of the stock). We have the potential
to make a large profit (if the price of the stock goes up), but we now also have the
potential to lose money (if the price of the stock goes down).
There is a difficulty here in that we don't really have the flexibility to buy the stock
at whatever price we want today (depending on our calculation of the future value
of the stock), but can only buy at today's price.
One thing we can do (at least potentially, assuming there are sellers willing), is to purchase
an ``option" to buy the stock at some fixed price at some specific time in the future.
Let's simplify things a bit, and assume that over the time period in question, the stock
will not pay any dividends (in other words, our profit/loss will only depend on
changes in the price of the stock).
Once a market in ``options" is developed, a variety of things become possible. Not only
can we purchase options to buy a stock at a given price at a given time in the future
(a ``call" option), but we can also purchase an option to sell a stock at a given price at
a given time in the future (a ``put" option). Note that an option protects us against
an adverse change in the price of the stock. For example, if we purchase a ``call"
option, and the price of the stock goes down, we let the option expire, and we lose
only the amount we paid for the option (the premium). We are thus protected against
large losses. More generally, if we buy both ``put" and ``call" options, we can ``hedge"
our bets, and (we believe) protect ourselves against large losses, at the price of
limiting the amount of gain we might make.
Our task, then, is to develop a model that will allow us to determine (estimate?)
the price we should be willing to pay for an option.
In order to develop our model, we will have to make a variety of assumptions,
many of them ``simplifying" assumptions. It is possible (likely?) that at least some
or our assumptions will be unrealistic, but at least will allow us to do computations.
This presents us with an interesting dilemma -- if our model is unrealistic, we may
make very bad decisions if we depend on the model, but, if we make the model
``realistic," it may be useless to us because we can't do the computations.
Apparently, such is life \ldots
\pagedone
\item For this example, we're going to develop (a version of) the Black-Scholes
option pricing model.
We'll have to make a variety of assumptions. These will include:
\begin{enumerate}
\item There is a completely safe (e.g., ``FDIC insured savings account")
fixed rate asset available.
\item There are ``frictionless" markets (i.e., we can buy or sell any instrument at any
time in any amount).
\item There are no transaction costs.
\item No ``arbitrage" (there are no financial instruments that provide ``risk free"
profits above the fixed rate asset).
\end{enumerate}
\pagedone
The most critical assumptions we will have to make concern the form of variability of
the ``price" of the stock on which we will be buying our options.
\begin{enumerate}
\item Variability is continuous (and possibly even smooth?).
This will allow us to work in continuous time.
\item The distribution is ``stable" (i.e., the distribution of the variability
does not change over time).
\item Increments are independent (i.e., variability does not depend
on history -- there is no ``memory" in the distribution).
\item The distribution has a finite mean and finite variance.
\item Variability is ``independent of price" (i.e., the ``value" of a change
in price does not depend on the specific current price).
\end{enumerate}
\pagedone
Putting all of this together, we will assume that the stock price $x(t)$ gives us a
continuous random variable
$$R(x, t) = \ln\left(\frac{x(t)}{x(0)}\right).$$
This random variable $R(x, t)$ will be the return on the stock at time $t$, associated
with stock price $x(t)$. More specifically, we will assume that the price $x(t)$
satisfies the stochastic differential equation
$$dx(t) = \mu x(t)\,dt + \sigma x(t)\,dB_t$$
where $B_t$ is Brownian motion, $\mu$ is the ``drift," and $\sigma$ is the volatility.
This is generally called {\em geometric Brownian motion}. This equation has
the solution
$$x(t) = x(0)e^{\left( \left(\mu - \frac{\sigma^2}{2} \right)t + \sigma B_t\right)},$$
which is a log-normally distributed random variable with expected value
$\expect{x(t)}= x(0)e^{\mu t}$, and variance
$\operatorname{Var}(x(t)) = x(0)^2e^{2\mu t} \left( e^{\sigma^2 t}-1\right)$.
The random variable $R(x,t)$ is normally distributed, with mean
$(\mu - \sigma^2 / 2)t$ and variance $\sigma^2t$.
We are interested in determining the current value of a financial instrument that will have
a value in the future that depends on the price $x(t)$ (or return $R(x(t), t))$ of the underlying
asset (stock). Let us call the value of this derived instrument $w(x,t)$. In the Black-Scholes
case, $w(x,t)$ will be the value of a ``call' option.
To understand this better, we will study ``portfolios" of various financial instruments.
So, suppose we form a portfolio by putting one unit of our money into the secure
interest bearing asset, and an amount $-1/w_1$ (where
$w_1 = (\partial w / \partial x)$) in a ``call" option with
strike price $K$ at time $T$ (i.e., at time $T$ we can buy the stock a price $K$).
Then the value of the portfolio will be $p = x - w/w_1$.
During a short period of time $\Delta t$, the value of the portfolio will change by
$\Delta p = \Delta x - \Delta w / w_1$. We want to expand this formula, and
fortunately there is a nice method (part of the {\em Ito calculus}) that allows us
to write
$$\Delta w = \frac{\partial w}{\partial x} \Delta x + \frac{1}{2}\sigma^2 x^2
\frac{\partial^2 w}{\partial x \partial t} \Delta t + \frac{\partial w}{\partial t}\Delta t$$
(this is essentially the {\em chain rule} for {\em stochastic differentials}).
We then have
$$\Delta p = -\frac{1}{w_1}\left(\frac{1}{2}\sigma^2 x^2w_{11} + w_2\right)\Delta t$$
(where $w_1, w_{11},$ and $w_2$ are the appropriate partial derivatives, and
$\sigma^2$ is a constant depending on $w$, essentially its variance or volatility).
Now we will use the ``no arbitrage" assumption. Since $\Delta p$ is an assured return,
it must be that
$$\Delta p = r p \Delta t = r \left(x - \frac{w}{w_1}\right)\Delta t.$$
If we equate these two expressions for $\Delta p$, we get
$$ -\frac{1}{w_1}\left(\frac{1}{2}\sigma^2 x^2w_{11} + w_2\right)\Delta t
= r \left(x - \frac{w}{w_1}\right)\Delta t.$$
Dividing through by $\Delta t$, we can simplify to
$$ w_2 + r x w_1 + \frac{1}{2}\sigma^2 x^2 w_{11} -rw = 0.$$
This is the Black-Scholes differential equation (B-S PDE) for $w(x, t)$. It has boundary
conditions:
\begin{enumerate}
\item $w(0, t) = 0$ for all $t$.
\item $w(x, t) \sim x$ as $t \to \infty$.
\item $w(x, T) = \max(x - K, 0)$ (recall $K$ is the strike price at time $T$).
\end{enumerate}
\pagedone
\item Our next task is to develop a solution to the B-S equation.
We'll start by making some changes of variables. We'll reverse the order of time
(with some normalization), since the ``no arbitrage" rule allows us to use $t=T$ for
a boundary condition. We'll also move into a ``logarithmic" mode (since the riskless
asset grows exponentially), and normalize with respect to $K$ (the strike price) --
we shouldn't worry about the units of the price (e.g., dollars or pounds):
\begin{eqnarray*}
\tau & = & \frac{\sigma^2}{2}(T - t) \\
z & = & \ln(x/K) \\
v & = & \frac{w}{K}\\
\end{eqnarray*}
Now we do some calculations:
$$ \frac{\partial t}{\partial \tau} = - \frac{2}{\sigma^2}$$
and
$$\frac{\partial x}{\partial z} = x.$$
Then we have
\begin{eqnarray*}
\frac{\partial v}{\partial \tau} & = & \frac{1}{K}\frac{\partial w}{\partial \tau}
= \frac{1}{K}\left(\frac{\partial w}{\partial x}\frac{\partial x}{\partial \tau}
+ \frac{\partial w}{\partial t}\frac{\partial t}{\partial \tau}\right)\\
& = & \frac{1}{K}\left( - \frac{2}{\sigma^2}\right)w_2\\
\end{eqnarray*}
and
\begin{eqnarray*}
\frac{\partial v}{\partial z} & = & \frac{1}{K}\frac{\partial w}{\partial z}
= \frac{1}{K}\left(\frac{\partial w}{\partial x}\frac{\partial x}{\partial z}
+ \frac{\partial w}{\partial t}\frac{\partial t}{\partial z}\right)\\
& = & \frac{1}{K} x w_1. \\
\end{eqnarray*}
Using this, we get
\begin{eqnarray*}
\frac{\partial^2 v}{\partial z^2} & = & \frac{1}{K}\frac{\partial xw_1}{\partial z}
= \frac{1}{K}\left(\frac{\partial w_1}{\partial z}x
+ w1\frac{\partial x}{\partial z}\right)\\
& = & \frac{1}{K}\left(\frac{\partial w_1}{\partial x}\frac{\partial x}{\partial z}x
+ \frac{\partial w_1}{\partial t}\frac{\partial t}{\partial z}x
+ x w1\right)\\
& = & \frac{1}{K} (x^2 w_{11} + x w_1).\\
\end{eqnarray*}
\pagedone
Now we'll do some rearranging of the B-S equation, and then put in what
we have gotten from the change of variables. Starting from
$$ w_2 + r x w_1 + \frac{1}{2}\sigma^2 x^2 w_{11} -rw = 0.$$
we multiply by $\frac{-2}{K\sigma^2}$
$$ \frac{-2w_2}{K\sigma^2} + \frac{-2}{K\sigma^2} r x w_1
- \frac{1}{K} x^2 w_{11} + \frac{2}{K\sigma^2} rw = 0.$$
We rearrange, and add/subtract appropriate terms:
\begin{eqnarray*}
\frac{-2w_2}{K\sigma^2} & = & \frac{ x^2 w_{11}}{K}
+\frac{2 r x w_1}{K\sigma^2} - \frac{2 rw}{K\sigma^2} \\
\frac{-2w_2}{K\sigma^2} & = & \frac{ x^2 w_{11}}{K}
+ \frac{w_1 x}{K}
+\frac{2 r x w_1}{K\sigma^2}
+ \frac{w_1 x}{K}
- \frac{2 rw}{K\sigma^2} \\
\end{eqnarray*}
or,
\begin{eqnarray*}
\frac{-2w_2}{K\sigma^2} & = & \frac{1}{K}(x^2w_{11} + x w_1) \\
& \ & + \left(\frac{2r}{\sigma^2} - 1 \right)\frac{x w_1}{K}
- \frac{2 rw}{K\sigma^2} \\
\end{eqnarray*}
\pagedone
Then, putting in the change of variables, we have:
$$\frac{\partial v}{\partial \tau} = \frac{\partial^2 v}{\partial z^2}
+ (k - 1) \frac{\partial v}{\partial z} - kv$$
where $k = \frac{2r}{\sigma^2}$, and we have the
boundary condition $v(z, 0) = \max(e^z - 1, 0)$.
This is close to a heat equation, but we need one more change
of variables. If we let, for some constants $\alpha$, $\beta$,
$$u(z, \tau) = e^{\alpha x + \beta \tau} v(z, \tau)$$
we will have
\begin{eqnarray*}
\beta u + \frac{\partial u}{\partial \tau} & = & \alpha^2 u
+ 2\alpha \frac{\partial u}{\partial z}
+ \frac{\partial^2 u}{\partial z^2} \\
& & + (k - 1)\left(\alpha u + \frac{\partial u}{\partial z}\right) - ku \\
\end{eqnarray*}
If we choose
$$ \alpha = - \frac{1}{2}(k - 1)$$
and
$$ \beta = \alpha^2 + (k - 1) \alpha - k = - \frac{1}{4}(k + 1)^2,$$
\pagedone
then we are left with the heat equation:
$$ \frac{\partial u}{\partial \tau} = \frac{\partial^2 u}{\partial z^2}\ \ . $$
At this point we'll just appeal to standard methods for solving the heat
equation (e.g., Fourier series methods). The general solution will
be of the form:
$$ u(z, \tau) =
\frac{1}{\sigma\sqrt{2\pi\tau}}\int_{-\infty}^{\infty}u_0(y)e^{-(z - y)^2/(2\sigma^2\tau)}\,dy.$$
Undoing the changes of variable, we get:
$$w(x,t; K, T, r) = x\Phi(d_1) - Ke^{-r(T - t)}\Phi(d_2)\ ,$$
where $\Phi$ is the standard normal cumulative distribution function
$$\Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x e^{(-\frac{u^2}{2})}du$$
and
\begin{eqnarray*}
d_1 & = & \frac{\ln(x/K) + (r + \frac{1}{2}\sigma^2)(T - t)}{\sigma\sqrt{T - t}}\\
d_2 & = & \frac{\ln(x/K) + (r - \frac{1}{2}\sigma^2)(T - t)}{\sigma\sqrt{T - t}}\\
\end{eqnarray*}
\end{itemize}
c\`adl\`ag
\pagedone
\footnotesize
\bibliographystyle{plain}
\tthdump{\hypertarget{References}{}\hyperlink{Our general topics:}{\hfil \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ To top $\leftarrow$}}
%%tth:{\special{html: Top}}
\vspace{-1.0 in}
\begin{thebibliography}{12}
%%tth:{\special{html: }}
\bibitem{spitzer}
Spitzer, Frank,
{\em Principles of Random Walk}, 2nd ed.,
Springer, New York, 1976.
\bibitem{denny}
Denny, Mark and Gaines, Steven,
{\em Chance in Biology} - Using Probability to Explore Nature,
Princeton University Press, Princeton, 2000.
\bibitem{bachelier}
Bachelier, Louis, {\em Theory of \$peculation} - The Origins of Modern Finance,
Translated and with Commentary by Mark Davis and Alison Etheridge,
Princeton University Press, Princeton, 2006.
\end{thebibliography}
\tthdump{\hyperlink{Our general topics:}{\hfil To top $\leftarrow$}}
%%tth:{\special{html: Back to top of file}}
%
% \sectionhead{On-line references}
%
%
% Some of the references listed above are available on line. They are listed again here for easy access:
%
% %\bibitem{abrams2}
% Abrams D S and Lloyd S,
% Non-Linear Quantum Mechanics implies Polynomial Time
% solution for NP-complete and $\#$P problems,
% %in {\it LANL e-print} quant-ph/9801041, http://xxx.lanl.gov (1998)
% \hyperref{http://xxx.lanl.gov/abs/quant-ph/9801041}{}{}
% %\hyperURL{http}{xxx.lanl.gov/abs/quant-ph}{9801041}
% {http://xxx.lanl.gov/abs/quant-ph/9801041}
%
%
\end{document}
%% \item Oh, all right. Since we're here, let's use Stirling's approximation
%% $$n! \approx \sqrt{2\pi}\ n^ne^{-n}\sqrt{n}.$$
%% to calculate:
%%
%% $$P(S_n = k) = \frac{n!}{(\frac{n + k}{2})!
%% \ (\frac{n - k}{2})!\ 2^n} $$
%%
%% \begin{eqnarray*} & \approx & \frac{\sqrt{2\pi}\ n^ne^{-n}\sqrt{n}}
%% {\sqrt{2\pi}\ (\frac{n + k}{2})^{\frac{n + k}{2}}
%% e^{-\frac{n + k}{2}}\sqrt{\frac{n + k}{2}}
%% \ \sqrt{2\pi}\ (\frac{n - k}{2})^{\frac{n - k}{2}}
%% e^{-\frac{n - k}{2}}\sqrt{\frac{n - k}{2}}\ 2^n} \\
%% \\
%% & = & \frac{n^n\sqrt{n}}
%% {\sqrt{2\pi}\ (\frac{n + k}{2})^{\frac{n + k}{2}}
%% \sqrt{\frac{n + k}{2}}
%% \ \ (\frac{n - k}{2})^{\frac{n - k}{2}}
%% \sqrt{\frac{n - k}{2}}\ 2^n} \\
%% \\
%% & = & \frac{n^n\sqrt{n}}
%% {\sqrt{2\pi}\ (n + k)^{\frac{n + k}{2}}
%% \sqrt{\frac{n + k}{2}}
%% \ \ (n - k)^{\frac{n - k}{2}}
%% \sqrt{\frac{n - k}{2}}} \\
%% \\
%% & = & \frac{n^n\sqrt{\frac{4n}{(n+k)(n-k)}}}
%% {\sqrt{2\pi}\ (n + k)^{\frac{n + k}{2}}
%% \ (n - k)^{\frac{n - k}{2}}}
%% \end{eqnarray*}
%%
%% \pagedone
%%
%% \begin{eqnarray*}
%% & = & \frac{e^{n \ln(n)}\sqrt{\frac{4n}{(n + k)(n - k)}}}
%% {\sqrt{2\pi}\ e^{\frac{n + k}{2}\ln(n + k)}
%% \ e^{\frac{n - k}{2}\ln(n - k)}} \\
%% \\
%% & = & \frac{1}{\sqrt{2\pi}}e^{n \ln(n)-\frac{n + k}{2}\ln(n + k)
%% - \frac{n - k}{2}\ln(n - k)} \\
%% & & * \sqrt{\frac{4n}{(n + k)(n - k)}}
%% \end{eqnarray*}
%% so
%%
%% $\ln(P(s_n = k)) $
%% \begin{eqnarray*}
%% & \approx & \ln(\frac{1}{\sqrt{2\pi}})
%% + n \ln(n)-\frac{n + k}{2}\ln(n + k)\\
%% & & - \frac{n - k}{2}\ln(n - k)
%% + \frac{1}{2}\ln\left(\frac{4n}{(n + k) (n - k)}\right) \\
%% \\
%% & = & \ln(\frac{1}{\sqrt{2\pi}})
%% + n \ln(n)-\frac{n + k}{2}\ln(n + k)\\
%% & & - \frac{n - k}{2}\ln(n - k)
%% + \frac{1}{2}(\ln(4)) + \frac{1}{2}\ln(n)\\
%% & & - \frac{1}{2}\ln((n + k)(n - k))
%% \end{eqnarray*}
%%
%% \pagedone
%% \begin{eqnarray*}
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \frac{2n + 1}{2} \ln(n)-\frac{n + k + 1}{2}\ln(n + k)\\
%% & & - \frac{n - k + 1}{2}\ln(n - k) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \frac{2n + 1}{2}(\ln(n)
%% - \frac{n+k+1}{2n + 1}\ln(n + k) \\
%% & & - \frac{n-k+1}{2n + 1}\ln(n - k)) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \frac{1}{\frac{2}{2n + 1}}(\ln(n)
%% - \frac{n+k+1}{2n + 1}\ln(n + k) \\
%% & & - \frac{n-k+1}{2n + 1}\ln(n - k)) \\
%% \end{eqnarray*}
%%
%% so $ \lim_{n \to \infty} \ln(P(S_n = k)) $
%% \begin{eqnarray*}
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% - \lim_{n \to \infty}\frac{1}{\frac{4}{(2n+1)^2}}
%% (\frac{1}{n} \\
%% & & + \frac{2k + 1}{(2n + 1)^2}\ln(n+k)
%% - \frac{n + k + 1}{(2n + 1)(n + k)} \\
%% & & - \frac{2k - 1}{(2n + 1)^2}\ln(n-k)
%% - \frac{n - k + 1}{(2n + 1)(n - k)})
%% \end{eqnarray*}
%%
%% \pagedone
%%
%% \begin{eqnarray*}
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{1}{4}
%% (\frac{(2n+1)^2}{n} \\
%% & & + (2k + 1)\ln(n+k)
%% - \frac{(n + k + 1)(2n + 1)}{(n + k)} \\
%% & & - (2k - 1)\ln(n-k)
%% - \frac{(n - k + 1)(2n + 1)}{(n - k)}) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{1}{4}
%% (\frac{(2n+1)^2}{n} \\
%% & & - \frac{(n - k + 1)(2n + 1)}{(n - k)}
%% - \frac{(n + k + 1)(2n + 1)}{(n + k)} \\
%% & & + (2k + 1)\ln(n+k) - (2k - 1)\ln(n-k)) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{2n+1}{4}
%% (\frac{(2n+1)}{n} \\
%% & & - \frac{n - k + 1}{(n - k)}
%% - \frac{n + k + 1}{(n + k)}) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{2n+1}{4}
%% (\frac{(2n+1)(n-k)(n+k)}{n(n-k)(n+k)} \\
%% & & - \frac{n(n-k+1)(n+k) + n(n-k)(n+k+1)}{n(n-k)(n+k)})
%% \end{eqnarray*}
%%
%% \pagedone
%%
%% \begin{eqnarray*}
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{2n+1}{4}
%% (\frac{(2n+1)(n^2 - k^2)}{n^3 - nk^2} \\
%% & & - \frac{(n-k+1)(n^2+nk) + (n^2-nk)(n+k+1)}{n^3 - nk^2}) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{2n+1}{4} \\
%% & & (\frac{2n^3 - 2nk^2 + n^2 - k^2}{n^3 - nk^2} \\
%% & & - \frac{n^3 +n^2k - kn^2 -nk^2 +n^2 +nk}{n^3 - nk^2} \\
%% & & - \frac{n^3 +n^2k + n^2 -n^2k -nk^2 -nk}{n^3 - nk^2}) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}})
%% + \lim_{n \to \infty}\frac{2n+1}{4}
%% (\frac{- n^2 - k^2 -2nk}{n^3 - nk^2}) \\
%% \\
%% & = & \ln(\frac{2}{\sqrt{2\pi}}) - \frac{1}{2}
%% \end{eqnarray*}
%%
%%