\documentclass{article} \usepackage{amssymb} \usepackage{amsmath} \usepackage{slide-article-tom} \ifx\pdfoutput\undefined \usepackage[dvips]{graphicx} \else \usepackage[pdftex]{graphicx} %% \usepackage{type1cm} %% \usepackage{color} \pdfcompresslevel9 \fi %\usepackage{psfig} \usepackage{hyperref} %\usepackage{hyper} %\usepackage{hthtml} %\def\hyperref#1#2#3#4{\hturl{#1}} \def\pagedone{\newpage} \def\sectionhead#1{\begin{center}{\LARGE #1}\end{center}} %\def\sectionhead#1{\section{#1}} % kets and bras etc. \def\ket#1{|{#1}\rangle} \def\bra#1{\langle{#1}|} \def\braket#1#2{\langle{#1}|{#2}\rangle} \newcommand {\ra} {\rangle} \newcommand {\la} {\langle} %Symbol \def\hilbert{\mathit{H}} \def\complex{\mathbb{C}} \def\tensor{\otimes} \def\lnanda{\land\negthickspace\negthickspace\negthinspace\sim} \def\lnandb{\land\negthickspace\negthickspace\negmedspace\negthinspace\sim} \def\lnandc{\land\negthickspace\negthickspace\negmedspace\negthinspace\negthinspace\sim} \def\lnandd{\land\negthickspace\negthickspace\negmedspace\negthinspace\negthinspace\negthinspace\sim} % defines a 2 element column vector. \def\col#1#2{\left(\begin{array}{c}#1\\#2\end{array}\right)} \def\tcol#1#2{(#1, #2)^T} \begin{document} \raggedright \pagestyle{myfooters} %\pagestyle{plain} \thispagestyle{empty} %Slide 1 \title{{\LARGE\bf A Little Probability} \\ \dots \\ \dots \\ Coding and Information Theory \\ Fall, 2004} \author{Tom Carter \newline \newline http://astarte.csustan.edu/\~\ tom/} \date{October, 2004} \maketitle %Slide 2 \sectionhead{Some probability background} \begin{itemize} \item There are two notions of the {\em probability} of an event happening. The two general notions are: \begin{enumerate} \item A {\em frequentist} version of probability: In this version, we assume we have a set of possible events, each of which we assume occurs some number of times. Thus, if there are $N$ distinct possible events $(x_1, x_2, \ldots , x_N),$ no two of which can occur simultaneously, and the events occur with frequencies $(n_1, n_2, \ldots , n_N)$, we say that the probability of event $x_i$ is given by $$P(x_i) = \frac{n_i}{\sum_{j=1}^{N}{n_j}}$$ This definition has the nice property that $$\sum_{i=1}^{N}P(x_i) = 1$$ \item An {\em observer relative} version of probability: In this version, we take a statement of {\em probability} to be an assertion about the belief that a specific observer has of the occurrence of a specific event. Note that in this version of {\em probability}, it is possible that two different observers may assign different probabilities to the same event. Furthermore, the {\em probability} of an event, for me, is likely to change as I learn more about the event, or the context of the event. \pagedone \item In some (possibly many) cases, we may be able to find a reasonable correspondence between these two views of probability. In particular, we may sometimes be able to understand the {\em observer relative} version of the probability of an event to be an approximation to the {\em frequentist} version, and to view new knowledge as providing us a better estimate of the relative frequencies. \end{enumerate} \end{itemize} \pagedone %Slide 4 \begin{itemize} \item I won't go through much, but some probability basics, where $a$ and $b$ are events: \newline $ P(not\ a) = 1 - P(a).$\newline $ P(a\ or\ b) = P(a) + P(b) - P(a\ \mathrm{and}\ b).$ \newline We will often denote $P(a\ \mathrm{and}\ b)$ by $P(a, b)$. If $P(a, b) = 0$, we say $a$ and $b$ are mutually exclusive.\newline \item Conditional probability: \newline\newline $ P(a \vert b) $ is the probability of $a$, given that we know $b$. The joint probability of both $a$ and $b$ is given by: $$P(a, b) = P(a \vert b) P(b).$$ Since $P(a, b) = P(b, a)$, we have Bayes' Theorem: $$P(a \vert b)P(b) = P(b \vert a) P(a),$$ or $$P(a \vert b) = \frac{P(b \vert a) P(a)}{P(b)}.$$ \end{itemize} \pagedone %Slide 5 \begin{itemize} \item If two events $a$ and $b$ are such that $$P(a \vert b) = P(a),$$ we say that the events $a$ and $b$ are {\em independent}. Note that from Bayes' Theorem, we will also have that $$P(b \vert a) = P(b),$$ and furthermore, $$P(a, b) = P(a \vert b)P(b) = P(a)P(b).$$ This last equation is often taken as the definition of {\em independence}. \end{itemize} \pagedone %Slide 5 \begin{itemize} \item A quick example: \newline Suppose that you are asked by the government to help them understand the results of a ``terrorist screening system'' they are developing. They have been told that the system is 99.9\% accurate. What is the probability that when the system identifies a potential ``terrorist'' that they have actually found one? %% (Hint: We don't yet have enough information \ldots) \pagedone \item You do some research, and find out that independent estimates put the number of actual ``terrorists'' in the US at around 250. The creators of the system assert that the system is 99.9\% accurate. You push the question, and find that they say that one tenth of one percent of the time, the test falsely clears someone who is a ``terrorist'', and one tenth of one percent of the time the system falsely reports someone to be a ``terrorist'' when they are not. If the system identifies someone as a ``terrorist,'' how seriously should the government take the identification? Given this much information, what can you calculate as the probability the individual is a ``terrorist''? \pagedone In general, there are four possible situations for an individual being identified: \begin{enumerate} \item Test positive (Tp), and are a ``terrorist'' (T). \item Test negative (Tn), and are not a ``terrorist'' (NT). \item Test positive (Tp), and are not a ``terrorist'' (NT). \item Test negative (Tn), and are a ``terrorist'' (T). \end{enumerate} \pagedone We would like to calculate the probability someone is a ``terrorist'' (T) given, that they have been identified as such by the system (Tp): $$P(T \vert Tp).$$ We can do this using Bayes' Theorem. We can calculate: \begin{eqnarray*} P(T \vert Tp) & = & \frac{P(Tp \vert T) * P(T)}{P(Tp)}. \end{eqnarray*} We need to figure out the three items on the right side of the equation. We can do this by using the information given. \pagedone Suppose the screening test was done on 250,000,000 people in the US. Out of these $2.5 * 10^8$ people, we expect there to be 250 people who are ``terrorists'', and 249,999,750 people who are not. According to the creators of the system, we would expect the test results to be: \begin{itemize} \item Test positive (Tp), and are ``terrorists'' (T): 250 people. \item Test negative (Tn), and are not ``terrorists'' (NT): $$ 0.999 * 249,999,750 = 249,749,750\ \mathrm{people}.$$ \item Test positive (Tp), and are not ``terrorists'' (NT): $$ 0.001 * 249,999,750 = 250,000\ \mathrm{people}.$$ \item Test negative (Tn), and are ``terrorists'' (T): 0 people. \end{itemize} \pagedone Now let's put the the pieces together: \begin{eqnarray*} P(T) & = & \frac{250}{250,000,000}\\ \\ & = & 10^{-6}\\ \\ P(Tp) & = & \frac{250 + 250,000}{250,000,000}\\ \\ & = & \frac{250,250}{250,000,000}\\ \\ & = & 0.001001\\ \\ P(Tp \vert T) & = & 0.999 \end{eqnarray*} \pagedone Thus, our calculated probability that someone identified as a ``terrorist'' actually is one: \begin{eqnarray*} P(T \vert Tp) & = & \frac{P(Tp \vert T) * P(T)}{P(Tp)}\\ \\ & = & \frac{0.999 * 10^{-6}}{0.001001}\\ \\ & = & \frac{9.99 * 10^{-7}}{1.001 * 10^{-3}}\\ \\ & = & 9.98002 * 10^{-4}\\ \\ & < & 10^{-3} = .001 \end{eqnarray*} In other words, an individual identified by the system as a ``terrorist'', with a test that is promised to be 99.9\% correct, has less that one chance in 1000 of actually being one! Another way of saying it is that for every one ``terrorist'' that is actually identified, 1000 innocent people are incorrectly identified as being one. \pagedone \item There are a variety of questions we could ask now, such as, how accurate would the system have to be for there to be a greater than 50\% probability that someone identified as a ``terrorist'' actually is one? For this, we need fewer false positives than true positives. Thus, in the example, we would need fewer than 250 false positives out of the 249,999,750 people who are not. In other words, the proportion of those who are not ``terrorists'' for whom the system would have to be correct would need to be greater than: $$\frac{249,999,500}{249,999,750} = 99.9999\% !!$$ \pagedone \item Another question we could ask is, ``How prevalent would ``terrorists'' have to be in order for a 99.9\% accurate test (0.1\% false positive and 0.1\% false negative) to give a greater than 50\% probability of actually being a ``terrorist'' when identified as one?'' Again, we need fewer false positives than true positives. We would therefore need the actual occurrence to be greater than 1 in 1000 (each false positive would be matched by at least one true positive, on average) -- in other words, there would have to be about 250,000 ``terrorists'' in the US! \pagedone \item Another example: consider another situation, with a test that is not so accurate. Suppose the test were 80\% accurate (20\% false positive and 20\% false negative). Suppose that we are testing for a condition expected to affect 1 person in 100. What would be the probability that a person testing positive actually has the condition? We can do the same sort of calculations. \pagedone Let's use 1000 people this time. Out of this sample, we would expect 10 to have the condition. \begin{itemize} \item Test positive (Tp), and have the condition (Ha): $$ 0.80 * 10 = 8\ \mathrm{people}.$$ \item Test negative (Tn), and don't have the condition (Na): $$ 0.80 * 990 = 792\ \mathrm{people}.$$ \item Test positive (Tp), and don't have the condition (Na): $$ 0.20 * 990 = 198\ \mathrm{people}.$$ \item Test negative (Tn), and have the condition (Ha): $$ 0.20 * 10 = 2\ \mathrm{people}.$$ \end{itemize} \pagedone Now let's put the the pieces together: \begin{eqnarray*} P(Ha) & = & \frac{1}{100}\\ \\ & = & 10^{-2}\\ \\ P(Tp) & = & \frac{8 + 198}{10^3}\\ \\ & = & \frac{206}{10^3}\\ \\ & = & 0.206\\ \\ P(Tp \vert Ha) & = & 0.80 \end{eqnarray*} \pagedone Thus, our calculated probability that a person testing positive actually has the condition is: \begin{eqnarray*} P(Ha \vert Tp) & = & \frac{P(Tp \vert Ha) * P(Ha)}{P(Tp)}\\ \\ & = & \frac{0.80 * 10^{-2}}{0.206}\\ \\ & = & \frac{8 * 10^{-3}}{2.06 * 10^{-1}}\\ \\ & = & 3.883495 * 10^{-2}\\ \\ & < & .04 \end{eqnarray*} In other words, one who has tested {\em positive}, with a test that is 80\% correct, has less that one chance in 25 of actually having this condition. (Imagine for a moment, for example, that this is a drug test being used on employees of some corporation \ldots) \pagedone \item We could ask the same kinds of questions we asked before: \begin{enumerate} \item How accurate would the test have to be to get a better than 50\% chance of actually having the condition when testing positive? (99\%) \item For an 80\% accurate test, how frequent would the condition have to be to get a better than 50\% chance? (1 in 5) \end{enumerate} \pagedone \item Some questions: \begin{enumerate} \item Are these examples realistic? If not, why not? \item What sorts of things could we do to improve our results? \item Would it help to repeat the test? For example, if the probability of a false positive is 1 in 100, would that mean that the probability of two false positives on the same person would be 1 in 10,000 ($\frac{1}{100} * \frac{1}{100}$)? If not, why not? \item In the case of a medical condition such as a genetic anomaly, it is likely that the test would not be applied randomly, but would only be ordered if there were other symptoms suggesting the anomaly. How would this affect the results? \end{enumerate} \end{itemize} \pagedone \end{document}