What follows is an extremely abbreviated look at some of the important ideas of the general areas of automata theory, computability, and formal languages. In various respects, this can be thought of as the elementary foundations of much of computer science. The area also includes a wide variety of tools, and general categories of tools ...

Symbols, strings and languagesSymbols, strings and languages Top

The classical theory of computation traditionally deals with processing an input string of symbols into an output string of symbols. Note that in the special case where the set of possible output strings is just {`yes', `no'}, (often abbreviated {T, F} or {1, 0}), then we can think of the string processing as string (pattern) recognition.
We should start with a few definitions. The first step is to avoid defining the term `symbol' - this leaves an open slot to connect the abstract theory to the world ...
We define:
1. An alphabet is a finite set of symbols.
2. A string over an alphabet A is a finite ordered sequence of symbols from A. Note that repetitions are allowed. The length of a string is the number of symbols in the string, with repetitions counted. (e.g., |aabbcc | = 6)
3. The empty string, denoted by e, is the (unique) string of length zero. Note that the empty string e is not the same as the empty set Ć.
4. If S and T are sets of strings, then ST = {xy | x Î S and y Î T}
5. Given an alphabet A, we define
  
  A⁰
  
  =
  
  {e}
  Aⁿ⁺¹
  
  =
  
  AAⁿ
  A^*
  
  =
  
  Ľ
  Č
  n=0
  Aⁿ
6. A language L over an alphabet A is a subset of A^*. That is, L Ě A^*.

We can define the natural numbers, \mathbbN, as follows:
We let

0

=

Ć
1

=

{Ć}
2

=

{Ć, {Ć}}
and

in

general
n+1

=

{0, 1, 2, ź, n}.
Then

\mathbbN

=

{0, 1, 2, ź}.

Sizes of sets and countability:
1. Given two sets S and T, we say that they are the same size (|S | = | T |) if there is a one-to-one onto function f: S Ž T.
2. We write |S | Ł |T| if there is a one-to-one (not necessarily onto) function f: S Ž T.
3. We write |S | < |T | if there is a one-to-one function f: S Ž T, but there does not exist any such onto function.
4. We call a set S
  1. Finite if |S | < |\mathbbN|
  2. Countable if |S | Ł |\mathbbN|
  3. Countably infinite if |S | = |\mathbbN|
  4. Uncountable if |\mathbbN| < |S |.
5. Some examples:
  1. The set of integers \mathbbZ = {0, 1, -1, 2, -2, ź} is countable.
  2. The set of rational numbers \mathbbQ = {p/q | p,q Î \mathbbZ, q š 0} is countable.
  3. If S is countable, then so is SxS, the cartesian product of S with itself, and so is the general cartesian product Sⁿ for any n < Ľ.
  4. For any nonempty alphabet A, A^* is countably infinite.
  Exercise: Verify each of these statements.
6. Recall that the power set of a set S is the set of all subsets of S:
  
  P(S) = {T | T Ě S}.
  
  We then have the fact that for any set S,
  
  |S | < |P(S) |.
  
  Pf: First, it is easy to see that
  
  |S | Ł |P(S)|
  
  since there is the one-to-one function f : S Ž P(S) given by f(s) = {s} for s Î S.
  On the other hand, no function f : S Ž P(S) can be onto. To show this, we need to exhibit an element of P(S) that is not in the image of f. For any given f, such an element (which must be a subset of S) is
  
  R_f = {x Î S | x Ď f(x)}.
  
  Now suppose, for contradiction, that there is some s Î S with f(s) = R_f. There are then two possibilities: either s Î f(s) = R_f or s Ď f(s) = R_f. Each of these leads to a contradiction:
  If s Î f(s) = R_f, then by the definition of R_f, s Ď f(s). This is a contradiction.
  If s Ď f(s) = R_f, then by the definition of R_f, s Î R_f = f(s). Again, a contradiction.
  Since each case leads to a contradiction, no such s can exist, and hence f is not onto. QED

From this, we can conclude for any countably infinite set S, P(S) is uncountable. Thus, for example, P(\mathbbN) is uncountable. It is not hard to see that the set of real numbers, \mathbbR, is the same size as P(\mathbbN), and is therefore uncountable.
Exercise: Show this. (Hint: show that \mathbbR is the same size as (0,1) = {x Î \mathbbR | 0 < x < 1}, and then use the binary representation of real numbers to show that |P(\mathbbN) | = |(0,1) |).

We can also derive a fundamental (non)computability fact:
There are languages that cannot be recognized by any computation. In other words, there are languages for which there cannot exist any computer algorithm to determine whether an arbitrary string is in the language or not.
To see this, we will take as given that any computer algorithm can be expressed as a computer program, and hence, in particular, can be expressed as a finite string of ascii characters. Therefore, since ASCII^* is countably infinite, there are at most countably many computer algorithms/programs. On the other hand, since a language is any arbitrary subset of A^* for some alphabet A, there are uncountably many languages, since there are uncountably many subsets.

No royal roadNo royal road <-

There is no royal road to logic, and really valuable ideas can only be had at the price of close attention. But I know that in the matter of ideas the public prefer the cheap and nasty; and in my next paper I am going to return to the easily intelligible, and not wander from it again.
- C.S. Pierce in How to Make Our Ideas Clear, 1878

Finite automataFinite automata Top

This will be a quick tour through some of the basics of the abstract theory of computation. We will start with a relatively straightforward class of machines and languages - deterministic finite automata and regular languages.
In this context when we talk about a machine, we mean an abstract rather than a physical machine, and in general will think in terms of a computer algorithm that could be implemented in a physical machine. Our descriptions of machines will be abstract, but are intended to be sufficiently precise that an implementation could be developed.

A deterministic finite automaton (DFA) M = (S, A, s₀, d, F) consists of the following:

S, a finite set of states,

A, an alphabet,

s₀ Î S, the start state,

d: S x A Ž S, the transition function, and

F Ě S, the set of final (or accepting) states of the machine.

We think in terms of feeding strings from A^* into the machine. To do this, we extend the transition function to a function

^

d

: S x A^* Ž S

by

^

d

(s, e)

=

s,

^

d

(s, xa)

=

^

d

(d(s, a), x).

We can then define the language of the machine by

L(M) = {x Î A^* |
^

d

(s₀, x) Î F}.

In other words, L(M) is the set of all strings in A^* that move the machine via its transition function from the start state s₀ into one of the final (accepting) states.
We can think of the machine M as a recognizer for L(M), or as a string processing function

f_M : A^* Ž {1, 0}

where f_M(x) = 1 exactly when x Î L(M).

There are several generalizations of DFAs that are useful in various contexts. A first important generalization is to add a nondeterministic capability to the machines. A nondeterministic finite automaton (NFA) M = (S, A, s₀, d, F) is the same as a DFA except for the transition function:

S, a finite set of states,

A, an alphabet,

s₀ Î S, the start state,

d: S x A Ž P(S), the transition function,

F Ě S, the set of final (or accepting) states of the machine.

For a given input symbol, the transition function can take us to any one of a set of states.
We extend the transition function to [^(d)] : S x A^* Ž P(S) in much the same way:

^

d

(s, e)

=

s,

^

d

(s, xa)

=

Č
r Î d(s, a)

^

d

(r, x).

We define the language of the machine by

L(M) = {x Î A^* |
^

d

(s₀, x) ÇF š Ć}.

A useful fact is that DFAs and NFAs define the same class of languages. In particular, given a language L, we have that L = L(M) for some DFA M if and only if L = L(M˘) for some NFA M˘.
Exercise: Prove this fact.
In doing the proof, you will notice that if L = L(M) = L(M˘) for some DFA M and NFA M˘, and M˘ has n states, then M might need to have as many as 2ⁿ states. In general, NFAs are relatively easy to write down, but DFAs can be directly implemented.

Another useful generalization is to allow the machine to change states without any input (often called e-moves). An NFA with e-moves would be defined similarly to an NFA, but with transition function

d: S x (A Č{e}) Ž P(S).

Exercise: What would an appropriate extended transition function [^(d)] and language L(M) be for an NFA with e-moves?
Exercise: Show that the class of languages defined by NFAs with e-moves is the same as that defined by DFAs and NFAs.
Here is a simple example. By convention, the states in F are double circled. Labeled arrows indicate transitions. Exercise: what is the language of this machine?

Figure

Regular expressions and languagesRegular expressions and languages Top

In the preceding section we defined a class of machines (Finite Automata) that can be used to recognize members of a particular class of languages. It would be nice to have a concise way to describe such a language, and furthermore to have a convenient way to generate strings in such a language (as opposed to having to feed candidate strings into a machine, and hoping they are recognized as being in the language ...).
Fortunately, there is a nice way to do this. The class of languages defined by Finite Automata are called Regular Languages (or Regular Sets of strings). These languages are described by Regular Expressions. We define these as follows. We will use lower case letters for regular expressions, and upper case for regular sets.

Definition: Given an alphabet A, the following are regular expressions / regular sets over A:

Introduction to theory of computation

Tom Carter http://cogs.csustan.edu/~tom/SFI-CSSS Complex Systems Summer School

June, 2002

Tom Carter

http://cogs.csustan.edu/~tom/SFI-CSSS Complex Systems Summer School