How did life
emerge from non-life? It did so through a long succession of processes and
events in which more complex structures evolved from simpler ones. Beginning
with this post, I shall take you through a fascinating journey of easy comprehension,
explaining how complexity science answers such questions. This and the next few
posts will introduce some of the jargon and basic concepts in modern complexity
science.
The second law
of thermodynamics for open systems is the primary organizing principle
for all natural phenomena (cf. Part 6). The relentless
expansion and cooling of our universe has been creating gradients of various
types, which tend to get annulled as the blind forces of Nature take the local systems
towards old or new equilibria. New patterns and structures get created when new
equilibrium structures arise.
Take any
living entity; say the human body, or even a single-celled organism. The amount
of information needed for describing the structure of a single biological cell
is far more than the information needed to describe, say, an atom or a molecule.
The technical term one introduces here is 'complexity'. We say that the
biological cell has a much higher DEGREE OF COMPLEXITY than an atom or a
molecule.
Let us
tentatively define the degree of complexity of any object or system as the
amount of information needed for describing its structure and function.
Since the
degree of complexity has been defined in terms of 'amount of information', we
should be clear about the formal meaning of 'information'. The word 'bit' is commonplace
in this IT age. It was introduced by Claude Shannon in 1948, and is the
short form for 'binary digit'. A bit has two states: 0 or 1. Shannon took the
bit as the unit of information.
One bit is the quantity of information needed (it is the 'missing' or 'not-yet-available'
information) for deciding between two equally likely possibilities (for
example, whether the toss of a coin will be 'heads' or 'tails'). And the
information content of a system is the minimum number of bits needed for a
description of the system.
The term ‘missing
information’ is assigned a numerical measure by defining it as the uncertainty
in the outcome of an experiment yet to be carried out. The uncertainty may be
high either because only one of a large number (Ns) of outcomes is possible, or, what is the same thing,
the probability of a particular outcome is inherently low.
Suppose we
have a special coin with heads on both
sides. What is the probability that the result of a spin of the coin will be
'heads'? The answer is 100% or 1; i.e., certainty. Thus the carrying out of
this experiment gives us zero information. We were certain of the outcome, and
we got that outcome; there was no missing information.
Next, we
repeat the experiment with a normal, unbiased, two-sided coin. There are two
possible outcomes now (Ns = 2). In this case the actual
outcome gives us information which we did not have before the experiment was
carried out (i.e., we get the missing information).
Suppose we
toss two coins instead of one. Now there are four possible outcomes (Ns
= 4). Therefore, any particular experiment here gives us even more information
than in the two situations above. Thus: Low
probability means high missing information, and vice versa.
To assign a
numerical measure to information, we would like the following criteria to be
met:
1. Since
(missing) information (I) depends on Ns, the definition of information should be such that, if
we are dealing with a combination of two or more systems, Ns for the composite system should be correctly
accounted for in the definition of information. For example, for the case of
two dice tossed together or successively, Ns should be 6 x 6
= 36, and not 6 + 6 = 12.
2. Information
I for a composite or 'multivariate'
system should be a sum (and not, say,
a multiplication) of the information for the components comprising the system.
The following
relationship meets these two requirements:
Ns ~ baseI.
Let us see
how. Suppose system X has Nx states and the outcome of
an experiment gives information Ix.
Let Ny and Iy be the corresponding
quantities for system Y. For the
composite system comprising of X and Y, we get NxNy
~ baseIx baseIy. Since Ns = NxNy,
we can write
Ns ~ base(Ix
+ Iy).
Taking
logarithms of both sides, and writing Ix + Iy =
I, we get
logNs
~ I log(base)
or
I ~
logNs / log(base).
What kind of
logarithm we take (base = 10, 2, or e), and what proportionality
constant we select, is a matter of context. All such choices differ only by
some scale factor, and units. The important thing is that this approach for the
definition of information has given us a correct accounting of the number of
states, 'bins', or classes (i.e. by a multiplication (NxNy) of the
individual states), and a correct accounting of the individual measures of
information (i.e. by addition).
All the cases
considered above are equiprobability
cases: When the die is thrown, the probability P1 that
the face with ‘1’ will show up is 1/6, as are the probabilities P2, P3, .. P6 that
'2', '3', .. '6' will show up. For such examples, the constant probability P is simply the reciprocal of the number
of possible outcomes, classes, or bins; i.e., Ns:
P =
1 / Ns.
Substituting
this in the above relation, we get
I ~
log (1/P) / log (base).
Introducing a
suitable proportionality constant c,
we can write
I = c
log (1/P).
This is close
to the SHANNON FORMULA for missing information. Even more pertinently, this
equation is similar to the famous Boltzmann equation for entropy:
S = k
log W.
Entropy has
the same meaning as missing information, or uncertainty. More on this next
time.
No comments:
Post a Comment