ONT Re: What Is Information That A Sign May Bear It?
o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o
WIS Discussion. Note 4
o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o
HT = Hugh Trenchard
JA = Jon Awbrey
Re: WIS 2. http://suo.ieee.org/ontology/msg05296.html
In: WIS. http://suo.ieee.org/ontology/thrd19.html#04317
Hugh,
Old bits indented and tagged by author, new bits unindentional.
HT: These are some really interesting thoughts -- they look to me very well
thought out and articulately presented. As I have mentioned in previous
emails, I am a neophyte to the concept of information and am making some
efforts to understand the notion in its various subtleties.
And all the archeophytes I ever read were ever thus endeavoring.
HT: In order to understand better what you are presenting, I have
some questions/comments which follow -- wherever you see "[Hugh]".
I've put in "[Jon]" to show where your paragraphs resume.
JA: Speaking of recurring visitations, here is one that various among you
will have seen somewhere varying between one and three or seven times.
It is the simplest way I know of explaining information theory as one
finds it during the middle ages of its buzzy boom, circa 1920-1970 or
thereabouts. This picture of information is not quite general enough
to fully cover what Peirce had in mind at its conception, but it does
the job under many commonly occurring circumstances, and so it repays
careful consideration even in the more general context of logic taken
as a form of normative semiotics.
JA: What Is Information That A Sign May Bear It?
JA: Here is a trio of questions that I try to keep in mind:
1. How is a sign empowered to contain information?
2. What is the practical context of communication?
3. Why do we care about these bits of information?
JA: My way of addressing these questions is a bit like this:
JA: We are mostly concerned with our own lives,
but then a world obtrudes on our existence,
and so we find ourselves forced to take up
an interest in the realities of its nature.
JA: In pragmatic terms our initial pragma is a bit
like the verbal infinitive "to live", but then
it gets turned into the derivative substantial
forms of "nature", "reality", "the world", etc.
JA: Against this backdrop one finds oneself cast as
a protagonist on a "scene of uncertainty" (SOU).
I picture this as a juncture where I have a set
of n options that fan out before me. It may be
a question of "What is true?", or "What to do?",
or "What to hope?", where the last is a codebit
for "What regulative principle has any chance?",
but the main uncertainty is that I am called on
to make a choice and often do not have any clue
what is fit to pick. (By the way, this picture
of the human practical fix is credited to Kant.)
JA: Just to make up a discrete example let us suppose
that the cardinality of this choice is a finite n,
and just to make it fully concrete let us say n=5.
Here is the picture that I would have in mind for
such a situation:
o-------------------------------------------------o
| |
| ? ? ? ? ? |
| o o o o o |
| |
| o o o o o |
| |
| o o o o o |
| |
| o o o o o |
| |
| o o o o o |
| |
| ooooo |
| |
| @ n = 5 |
| |
o-------------------------------------------------o
Figure 1. Juncture of Degree 5
JA: This pictures a juncture, represented by "@",
where there are n options for the outcome of
a conduct, and I do not have a clue which it
must be. In a sense the degree of this node,
in this case n = 5, measures the uncertainty
that I have at this point.
JA: As best I can figure, this is the minimal sort of
setting in which a sign can make any sense at all.
A sign has significance for an agent, interpreter,
or observer because its actualization, its being
given or its being present, serves to reduce the
uncertainty of a decision that the agent has to
make, whether it concerns the actions that the
agent ought to take in order to achieve some
objective of interest, or whether it concerns
the predicates that the agent ought to treat
as being true of some object in the world.
JA: The way that signs come into this setting,
to make the scene, as one used to say, is
something that I could picture as follows:
o-------------------------------------------------o
| |
| k_1 = 3 k_2 = 2 |
| o-----o-----o o-----o |
| "A" "B" |
| o----o----o o----o |
| |
| o---o---o o---o |
| |
| o--o--o o--o |
| |
| o-o-o o-o |
| |
| ooooo |
| |
| @ n = 5 |
| |
o-------------------------------------------------o
Figure 2. Partition of Degrees 3 and 2
JA: This illustrates a situation of uncertainty
that has been augmented by a classification.
JA: In the particular pattern of classification that is shown here,
the first three outcomes fall under the sign "A", and the next
two outcomes fall under the sign "B". If the outcomes make up
a set of "things that might be true about an object", then the
signs could be read as nomens (terms) or notions (concepts) of
a relevant empirical, ontological, taxonomical, or theoretical
scheme, that is, as predicates and predictions of the outcomes.
If the outcomes make up a set of "things that might be good to
do in order to achieve an objective", then the signs could be
read as bits of advice or other sorts of indicators that tell
us what to do in the situation, relative to our active goals.
HT: I am wondering whether there are broad categories of outcomes that signs
may lead to in addition to leading us to some degree of certain knowledge
("things that might be true") or toward achieving an objective. You did
say that this is the "minimal setting in which signs can make sense", which
causes me to consider what else results from the receipt of a sign -- things
that come to mind are false beliefs (no truth arrived at about an object),
and signs deliberately designed by a sign-sender to mislead. I know you
are starting with basic scenarios, so there probably isn't any need
for me to complicate your formalization at this point, but I am
really just thinking aloud here.
Well, I can't really think of any new bits here,
so I stole a few lines from I. Kant that give
me the slack to incorporate Peirce's ideas,
but whether it's too exorbitant a frame
in the end I can't tell just yet.
At any rate, Kant's three questions went a bit like this:
1. What's true?
2. What's to do?
3. What's to hope?
I interpret these questions as marking out the
Descriptive, the Normative, and the Regulative
facets of the life of inquiry. Many questions
about the relationship of "Is" to "Ought" will
turn on the first two, and how one answers the
third question will say whether one has a hope
of resolving any of them. Within the frame of
pragmatic thinking, the Greek 'pragma' has too
many senses to pin down easily, but much turns
on the trope between "object" and "objective".
For my part, I started by pointing to the objective infinitive "to live",
and that does seem like a bare "contingency of doing aught else" (CODAE).
And as you note, there is nothing about the instrumentality of signs in
conveying information that says they all of them will always be perfect
or even positive instruments. And if I start with Kant's questions, it
doesn't say I'll end with his answers, not if it comes to life or death.
JA: This is the basic framework for talking about information and signs
in regard to communication, decision, and the uncertainties thereof.
JA: Just to unpack some of the many things that may be getting
glossed over in this little word "sign", it encompasses all
of the "data of the senses" (DOTS) that we take as informing
us about inner and outer worlds, plus all of the concepts and
terms that we use to argue, to communicate, to inquire, or even
to speculate about our ontologies for beings and our policies for
action in the world.
JA: Here is one of the places where it is tempting to try to
collapse the 3-adic sign relation into a 2-adic relation.
For if these DOTS are so closely identified with objects
that we can scarcely reckon how they might be discrepant,
then it will appear to us that one role of beings can be
eliminated from our picture of the world. In this event,
the only things that we are required to inform ourselves
about via the inspection of these DOTS are yet more DOTS,
whether past, or present, or prospective, just more DOTS.
This is the special form to which we frequently find the
idea of an information channel being reduced, namely, as
a "source" that has nothing better to tell us about than
its own conceivable conducts or its own potential issues.
HT: Your statement above reminds me of one definition of randomness:
a set of items occurs randomly if its description is identical
to the number of components; i.e. there is no way to reduce
its description into something smaller than itself. Thus,
I understand, there are no computer programs that truly
generate random numbers, because the program itself is
still smaller than the complete set of numbers.
Again, just thinking aloud ...
Yes, some people define it that way.
And there may be uncertainties that
are irreducible past certain points.
JA: As a matter of fact, at least in this discrete type of case, it would
be possible to use the degree of the node as a measure of uncertainty,
but it would operate as a multiplicative measure rather than the sort
of additive measure that we would normally prefer. To illustrate how
this would work out, let us consider an easier example, one where the
degree of the choice point is 4.
o-------------------------------------------------o
| |
| ? ? ? ? |
| o o o o |
| |
| o o o o |
| |
| o o o o |
| |
| o o o o |
| |
| o o o o |
| |
| oo oo |
| |
| @ n = 4 |
| |
o-------------------------------------------------o
Figure 3. Juncture of Degree 4
JA: Suppose that we contemplate making another decision after
the present issue has been decided, one that has a degree
of 2 in every case. The compound situation looks like so:
o-------------------------------------------------o
| |
| o o o o o o o o |
| \ / \ / \ / \ / |
| o o o o n_2 = 2 |
| |
| o o o o |
| |
| o o o o |
| |
| o o o o |
| |
| o o o o |
| |
| oo oo |
| |
| @ n_1 = 4 |
| |
o-------------------------------------------------o
Figure 4. Compound Junctures of Degrees 4 and 2
JA: This depicts the fact that the compound uncertainty, 8,
is the product of the two component uncertainties, 4 x 2.
To convert this to an additive measure, we simply take the
logarithms to a convenient base, say 2, and thus we arrive
at the not too astounding fact that the uncertainty of the
first choice is 2 bits, the uncertainty of the next choice
is 1 bit, and the compound uncertainty is 3 = 2 + 1 bits.
JA: In many ways, the provision of information, a process that
reduces uncertainty, is the inverse process to the kind of
uncertainty augmentation that occurs in compound decisions.
By way of illustrating this relationship, let us return to
our initial example.
HT: It looks to me your scenario assumes that every possible outcome at the first
stage results in a possible second set of choices. I am wondering if this is
inconsistent with "suppose we contemplate making another decision after the
present issue has been decided", which sounds to me means that in only one
of the first set of possible outcomes, a second choice becomes apparent.
If that is the case, would that not mean that the compounding can only
follow one of the lines upward -- it wouldn't matter which one, but
nevertheless only one, thus reducing the compounding to 6 from 8?
HT: I guess I'm asking if it is reasonable to assume that a second set of choices
implies that the uncertainty of making it applies at all possible outcomes at
the first stage, or do we have to factor in the potential that not all of the
first set of alternatives leads to a second set of choices. I am only relying
on a gut feeling here, as my powers of reasoning through a formal system are
pretty weak, so please forgive me if I'm completely out in left field.
Okay, you know that I was only trying to provide intuitive grounding for
measures of uncertainty and information. To do this full tilt, one might
set it within a game-theoretic framework, where the options of each player,
here something like the Intelligent Agent and the Observed Nature, have far
more general types of options at each turn, and where the measures would be
defined in terms of averages taken over appropriate sets of configurations,
and so on and on, into ever increasing orders of generalization, no doubt.
JA: A set of signs enters on a setup like this as a system
of middle terms, which I'm apt to regard as a "medium":
o-------------------------------------------------o
| |
| k_1 = 3 k_2 = 2 |
| o-----o-----o o-----o |
| "A" "B" |
| o----o----o o----o |
| |
| o---o---o o---o |
| |
| o--o--o o--o |
| |
| o-o-o o-o |
| |
| ooooo |
| |
| @ n = 5 |
| |
o-------------------------------------------------o
Figure 2. Partition of Degrees 3 and 2
JA: The "language" or "medium" here is the set of signs {"A", "B"}.
On the assumption that the initial outcomes are equally likely,
we associate a frequency distribution <k_1, k_2> = <3, 2>, and
a probability distribution <p_1, p_2> = <3/5, 2/5> = <0.6, 0.4>
with this language, and thus define a communication "channel".
JA: The most important thing here is really just to get a handle on
the "conditions for the possibility of signs making sense", but
once we have this much of a setup we find that we can begin to
construct some rough and ready bits of information-theoretic
furniture, like measures of uncertainty, channel capacity,
and the amount of information that can be associated with
the reception or the recognition of a single sign. Still,
before I get into all of this, I want to emphasize that,
even when these measures are too ad hoc and insufficient
to be of much use per se, the significance of the setup
that it takes to support them is not at all diminished.
JA: Consider the augmented situation of uncertainty that was depicted above.
What happens if we receive one or the other of the two signs "A" or "B"?
A. If we receive "A" our uncertainty is reduced from log 5 to log 3.
B. If we receive "B" our uncertainty is reduced from log 5 to log 2.
JA: It is from these characteristics that the "information capacity"
of a communication channel can be defined, specifically, as the
"average uncertainty reduction on receiving a sign" (AURORAS).
JA: In the present example we have:
| Channel Capacity
|
| = (1/n)(k_1·(log n - log k_1) + k_2·(log n - log k_2))
|
| = (k_1/n)(log n - log k_1) + (k_2/n)(log n - log k_2)
|
| = (-k_1/n)(log k_1 - log n) + (-k_2/n)(log k_2 - log n)
|
| = (-k_1/n)(log k_1/n) + (-k_2/n)(log k_2/n)
|
| = - (p_1 log p_1 + p_2 log p_2)
|
| = - (0.6 log 0.6 + 0.4 log 0.4)
|
| = 0.971
JA: In other words, the capacity of the channel is slightly under 1 bit.
This makes intuitive sense, as 3 versus 2 is a near-even split of 5,
and the measure of channel capacity or the "entropy" is supposed to
attain its maximum of 1 bit whenever a two-way partition is 50-50.
HT: Is the "channel" in this case a reference to the possible route the
information flow may take? Does a formal system like this imply that
one could take a set of language instructions, for example, and plot
their channel capacit(ies)? Is the measure of information ultimately
the channel capacity of communication? By extension, I can imagine
plotting a conversation on a moving graph with peaks and valleys
rather like an eeg readout, indicating the channel capacities
and therefore the information content of the conversation.
But that's getting a bit ahead of ourselves ...
Until the day when we actually find cannoli on Mars, it's best to keep
the word 'channel' in italics, and not take this complex of words too
literally, as a lot of science has been misdirected thereby. The way
that my present understanding was dinned into me, the word "channel"
refers to a probability density, and should not lead us to expect
a physical conduit of any too literal a kind.
Still, if we ask the question, "How do these media of middle terms arise?",
then we will be faced with the like of what Aristotle, Peirce, and company
asked about abduction, namely, "What are the rules that make a concept fit
to conceive in the first place?".
HT: Like I say, I am really at the point of trying to understand
what you are saying, so please be patient with my inquiries.
Patience is an all-around virtue ...
Jon Awbrey
o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o
http://www.cs.bsu.edu/homepages/mighty/history.html
o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o~~~~~~~~~o