Re: Ontology and Physics
Rob,
Thanks for the pointers to the articles by Chaitin.
In particular, the lecture transcription is a very
readable summary of a lot of ideas. At the end of this
note, I selected a few excerpts I'd like to comment on
(but I recommend the whole article).
1. One significant idea is that axioms are essentially
a way of compressing a lot of data into a small
number of general principles.
2. That gives a good way of viewing learning: it's
a method of data compression that enables people
to encode a broader range of knowledge than would
be possible by rote memory of isolated facts.
3. But the various negative results, such as Goedel's
incompleteness theorem, Turing's halting problem,
and Chaitin's information-theoretic incompleteness,
demonstrate that not all facts can be compressed.
4. However, the success of mathematics in science shows
that a lot of important facts can be compressed.
But trying to compress *everything* is impossible.
This is just one more bit of theoretical justification
for the point I've been trying to make: the hope of
getting an ontology with detailed axioms for everything
is wishful thinking. But it is possible to have a
collection of low-level microtheories with detailed
axioms for many important applications.
In other words, the upper ontology should *not* have
any axioms other than basic definitions, which just
say how the types are related to one another. That
would make it more of a taxonomy or terminology than
what people have been calling an ontology.
All the practical experience shows that this approach
works: taxonomies and terminologies are very important
for communication, for database systems, and for information
retrieval. For deduction, the applications that have been
successful are always very narrow, context-dependent cases.
Doug Lenat and the Cyc group have come to this realization
through 22 years of hard work. They had tried very hard
to develop a top-down theory of everything, but they were
forced to break it into microtheories. And Lenat has said
that the upper-levels of the ontology are much less important
than the context-dependent lower levels. There is no evidence
that anybody else has a clue about how to do any better.
John Sowa
________________________________________________________
http://www.cs.auckland.ac.nz/CDMTCS/chaitin/cmu.html
And the idea that I came up with --- and Kolmogorov came up with at the
same time independently --- is the idea that something is random if it
can't be compressed into a shorter description, if essentially you just
have to write it out as it is. In other words, there's no concise theory
that produces it. For example, a set of physical data would be random if
the only way to publish it is as is in a table, but if there's a theory
you're compressing a lot of observations into a small number of physical
principles or laws. And the more the compression, the better the theory:
in accord with Occam's razor, the best theory is the simplest theory. I
would say that a theory is a program --- also Ray Solomonoff did some
thinking along these lines for doing induction --- he didn't go on to
define randomness, but he should have! If you think of a theory as a
program that calculates the observations, the smaller the program is
relative to the output, which is the observations, the better the theory is.
By the way, this is also what axioms do. I would say that axioms are the
same idea. You have a lot of theorems or mathematical truth and you're
compressing them into a set of axioms. Now why is this good? Because
then there's less risk. Because the axioms are hypotheses that you have
to make and every time you make a hypothesis you have to take it on
faith and there's risk --- you're not proving it from anything, you're
taking it as a given, and the less you assume, the safer it is. So the
fewer axioms you have, the better off you are. So the more compression
of a lot of theorems, of a body of theory, into a small set of axioms,
the better off you are, I would say, in mathematics as well as physics....
Now what is the reaction of the world to this work?! Well, I think it's
fair to say that the only people who like what I'm doing are physicists!
This is not surprising, because the idea came in a way from physics. I
have a foreign idea called randomness that I'm bringing into logic, and
logicians feel very uncomfortable with it. You know, the notion of
program size, program-size complexity is like the idea of entropy in
thermodynamics. So it turns out that physicists find this nice because
they view it as ideas from their field invading logic. But logicians
don't like this very much.
I think there may be political reasons, but I think there are also
legitimate conceptual reasons, because these are ideas that are so
foreign, the idea of randomness or of things that are true by accident
is so foreign to a mathematician or a logician, that it's a nightmare!
This is their worst nightmare come true! I think they would prefer not
to think about it.
On the other hand, physicists think this is delightful! Because they
remember well the crisis that they went through in the 1920's about
randomness at the foundations of physics, and they say, it's not just
us, we're not the only people who have randomness, pure math has it too,
they're not any better than we are! ...
And unexpectedly there are physicists who are interested in my notion of
program-size complexity; they view it as another take on thermodynamical
entropy. There's some work by real physicists on Maxwell's demon using
my ideas; I mention this for those of you who have some physics background.
But I must say that philosophers have not picked up the ball. I think
logicians hate my work, they detest it! And I'm like pornography, I'm
sort of an unmentionable subject in the world of logic, because my
results are so disgusting! ...