Free Republic
Browse · Search
News/Activism
Topics · Post Article

To: Alamo-Girl
This particular difference of opinion I find very relevant and would like to know more of your reasoning for not leaning to Shannon.

Shannon information theory had some theoretical deficiencies, such that it is possible to construct hypothetical situations where the results from a pure Shannon perspective gives results that don't seem correct upon casual inspection. Shannon did much of his work around coding theory, which is really a narrow "static" case of what we now file under Information Theory. Kolmogorov took more of a computational and system theoretic perspective, and attempted to resolve the apparent problems and inconsistencies that fell out of Shannon information theory.

Today we generally consider the Kolmogorov version and its derivatives (e.g. Solomonoff, Chaitin, and others) of information theory to be the correct one, and view Shannon's version in a manner that is similar to Newtonian mechanics -- a special case of the true model but still useful for many things because it is well-studied and much simpler to apply. Nonetheless, for many things you will get incorrect results if you use the "simple" model when you should be using the "correct" model. Most people who are not planning on being experts in information theory learn the Shannon model, though they are generally very casually familiar with Kolmogorov. In the same fashion, most people who learn physics learn Newtonian mechanics and don't really go into Einsteinian mechanics unless they plan on being specialists in the field. It is the old 80-20 rule; the simple models are sufficient for 80% of the cases and require 20% of the effort of actually learning the entire field.

I think there are a number of aspects of biological systems that make it clear that Shannon is inappropriate and that Kolmogorov must be used. First, there is the fact that biological systems are complex dynamic systems, which is Kolmogorov's domain in general. Second, biological systems, genomes, and similar deal with a lot of non-sequential pattern transforms, which Shannon only partially handles and badly when it does. These two facts alone make Shannon almost useless for a really good analysis. Kolmogorov offers many solutions to these issues that you won't have at your disposal if you only have a purely Shannon model to use.

In fact, one could almost say that biological systems of the type we are talking about are the pathological case scenario from the standpoint of trying to use Shannon information theory.

635 posted on 06/29/2003 2:40:44 PM PDT by tortoise (Would you like to buy some rubber nipples?)
[ Post Reply | Private Reply | To 629 | View Replies ]


To: tortoise
Thank you for sharing your views on Shannon! That's exactly what I wanted to know.
644 posted on 06/29/2003 8:54:02 PM PDT by Alamo-Girl
[ Post Reply | Private Reply | To 635 | View Replies ]

Free Republic
Browse · Search
News/Activism
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson