Free Republic
Browse · Search
News/Activism
Topics · Post Article

To: Alamo-Girl; All
This sort of entropy is clearly different. Physical units do not pertain to it, and (except in the case of digital information) an arbitrary convention must be imposed before it can be quantified.

I opened up my copy of Li & Vitanyi (the de facto bible of modern information theory) to look up a reference for a paper I'm writing, and discovered that the entire last chapter is dedicated to the mathematical unification of the various notions of entropy in physics, chemistry, etc with the definition used in broader algorithmic information theory, and also maps other concepts in science into their formal AIT analogs. It is a little out of date (1997) and there is some newer computational theory work it does not include, but it pretty much settles the question as to how "entropy" as used in science and engineering maps into the broader mathematical definitions.

It did not even occur to me to see if Li & Vitanyi addressed this topic; I assumed the book was pure math and I hardly ever open it any more. Anyway, if you want a properly rigorous overview of this topic, you'll find it in Chapter 8 of that book.

564 posted on 01/09/2005 4:39:09 PM PST by tortoise (All these moments lost in time, like tears in the rain.)
[ Post Reply | Private Reply | To 470 | View Replies ]


To: tortoise; betty boop; Doctor Stochastic; StJacques; cornelis; marron; PatrickHenry; Matchett-PI
Thank you so much for the heads up on Li & Vitanyi's 8th chapter! I just put the Kolmogorov book on my wish list on Amazon (my birthday is coming up.)

I also wanted to bring to y'alls attention Adami's article on Information Theory and Molecular Biology. He does not approach the subject in a way substantively different from Schneider, but he associates the measure of Shannon Information to Physical Complexity and makes the distinction between Shannon entropy, thermodynamic entropy and Shannon information very clear.

574 posted on 01/09/2005 9:01:47 PM PST by Alamo-Girl
[ Post Reply | Private Reply | To 564 | View Replies ]

Free Republic
Browse · Search
News/Activism
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson