But to paraphrase Mark Twain, I believe the rumors of Yockey’s irrelevance may have been greatly exaggerated. The second edition of his greatest work, Information Theory and Molecular Biology is not yet available for shipment – so who can say what he has or has not incorporated?
I’ve been following Jurgen Schmidhuber as well as Yockey, Schneider, Chaitin, Tegmark, Penrose, Rocha and many others. But again I see a huge fork in the direction being taken by those working in artificial intelligence and those working on information theory in molecular biology.
In the biological research, at the level of the molecular machine, the issue is one of communications, semiotics. Within that research, it forks again between those like Schneider whose research with NIH is oriented to the medical implications and evolution while the likes of Rocha, Wolfram and Yockey are examining how it may have emerged. However, from what I have read concerning complex systems, those investigators are interested in both algorithm and communications.
I am not aware of any advancement in unified theoretical constructs which would change the Shannon model for communications like Einstein's special and general relativity changed Newton's theory of gravity. Considering the high profile of Schneider's work for NIH, and the fact that he has an exhaustive website - I would have expected him to make note of such things.
Of all these people listed, the only ones that can reasonably be considered theoretical experts in information theory are Schmidhuber and Chaitin. These two guys are among the half dozen that actually author the theory and who I know for a fact understand it as well as anyone else does. A couple of the other guys are physicists, but have no special understanding of information theory (e.g. Tegmark and Penrose). Most of these folks have an undergrad CS level of understanding by my reckoning.
You are correct to a certain extent about a "fork" in the theory, though it is more pragmatic than substantive. Information theory was born out of electrical engineering, which has a very limited view and use for those maths. Dealing with stuff like algorithmic predictive limits really do not have much utility for them. But as information theory matured and was theoretically generalized, it's scope has increased immensely over time. The EEs have kept their own Shannon-esque perspective, but it isn't a separate field. It is actually a narrow expression of a much broader and more general mathematical field that grew out from the original Shannon perspective. You get the Shannon version by taking the modern general theory and reducing the degrees of freedom, making certain arbitrary assumptions axiomatic.
But when you do this, you have to understand the assumptions that are taken as axiomatic when you apply it. If you develop a model that tacitly rejects an assumption of the Shannon model, you have to use the general theory. The general theory that has developed are incredibly broad in scope, completely consuming fields like computation theory and transaction theory. Many mathematicians in the field are specialists in narrow sub-regions -- I would describe Chaitin this way, for example. The number of mathematicians who truly grok the nature and scope of the general theory are relatively small subset of these folks, and I would put people like Schmidhuber in that category. Most of the really interesting work in information theory is in regards to its unification with computation and decision theory IMO, and that is where a lot of the brain power is being expended. Most of the questions you are talking about are really well-framed narrow sub-problems of the general theory.
The ID theory needs to be worked at the level of the people at IDSIA in Switzerland (Schmidhuber, Hutter, and Legg), or who at least have the level of understanding of Merhav and his crew. It is a very rich field, but poor of real experts. At the very least, I would appreciate it if discussions were cited from Li and Vitanyi as a baseline, which is the de facto bible of the field. I will occasionally make assertions that are not citable, but I can always make an argument from citable sources when I do. I don't expect most people to be current on any of this stuff -- it is the very definition of esoterica -- but I do expect any cited "experts" to be reasonably current.