Try this: A Gentle Introduction to the Universal Algorithmic Agent AIXI. I found it after a similar discussion with tortoise a while back, and was able to get the general idea (at least I think so) without having the ideal prerequisites. Algorithmic information theory is a fascinating field.
Even better, the reference bibliographies in his papers are nearly ideal reading lists for the foundational mathematics, including some key topics that he does not write about. The idea of predictive error complexity actually comes from some seminal work by Feder, Merhav, and Gutman in the mid-90s.
Thanks!