[ThinkDifferent]

Could you recommend books or websites where I could learn about this?

Try this: A Gentle Introduction to the Universal Algorithmic Agent AIXI. I found it after a similar discussion with tortoise a while back, and was able to get the general idea (at least I think so) without having the ideal prerequisites. Algorithmic information theory is a fascinating field.

[tortoise]

Yeah, that is a good one, though I frequently see the question "Is there anything gentler than 'A Gentle Introduction...'". Hutter probably explains more of this area of mathematics in his various writings than anyone else I can think of offhand. The real value of AIXI models is that they make a good concensus model to work from when discussing theory, kind of like Turing machines. The AIXItl model (finite bounds in time and space) is actually of more relevance than the fully general version when discussing intelligence and predictive error complexity.

Even better, the reference bibliographies in his papers are nearly ideal reading lists for the foundational mathematics, including some key topics that he does not write about. The idea of predictive error complexity actually comes from some seminal work by Feder, Merhav, and Gutman in the mid-90s.

[Virginia-American]

Thanks!