Speaking of the panel discussion on information th...

2007-09-13T03:19:00.000-07:00

Speaking of the panel discussion on information theory, here is the paper I mentioned in my comment, in which Edwin Jaynes analyses data from rolling a die and uses information theory to deduce certain physical facts about how the dice was made:

Jaynes, E. T., 1979, `Where do we Stand on Maximum Entropy?' (2.6Mb) in The Maximum Entropy Formalism, R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA, p. 15;

It was relevant to the discussion because it was being claimed that information theory will never tell you about the 'whys' of what's happening in an agent, only the 'whats.' Although it's an analysis of a die rather than a dynamical agent I think this demonstrates rather nicely a way in which information theory can give you the whys.

Unfortunately the paper is rather long - it's practically a book - but it's well worth a read by anyone with an interest in information theory.

Also I'd like to address the comment in your blog post that information theory requires that the agent be an "input-output system." In my view this is simply an artifact of the way that information theory has been applied in ALife studies so far. There is no such requirement in the mathematical formalism of the theory. An analysis more along the lines of the one in Jaynes' paper would not require this, for instance. I think we should see information theory simply as another tool for analysing dynamical systems - one that can compliment phase portraits etc. rather than being somehow in opposition to them.

I'd just like to make one final claim: If you, as a dynamicist, wanted to say that a particular agent was not, in fact, gathering all the information about its environment in some kind of optimal manner, the only way you could prove it would be to use information theory (e.g. to show that the mutual information between the agent's internal state and its environment was not maximised after all). So, far from being opposed to such dynamicist claims, information theory is *the* tool which can be used to support them.

Comments on ECAL2007: Food for thought

Speaking of the panel discussion on information th...