Empowerment Gain as Causal Learning, Causal Learning as Empowerment Gain: A bridge between Bayesian causal hypothesis testing and reinforcement learning

Alison Gopnik (University of California, Berkeley)

Learning about the causal structure of the world is a fundamental problem for human cognition, and causal knowledge is central to both intuitive and scientific world models. However, causal models and especially causal learning have proved to be difficult for standard Large Models using standard techniques of deep learning. In contrast, cognitive scientists have applied advances in our formal understanding of causation in computer science, particularly within the Causal Bayes Net formalism, to understand human causal learning. These approaches also face challenges when it comes to learning however. In parallel, in the very different tradition of reinforcement learning, researchers have developed the idea of an intrinsic reward signal called “empowerment”. An agent is rewarded for maximizing the mutual information between its actions and their outcomes, regardless of the external reward value of those outcomes. In other words, the agent is rewarded if variation in an action systematically leads to parallel variation in an outcome so that variation in the action predicts variation in the outcome. Empowerment, then has two dimensions , it involves both controllability and variability. The result is an agent that has maximal control over the maximal part of its environment. “Empowerment” may be an important bridge between classical Bayesian causal learning and reinforcement learning and may help to characterize causal learning in humans and enable it in machines.  If an agent learns an accurate causal model of the world they will necessarily increase their empowerment, and, vice versa, increasing empowerment will lead to a more accurate (if implicit) causal model of the world. Empowerment may also explain distinctive empirical features of children’s causal learning, as well as providing a more tractable computational account of how that learning is possible. 

Please send us a message on [email protected] if you wish to attend this talk (indicating if in person or on zoom), so we can keep track of numbers.

Hybrid event: https://monash.zoom.us/j/89623751239?pwd=XFbsbJy76Fk5rxlwzNBoLHcqyHxaoW.1