Abstract: Intelligent creatures can explore their environments and learn useful skills without supervision. In this paper, we propose a method for learning useful skills without a reward function. We maximize an information theoretic objective using a maximum entropy policy.