Abstract: When playing from pixels, AIs share some of the struggles that humans face when playing a game, namely not knowing its internal state. In this paper we begin the exploration of the ...