Deep Reinforcement Learning with a Natural Language Action Space - He, Ji; Chen, Jianshu; He, Xiaodong; Gao, Jianfeng; Li, Lihong; Deng, Li; Ostendorf, Mari (Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), 2016)
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games. Termed a deep reinforcement relevance network (DRRN), the architecture represents action and state spaces with separate embedding vectors, which are combined with an interaction function to approximate the Q-function in reinforcement learning. We evaluate the DRRN on two popular text games, showing superior performance over other deep Q-learning architectures. Experiments with paraphrased action descriptions show that the model is extracting meaning rather than simply memorizing strings of text.