D4RL: Datasets for Deep Data-Driven Reinforcement Learning

Submitted by on Mar 16 2021 } Suggest Revision
By: Justin Fu and Aviral Kumar and Ofir Nachum and George Tucker and Sergey Levine
From: RAIL Berkely
Resource Type:
Apache-2.0 License
not code
Data Format:


A collection of benchmarks and datasets for offline reinforcement learning. The offline reinforcement learning (RL) problem, also known as batch RL, refers to the setting where a policy must be learned from a static dataset, without additional online data collection. This setting is compelling as potentially it allows RL methods to take advantage of large, pre-collected datasets, much like how the rise of large datasets has fueled results in supervised learning in recent years. However, existing online RL benchmarks are not tailored towards the offline setting, making progress in offline RL difficult to measure. In this work, we introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL. Examples of such properties include: datasets generated via hand-designed controllers and human demonstrators, multi-objective datasets where an agent can perform different tasks in the same environment, and
Post comment