---
abstract: |
  Despite recent successes of reinforcement learning (RL), it remains a challenge for agents to transfer learned skills to related environments. To facilitate research addressing this problem, we propose *CausalWorld*, a benchmark for causal structure and transfer learning in a robotic manipulation environment. The environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strength of *CausalWorld* is that it provides a combinatorial family of such tasks with common causal structure and underlying factors (including, e.g., robot and object masses, colors, sizes). The user (or the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. One can thus easily define training and evaluation distributions of a desired difficulty level, targeting a specific form of generalization (e.g., only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to very challenging, all of which require long-horizon planning as well as precise low-level motor control. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.^4^
author:
- |
  Ossama Ahmed [^1]\^ \^ [^2] `\And `{=latex}Frederik Träuble `\footnotemark[1]`{=latex}\^ \^ [^3] `\And `{=latex}Anirudh Goyal [^4]`\And `{=latex}Alexander Neitz `\footnotemark[3] `{=latex}`\And `{=latex}Manuel Wütrich `\footnotemark[3] `{=latex}`\And `{=latex}Yoshua Bengio `\footnotemark[4]`{=latex}`\And `{=latex}Bernhard Schölkopf `\footnotemark[3] `{=latex}`\And `{=latex}Stefan Bauer`\footnotemark[3]
  `{=latex}
- |
  Ossama Ahmed ^1,\*^ Frederik Träuble ^2,\*^ Anirudh Goyal ^3^ Alexander Neitz ^2^\
  **Yoshua Bengio ^3^ Bernhard Schölkopf ^2^ Stefan Bauer ^2,$\dag$^ Manuel Wüthrich ^2,$\dag$^**
bibliography:
- iclr2021\_conference.bib
title: 'CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning '
---

```{=latex}
\newcommand{\cmark}{\ding{51}}
```
```{=latex}
\newcommand{\xmark}{\ding{55}}
```
```{=latex}
\newcommand{\Ani}[1]{{\color{blue}[AG: #1]}}
```
```{=latex}
\newcommand{\fix}{\marginpar{FIX}}
```
```{=latex}
\newcommand{\new}{\marginpar{NEW}}
```
```{=latex}
\let\footnote\relax
```
```{=latex}
\footnotetext{
\textsuperscript{*} Equal Contribution ,
\textsuperscript{$\dag$} Equal Advising,
\textsuperscript{1} ETH Zürich,
\textsuperscript{2} MPI Tübingen,
\textsuperscript{3} Mila, University of Montreal,\\
Corresponding author: \texttt{ossama.ahmed@mail.mcgill.ca, frederik.traeuble@tuebingen.mpg.de}
\textsuperscript{4} https://sites.google.com/view/causal-world/home
}
```
```{=latex}
\maketitle
```
Introduction {#sec:introduction}
============

```{=latex}
\input{content/introduction_arxiv.tex}
```
CausalWorld Benchmark {#sec:benchmark}
=====================

```{=latex}
\input{content/benchmark_v2_arxiv.tex}
```
Related Work {#sec:related_work}
============

```{=latex}
\input{content/related_work_arxiv.tex}
```
Experiments {#sec:experiment}
===========

```{=latex}
\input{content/experiments_arxiv.tex}
```
Conclusion {#sec:conclusion}
==========

```{=latex}
\input{content/conclusion_arxiv.tex}
```
Acknowledgments {#sec:acknowledgments}
===============

The authors would like to thank Felix Widmaier, Vaibhav Agrawal and Shruti Joshi for the useful discussions and for the development of the TriFinger robot's simulator [@trifinger-simulation], which served as a starting point for the work presented in this paper. AG is also grateful to Alex Lamb and Rosemary Nan Ke for useful discussions. The authors are grateful for the support from CIFAR.

```{=latex}
\bibliographystyle{iclr2021_conference}
```
```{=latex}
\clearpage
```
Appendix
========

```{=latex}
\appendix
```
```{=latex}
\input{content/appendix.tex}
```

[^1]: These authors contributed equally: ossama.ahmed\@mail.mcgill.ca, frederik.traeuble\@tuebingen.mpg.de

[^2]: ETH Zürich

[^3]: Max Planck Institute for Intelligent Systems

[^4]: Mila, University of Montreal