---
abstract: |
  Imitation Learning has empowered recent advances in learning robotic manipulation tasks by addressing shortcomings of Reinforcement Learning such as exploration and reward specification. However, research in this area has been limited to modest-sized datasets due to the difficulty of collecting large quantities of task demonstrations through existing mechanisms. This work introduces `\sysAbbr `{=latex}to address this challenge. `\sysAbbr `{=latex}is a crowdsourcing platform for high quality 6-DoF trajectory based teleoperation through the use of widely available mobile devices (e.g. iPhone). We evaluate `\sysAbbr `{=latex}on three manipulation tasks of varying timescales (15-120s) and observe that our user interface is statistically similar to special purpose hardware such as virtual reality controllers in terms of task completion times. Furthermore, we observe that poor network conditions, such as low bandwidth and high delay links, do not substantially affect the remote users' ability to perform task demonstrations successfully on `\Sys`{=latex}. Lastly, we demonstrate the efficacy of `\sysAbbr `{=latex}through the collection of a pilot dataset; using `\sysAbbr`{=latex}, we collected 137.5 hours of manipulation data from remote workers, amounting to over 2200 successful task demonstrations in 22 hours of total system usage. We show that the data obtained through `\sysAbbr `{=latex}enables policy learning on multi-step manipulation tasks with sparse rewards and that using larger quantities of demonstrations during policy learning provides benefits in terms of both learning consistency and final performance. For additional results, videos, and to download our pilot dataset, visit [`roboturk.stanford.edu`](http://roboturk.stanford.edu/)
author:
- |
  Ajay Mandlekar$^\ddagger$, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung,\
  \
  **Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei**\
  \
  Department of Computer Science, Stanford University\
  $^\ddagger$`amandlek@stanford.edu`
bibliography:
- ccr.bib
title: |
  `\sysAbbr`{=latex}: A Crowdsourcing Platform for\
  Robotic Skill Learning through Imitation
---

```{=latex}
\newcommand{\seclabel}[1]{\label{sec:#1}}
```
```{=latex}
\newcommand{\secref}[1]{Section~\ref{sec:#1}}
```
```{=latex}
\newcommand{\tabref}[1]{Table~\ref{tab:#1}}
```
```{=latex}
\newcommand{\Sys}{\textsc{RoboTurk}\xspace}
```
```{=latex}
\newcommand{\sysAbbr}{\textsc{RoboTurk}\xspace}
```
```{=latex}
\maketitle
```
```{=latex}
\keywords{Crowdsourcing, Imitation Learning, Skill Learning, Manipulation}
```
```{=latex}
\renewcommand{\textfloatsep}{3mm}
```
```{=latex}
\renewcommand{\dbltextfloatsep}{3mm}
```
```{=latex}
\input{1-intro-v3.tex}
```
```{=latex}
\input{3-sysdesign.tex}
```
```{=latex}
\input{4-analysis.tex}
```
```{=latex}
\input{5-learning-expt-v2.tex}
```
```{=latex}
\input{6-disc.tex}
```
```{=latex}
\clearpage
```
```{=latex}
\input{7-acknowledgments.tex}
```
```{=latex}
\renewcommand{\baselinestretch}{0.88}
```
```{=latex}
\renewcommand*{\bibfont}{\small}
```
```{=latex}
\renewcommand{\baselinestretch}{1}
```
```{=latex}
\input{8-appendix.tex}
```