ARMBench is a large-scale benchmark dataset for perception and manipulation challenges across Amazon Robotics systems. The datasets are collected in Amazon warehouses and capture a wide variety of objects and configurations. They comprise images, videos, and tabular data for different stages of robotic manipulation with high-quality annotations.
ARMBench currently includes datasets from two Amazon Robotics programs:
Sparrow - Amazon's intelligent robotic system that streamlines the fulfillment process by moving individual products before they get packaged. Sparrow is a robotic system in Amazon warehouses that can detect, select, and handle individual products in inventory using computer vision and artificial intelligence. It focuses on object segmentation, identification, and defect detection, enabling more efficient and safer operations while allowing employees to focus on higher-value tasks.
Vulcan Stow - Amazon's first robot with a sense of touch, built on advances in robotics, engineering, and physical AI. Vulcan can manipulate objects within fabric-covered pods that are divided into compartments, each holding up to 10 items. Using specialized end-of-arm tooling and force feedback sensors, Vulcan can detect when it makes contact with objects and apply appropriate force, making stowing operations more efficient and safer while achieving human levels of packing density and speed.
Currently, the dataset provides data annotations for four main tasks across these robotic systems.
450,000+ high-quality labels for object segments on 50,000+ images. Clutter and variety of objects present a novel challenge for instance segmentation algorithms.
Open-set object recognition challenge with 200,000+ unique objects. Benchmark will evaluate image retrieval and classification methods with uncertainty estimation.
19,000+ images and 4,000+ videos for rare, but costly, robot-induced defects such as multi-pick and packaging defects. Dataset contains 100,000+ no defect cases.
These datasets are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0). Use the following citations if you use these datasets for publication.
