To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset 6.7 million point clouds, grasps, and analytic metrics generated thousands 3D models Dex-Net 1.0 in randomized poses on table.We use the resulting dataset, 2.0, to train Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts probability success grasps depth images, where are specified as planar position, angle, gripper relative an RGB-D...

Universal picking (UP), or reliable robot grasping of a diverse range novel objects from heaps, is grand challenge for e-commerce order fulfillment, manufacturing, inspection, and home service robots. Optimizing the rate, reliability, UP difficult due to inherent uncertainty in sensing, control, contact physics. This paper explores "ambidextrous" grasping, where two more heterogeneous grippers are used. We present Dexterity Network (Dex-Net) 4.0, substantial extension previous versions...

This paper presents the Dexterity Network (Dex-Net) 1.0, a dataset of 3D object models and sampling-based planning algorithm to explore how Cloud Robotics can be used for robust grasp planning. The uses Multi- Armed Bandit model with correlated rewards leverage prior grasps in growing that currently includes over 10,000 unique 2.5 million parallel-jaw grasps. Each an estimate probability force closure under uncertainty gripper pose friction. Dex-Net 1.0 Multi-View Convolutional Neural...

Vacuum-based end effectors are widely used in industry and often preferred over parallel-jaw multifinger grippers due to their ability lift objects with a single point of contact. Suction grasp planners target planar surfaces on clouds near the estimated centroid an object. In this paper, we propose compliant suction contact model that computes quality seal between cup local surface measure resist external gravity wrench. To characterize grasps, estimate robustness perturbations end-effector...

To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset 6.7 million point clouds, grasps, and analytic metrics generated thousands 3D models Dex-Net 1.0 in randomized poses on table. We use the resulting dataset, 2.0, to train Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts probability success grasps depth images, where are specified as planar position, angle, gripper relative an RGB-D...

The ability to segment unknown objects in depth images has potential enhance robot skills grasping and object tracking. Recent computer vision research demonstrated that Mask R-CNN can be trained specific categories of RGB when massive hand-labeled datasets are available. As generating these is time-consuming, we instead train with synthetic images. Many robots now use sensors, recent results suggest training on data transfer successfully the real world. We present a method for automated...

Rapid and reliable robot grasping for a diverse set of objects has applications from warehouse automation to home decluttering. One promising approach is learn deep policies synthetic training datasets point clouds, grasps, rewards sampled using analytic models with stochastic noise domain randomization. In this letter, we explore how the distribution examples affects rate reliability learned policy. We propose data sampling that combines grasps policy action guiding samples robust...

Autonomous robot execution of surgical sub-tasks has the potential to reduce surgeon fatigue and facilitate supervised tele-surgery. This paper considers sub-task debridement: removing dead or damaged tissue fragments allow remaining healthy heal. We present an autonomous multilateral debridement system using Raven, open-architecture with two cable-driven 7 DOF arms. Our combines stereo vision for 3D perception trajopt, optimization-based motion planner, model predictive control (MPC)....

Precise control of industrial automation systems with non-linear kinematics due to joint elasticity, variation in cable tensioning, or backlash is challenging; especially that can only be controlled through an interface imprecise internal kinematic model. Cable-driven Robotic Surgical Assistants (RSAs) are one example such system, as they designed for master-slave teleoperation. We consider the problem learning a function modify commands inaccurate executing modified command on system...

The school-based Children's Support Group procedure teaches skills to cope with divorce-related events and provides strategies for mastering disrupted developmental tasks. Ss were 103 3rd-through 5th-grade children of separated or divorced parents who assigned 1 3 treatment groups: support; support skill building; support, building, transfer, parent training procedures; no-treatment control. Twenty-six from intact homes served as nonstressed controls. two skill-building conditions yielded...

Computing grasps for an object is challenging when the geometry not known precisely. In this paper, we explore use of Gaussian process implicit surfaces (GPISs) to represent shape uncertainty from RGBD point cloud observations objects. We study GPIS representations select on previously unknown objects, measuring grasp quality by probability force closure. Our main contribution GP-GPIS-OPT, algorithm computing parallel-jaw grippers 2D representations. Specifically, our method optimizes...

To facilitate automated bin picking when parts cannot be grasped, pushing actions have the potential to separate objects and move them away from walls corners. In context of Dexterity Network (Dex-Net) robot grasping framework, we present two novel push policies based on targeting free space diffusing clusters, compare three earlier using four metrics. We evaluate these in simulation Bullet Physics a dataset over 1,000 synthetic scenarios. Pushing outcomes are evaluated by comparing quality...

Motivated by recent advances in Deep Learning for robot control, this paper considers two learning algorithms terms of how they acquire demonstrations from fallible human supervisors. Human-Centric (HC) sampling is a standard supervised algorithm, where supervisor demonstrates the task teleoperating to provide trajectories consisting state-control pairs. Robot-Centric (RC) an increasingly popular alternative used such as DAgger, observes execute learned policy and provides corrective control...

Online learning from demonstration algorithms such as DAgger can learn policies for problems where the system dynamics and cost function are unknown. However they impose a burden on supervisors to respond queries each time robot encounters new states while executing its current best policy. The MMD-IL algorithm reduces supervisor by filtering with insufficient discrepancy in distribution maintaining multiple policies. We introduce SHIV (Svm-based reduction Human InterVention), which...

Parallel-jaw robot grippers can grasp almost any object and are ubiquitous in industry. Although the shape, texture, compliance of gripper jaw surfaces affect robustness, all commercially available provide a pair rectangular, planar, rigid surfaces. Practitioners often modify these with variety ad-hoc methods such as adding rubber caps and/or wrapping textured tape. This paper explores data-driven optimization over design space based on using rapid prototyping. In total, 37 surface...

Caging grasps are valuable as they can be robust to bounded variations in object shape and pose, do not depend on friction, enable transport of an without full immobilization. Complete caging is useful but may necessary cases where forces such gravity present. This letter extends theory by defining energy-bounded cages with respect energy field gravity. also introduces energy-bounded-cage-analysis-2-D (EBCA-2-D), a sampling-based algorithm for planar analysis that takes input function over...

In support of Cloud Robotics, Robotics and Automation as a Service (RAaaS) frameworks have the potential to reduce complexity software development, simplify installation maintenance, facilitate data sharing for machine learning. this proof-of-concept paper, we describe Berkeley (Brass), RAaaS prototype that allows robots access remote server hosts robust grasp-planning system (Dex-Net 1.0) maintains on hundreds candidate grasps thousands 3D object meshes uses perturbation sampling estimate...

Learning-based approaches to robust robot grasp planning can a wide variety of objects, but may be prone failure on some objects. Inspired by recent results in computer vision, we define class "adversarial objects that are physically similar given object significantly less "graspable" terms specified grasping policy. We present three algorithms for synthesizing adversarial under the reliability measure Dex-Net 1.0 parallel-jaw grippers: 1) two analytic perturb vertices antipodal faces (one...

Accessing software resources via the Cloud has become increasingly popular as a means to configure and manage automation systems with reduced infrastructure overhead. Dex-Net Service (DNaaS) is cloud-based grasp planning system for parallel-jaw grippers that provides graphical user interface API access 1.0, robust based on wrench mechanics stochastic sampling. DNaaS allows anyone online compute grasps triangular meshes visualize results. This paper presents architecture, examples of...

Robot grasping of deformable hollow objects such as plastic bottles and cups is challenging, the grasp should resist disturbances while minimally deforming object so not to damage it or dislodge liquids. We propose minimal work a novel quality metric that combines wrench resistance deformation. introduce an efficient algorithm compute required external for manipulation task by solving linear program. The first computes minimum force estimation gripper jaw displacements based on object's...

Deep Learning from Demonstrations (Deep LfD) is a promising approach for robots to perform precise bilateral automation tasks involving contact and deformation, where dynamics are difficult model explicitly. LfD methods typically use datasets of 1) human videos, which do not match robot kinematics capabilities or 2) waypoints collected with tedious move-and-record interfaces, such as teaching pendants kinesthetic teaching. We explore an alternative using the Intuitive Surgical da Vinci,...