This paper presents a method for one-shot learning of dexterous grasps and grasp generation novel objects. A model each type is learned from single kinesthetic demonstration several types are taught. These models used to select generate unfamiliar Both the stages use an incomplete point cloud depth camera, so no prior object shape used. The product experts, in which experts two types. first contact density over pose hand link relative local surface. second hand-configuration whole-hand...

This paper shows how a robot arm can follow and grasp moving objects tracked by vision system, as is needed when human hands over an object to the during collaborative working. While being arbitrarily moved co-worker, set of likely grasps, generated learned planner, are evaluated online generate feasible with respect both: current configuration respecting target grasp; constraints finding collision-free trajectory reach that configuration. A task-based cost function enables relaxation...

We present early pilot-studies of a new international project, developing advanced robotics to handle nuclear waste. Despite enormous remote handling requirements, there has been remarkably little use robots by the industry. The few deployed have directly teleoperated in rudimentary ways, with no control methods or autonomy. Most is still done an aging workforce highly skilled experts, using 1960s style mechanical Master-Slave devices. In contrast, this paper explores how novice human...

An important problem in robotic manipulation is the ability to predict how objects behave under manipulative actions. This necessary allow planning of object manipulations. Physics simulators can be used do this, but they model many kinds interaction poorly. alternative learn a motion for by interacting with them. In this paper we address learning interactions rigid bodies probabilistic framework, and demonstrate results domain push manipulation. A robot arm applies random pushes various...

This paper presents an algorithm for planning sequences of pushes, by which a robotic arm equipped with single rigid finger can move manipulated object (or manipulandum) towards desired goal pose. Pushing is perhaps the most basic kind manipulation, however it difficult challenges planning, because complex relationship between manipulative pushing actions and resulting manipulandum motions. The motion literature has well developed paradigms solving e.g. piano-mover's problem, where search...

The ability to predict how objects behave during manipulation is an important problem. Models informed by mechanics are powerful, but hard tune. An alternative learn a model of the object's motion from data, predict. We study this for push manipulation. paper starts formulating quasi-static prediction then pose problem learning in two different frameworks: (i) regression and (ii) density estimation. Our architecture modular: many simple, object specific, context specific predictors learned....

This paper concerns the problem of how to learn grasp dexterously, so as be able then novel objects seen only from a single viewpoint. Recently, progress has been made in data-efficient learning generative models that transfer well objects. These are learned demonstration (LfD). One weakness is that, this shall show, under challenging single-view conditions unreliable. Second, number model elements increases linearly training examples. This, turn, limits potential these for generalization...

Generalising dexterous grasps to novel objects is an open problem. We show how learn for high DoF hands that generalise objects, given as little one demonstrated grasp. During grasp learning two types of probability density are learned model the The first type (the contact model) models relationship individual finger part local surface features at its point. second hand configuration whole during approach When presented with a new object, many candidate generated, and kinematically feasible...

How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping unfamiliar objects. By we simply mean by any hand with more than two fingers, such that has some choice about where place each finger. Such grasps typically fail in one ways, either unmodeled objects scene cause collisions or object reconstruction is insufficient grasp points provide stable force closure. These problems can be solved easily if sensing guided...

Scene and object reconstruction is an important problem in robotics, particular planning collision-free trajectories or manipulation. This paper compares two strategies for the of nonvisible parts surface from a single RGB-D camera view. The first method, named DeepSDF predicts Signed Distance Transform to given point 3D space. second MirrorNet reconstructs occluded objects' by generating images other side observed object. Experiments performed with objects ShapeNet dataset, show that...

This paper addresses the problem of choosing between several different possible grasps on an object, in order to enable efficient manipulation that object after it has been grasped. In this work, we assume mass distribution object's inertia tensor, and robot's dynamic model are known a priori. We then show how each grasp can be expressed as augmented dynamics model, which combines inertias both robot grasped single formulation. The used estimate joint torques needed move along desired...

This paper presents a data-efficient approach to learning transferable forward models for robotic push manipulation.Our extends our previous work on contactbased predictors by leveraging information the pushed object's local surface features.We test hypothesis that, conditioning predictions features, we can achieve generalisation across objects of different shapes.In doing so, do not require CAD model object but rather rely point cloud (PCOM).Our involves motion that are specific contact...

This paper addresses the problem of jointly planning both grasps and subsequent manipulative actions. Previously, these two problems have typically been studied in isolation, however joint reasoning is essential to enable robots complete real tasks. In this paper, are addressed a solution that takes into consideration proposed. To do so, manipulation capability index defined, which function task execution waypoints object grasping contact points. We build on recent state-of-the-art...

Planning robust robot manipulation requires good forward models that enable plans to be found. This work shows how achieve this using a model learned from data plan push manipulations. We explore learning methods (Gaussian Process Regression, and an Ensemble of Mixture Density Networks) give estimates the uncertainty in their predictions. These are utilised by predictive path integral (MPPI) controller box goal location. The planner avoids regions high model. includes both inherent dynamics,...

Dexterous grasping of a novel object given single view is an open problem. This paper makes several contributions to its solution. First, we present simulator for generating and testing dexterous grasps. Second, dataset, generated by this simulator, 2.4 million simulated grasps variations 294 base objects drawn from 20 categories. Third, combine existing approach learn grasp generation model with three different learned evaluative models employing ResNet-50 or VGG16 as their visual backbone....

Dexterous grasping of objects with uncertain pose is a hard unsolved problem in robotics. This paper solves this using information gain re-planning. First we show how tactile information, acquired during failed attempt to grasp an object can be used refine the estimate that object's pose. Second, replan new reach trajectories for successive attempts. Finally reach-to-grasp modified, so they maximise expected gain, while simultaneously delivering hand configuration most likely succeed. Our...

Belief space planning is a viable alternative to formalise partially observable control problems and, in the recent years, its application robot manipulation has grown. However, this approach was tried successfully only on simplified problems. In paper, we apply belief problem of dexterous reach-to-grasp trajectories under object pose uncertainty. our framework, perceives be grasped on-the-fly as point cloud and compute full 6D, non-Gaussian distribution over object's (our space). The system...

Predicting the motions of rigid objects under contacts is a necessary precursor to planning robot manipulation objects. On one hand physics based body simulations are used, and on other learning approaches being developed. The advantage that because they explicitly perform collision checking respect kinematic constraints, producing physically plausible predictions. can capture effects motion unobservable parameters such as mass distribution, frictional coefficients, thus more accurate...

Visual tracking of an object can provide a powerful source feedback information during complex robotic manipulation operations, especially those in which there may be uncertainty about new pose result from planned manipulative action. At the same time, challenging environment for visual tracking, with occlusions by other objects or robot itself, and sudden changes that accompanied motion blur. Recursive filtering techniques use models predictor-corrector but simple typically used often fail...

Dexterous grasping of a novel object given single view is an open problem. This paper makes several contributions to its solution. First, we present simulator for generating and testing dexterous grasps. Second data set, generated by this simulator, 2.4 million simulated grasps variations 294 base objects drawn from 20 categories. Third, basic architecture generation evaluation that may be trained in supervised manner. Fourth, three different evaluative architectures, employing ResNet-50 or...

We present a parametric formulation for learning generative models grasp synthesis from demonstration. cast new light on this family of approaches, proposing that is computationally faster compared to related work and indicates better success rate performance in simulated experiments, showing gain at least 10% (p < 0.05) all the tested conditions. The proposed implementation also able incorporate arbitrary constraints ranking may include task-specific constraints. Results are reported...