Estimating the 6D pose of known objects is important for robots to interact with real world.The problem challenging due variety as well complexity a scene caused by clutter and occlusions between objects.In this work, we introduce PoseCNN, new Convolutional Neural Network object estimation.PoseCNN estimates 3D translation an localizing its center in image predicting distance from camera.The rotation estimated regressing quaternion representation.We also novel loss function that enables...

Robust estimation of correspondences between image pixels is an important problem in robotics, with applications tracking, mapping, and recognition objects, environments, other agents. Correspondence has long been the domain hand-engineered features, but more recently deep learning techniques have provided powerful tools for features from raw data. The drawback latter approach that a vast amount (labeled, typically) training data are required learning. This paper advocates new to visual...

We propose a novel approach for 3D video synthesis that is able to represent multi-view recordings of dynamic real-world scene in compact, yet expressive representation enables high-quality view and motion interpolation. Our takes the high quality compactness static neural radiance fields new direction: model-free, setting. At core our time-conditioned field represents dynamics using set compact latent codes. are significantly boost training speed perceptual generated imagery by hierarchical...

This paper introduces DART, a general framework for tracking articulated objects composed of rigid bodies connected through kinematic tree.DART covers broad set encountered in indoor environments, including furniture and tools, human robot bodies, hands manipulators.To achieve efficient robust tracking, DART extends the signed distance function representation to takes full advantage highly parallel GPU algorithms data association pose optimization.We demonstrate capabilities on different...

This work integrates visual and physical constraints to perform real-time depth-only tracking of articulated objects, with a focus on robot's manipulators manipulation targets in realistic scenarios. As such, we extend DART, an existing object tracker, additionally avoid interpenetration multiple interacting make use contact information collected via torque sensors or touch sensors. To achieve greater stability, the tracker uses switching model detect when is stationary relative table palm...

Object-oriented maps are important for scene understanding since they jointly capture geometry and semantics, allow individual instantiation meaningful reasoning about objects. We introduce FroDO, a method accurate 3D reconstruction of object instances from RGB video that infers their location, pose shape in coarse to fine manner. Key FroDO is embed shapes novel learnt space allows seamless switching between sparse point cloud dense DeepSDF decoding. Given an input sequence localized frames,...

We present STaR, a novel method that performs Self-supervised Tracking and Reconstruction of dynamic scenes with rigid motion from multi-view RGB videos without any manual annotation. Recent work has shown neural networks are surprisingly effective at the task compressing many views scene into learned function which maps viewing ray to an observed radiance value via volume rendering. Unfortunately, these methods lose all their predictive power once object in moved. In this work, we...

Estimating the 6D pose of known objects is important for robots to interact with real world. The problem challenging due variety as well complexity a scene caused by clutter and occlusions between objects. In this work, we introduce PoseCNN, new Convolutional Neural Network object estimation. PoseCNN estimates 3D translation an localizing its center in image predicting distance from camera. rotation estimated regressing quaternion representation. We also novel loss function that enables...

Object-oriented maps are important for scene understanding since they jointly capture geometry and semantics, allow individual instantiation meaningful reasoning about objects. We introduce FroDO, a method accurate 3D reconstruction of object instances from RGB video that infers location, pose shape in coarse-to-fine manner. Key to FroDO is embed shapes novel learnt space allows seamless switching between sparse point cloud dense DeepSDF decoding. Given an input sequence localized frames,...

The last several years have seen significant progress in using depth cameras for tracking articulated objects such as human bodies, hands, and robotic manipulators. Most approaches focus on skeletal parameters of a fixed shape model, which makes them insufficient applications that require accurate estimates deformable object surfaces. To overcome this limitation, we present 3D model-based system objects. Our is able to track body pose high resolution surface contours real time commodity...

In wearable AR/VR systems, data transmission between cameras and central processors can account for a significant portion of total system power, particularly in high framerate applications. Thus, it becomes necessary to compress video streams reduce the cost transmission. this paper we present CNN-based compression scheme such vision systems. We demonstrate that, unlike conventional techniques, our method be tuned specific machine This enables increased given application performance target....

Accurate 6-DoF camera pose estimation in known environments can be a very challenging task, especially when the query image was captured at viewpoints strongly differing from set of reference poses. While structure-based methods have proved to deliver accurate estimates, they rely on pre-computed 3D descriptors coming images often misaligned with images. This discrepancy subsequently harm downstream tasks. In this paper we introduce Feature Query Network (FQN), ray-based descriptor regressor...

Efficiently reconstructing complex and intricate surfaces at scale is a long-standing goal in machine perception. To address this problem we introduce Deep Local Shapes (DeepLS), deep shape representation that enables encoding reconstruction of high-quality 3D shapes without prohibitive memory requirements. DeepLS replaces the dense volumetric signed distance function (SDF) used traditional surface systems with set locally learned continuous SDFs defined by neural network, inspired recent...

Neural shape models can represent complex 3D shapes with a compact latent space. When applied to dynamically deforming such as the human hands, however, they would need preserve temporal coherence of deformation well intrinsic identity subject. These properties are difficult regularize manually designed loss functions. In this paper, we learn neural model that disentangles identity-induced variations from pose-dependent deformations using implicit We perform template-free unsupervised...

We propose a novel explicit dense 3D reconstruction approach that processes set of images scene with sensor poses and calibrations estimates photo-real digital model. One the key innovations is underlying volumetric representation completely in contrast to neural network-based (implicit) alternatives. encode scenes explicitly using clear understandable mappings optimization variables geometry their outgoing surface radiance. represent them hierarchical fields stored sparse voxel octree....

The last several years have seen significant progress in using depth cameras for tracking articulated objects such as human bodies, hands, and robotic manipulators. Most approaches focus on skeletal parameters of a fixed shape model, which makes them insufficient applications that require accurate estimates deformable object surfaces. To overcome this limitation, we present 3D model-based system objects. Our is able to track body pose high resolution surface contours real time commodity...