Human-robot co-transportation allows for a human and robot to perform an object transportation task cooperatively on shared environment. This range of applications raises great number theoretical practical challenges arising mainly from the unknown human-robot interaction model as well difficulty accurately dynamics. In this article, adaptive impedance controller is put forward in space. Vision force sensing are employed obtain hand position, measure between robot. Using latest developments...

In this article, we propose a hybrid framework using visual and force sensing for human-robot co-carrying tasks. Visual is utilized to obtain human motion an observer designed estimating control input of human, which generates robot's desired toward human's intended motion. An adaptive impedance-based strategy proposed trajectory tracking with neural networks used compensate uncertainties in dynamics. Motion synchronization achieved approach yields stable efficient interaction behavior...

In this letter, we propose a novel 3D shape sensing algorithm for flexible endoscopic surgery using multi-core fiber Bragg grating (FBG) sensors. Considering the signal noises and environmental perturbations, direct use of FBG measurements position estimation is regarded as far from accurate stable, especially when utilized in long surgical instruments. To solve problem, generic model-based filtering technique iterative curvature/twist by taking advantage configurations FBGs optical...

Autonomous surgical execution relieves tedious routines and surgeon's fatigue. Recent learning-based methods, especially reinforcement learning (RL) based achieve promising performance for dexterous manipulation, which usually requires the simulation to collect data efficiently reduce hardware cost. The existing platforms medical robots suffer from limited scenarios simplified physical interactions, degrades real-world of learned policies. In this work, we designed SurRoL, an RL-centered...

Objective: The computation of anatomical information and laparoscope position is a fundamental block surgical navigation in Minimally Invasive Surgery (MIS).Recovering dense 3D structure scene using visual cues remains challenge, the online laparoscopic tracking primarily relies on external sensors, which increases system complexity.Methods: Here, we propose learning-driven framework, an image-guided localization with reconstructions complex structures obtained.To reconstruct whole...

The integration of Large Language Models (LLMs) like GPT-4 with Extended Reality (XR) technologies offers the potential to build truly immersive XR environments that interact human users through natural language, e.g., generating and animating 3D scenes from audio inputs. However, complexity makes it difficult accurately extract relevant contextual data scene/object parameters an overwhelming volume artifacts. It leads not only increased costs pay-per-use models, but also elevated levels...

Laparoscopic Field of View (FOV) control is one the most fundamental and important components in Minimally Invasive Surgery (MIS), nevertheless traditional manual holding paradigm may easily bring fatigue to surgical assistants, misunderstanding between surgeons also hinders assistants provide a high-quality FOV. Targeting this problem, we here present data-driven framework realize an automated laparoscopic optimal FOV control. To achieve goal, offline learn motion strategy laparoscope...

This paper proposes a novel graph-based framework for 3-D shape sensing of flexible medical instruments using multi-core fiber Bragg grating (FBG) sensors. Due to noisy signals, deformability instruments, and environmental disturbances, conventional methods direct FBG measurements are far from accurate stable, especially long devices. The localization errors will substantially accumulate with the increase lengths. To tackle this challenge, we propose generic graph optimize entire globally...

Problems concerning transformation estimation for remote sensing image registration are studied in this letter. Based on the affine-invariance property of triangle-area representation, a new algorithm called robust sample consensus judging is proposed. It can be embedded into most algorithms with hypothesis-and-testing frameworks (such as random consensus) and significantly improve their computational efficiency without sacrificing accuracy. Simulation experimental results show merits...

To realize a higher-level autonomy of surgical knot tying in minimally invasive surgery (MIS), automated suture grasping, which bridges the stitching and looping procedures, is an important yet challenging task needs to be achieved. This paper presents holistic framework with image-guided automation techniques robotize this operation even under complex environments. The whole initialized by segmentation, we propose novel semi-supervised learning architecture featured suture-aware loss...

Task automation of surgical robot has the potentials to improve efficiency. Recent reinforcement learning (RL) based approaches provide scalable solutions automation, but typically require extensive data collection solve a task if no prior knowledge is given. This issue known as exploration challenge, which can be alleviated by providing expert demonstrations an RL agent. Yet, how make effective use demonstration efficiency still remains open challenge. In this work, we introduce...

Automated laparoscope field of view (FoV) control in minimal invasive surgery (MIS) poses challenges, as existing solutions failed to address dynamic surgical FoV requirements across different phases and they neglected the misorientation effect or potential obstacles during process which raised safety concerns. In this letter, we propose a Gaussian mixture model (GMM)-based heuristic decision framework that can achieve safe automatic provide phase-specific for surgeons. Leveraging GMM fit...

Smart laparoscope motion control for adjusting surgical field-of-view is an increasingly hot topic in robot-assisted surgery. Previous off-the-shelf methods have been conducted reactive ways which heavily rely on human input signals, e.g., gaze or voice, thus cannot avoid cognitive burdens to surgeons. In this paper, we introduce a novel proactive framework that learns the strategy from clinical videos achieve autonomous specific, first propose robust estimation method acquire dynamic...

Recent advancements toward perception and decision-making of flexible endoscopes have shown great potential in computer-aided surgical interventions. However, owing to modeling uncertainty inter-patient anatomical variation endoscopy, the challenge remains for efficient safe navigation patient-specific scenarios. This paper presents a novel data-driven framework with self-contained visual-shape fusion autonomous intelligent requiring no priori knowledge system models global environments. A...

Manipulation in tight environment is challenging but increasingly common vision-guided robotic applications. The significantly reduced amount of available feedback (limited visual cues, field view, robot motion space, etc.) hinders solving the hand-eye relationship accurately. In this article, we propose a new generic approach for online robot–camera calibration that could deal with least input environment: an arbitrarily restricted space and single feature point unknown position...

Modeling agile and versatile spatial behavior remains a challenging task, due to the intricate coupling of planning, control, perceptual processes. Previous results have shown that humans plan organize their guidance by exploiting patterns in interactions between agent or organism environment. These patterns, described under concept Interaction Patterns (IPs), capture invariants arising from equivalences symmetries interaction with environment, as well effects intrinsic properties human...

Virtual Reality (VR), together with the network infrastructure, can provide an interactive and immersive experience for multiple users simultaneously thus enables collaborative VR applications (e.g., VR-based classroom). However, satisfactory user requires not only high-resolution panoramic image rendering but also extremely low latency seamless experience. Besides, competition limited resources share total bandwidth) poses a significant challenge to experience, in particular under wireless...

Shape from Polarization (SfP) aims to recover surface normal using the polarization cues of light. The accuracy existing SfP methods is affected by two main problems. First, ambiguity partially results in false estimation. Second, widely-used assumption about orthographic projection too ideal. To solve these problems, we propose first approach that com-bines deep learning and stereo information not only but also disparity. Specifically, for problem, design a Consistency-based Mask Prediction...

Image-guided needle pose estimation is crucial for robotic autonomous suturing, but it poses significant challenges due to the needle's slender visual projection and dynamic surgical environments. Current state-of-the-art methods rely on additional prior information (e.g., in-hand grasp, accurate kinematics, etc.) achieve sub-millimeter accuracy, hindering their applicability in varying scenes. This paper presents a new generic framework monocular estimation: Visual learning network...

Automatic laparoscope motion control is fundamentally important for surgeons to efficiently perform operations. However, its traditional methods based on tool tracking without considering information hidden in surgical scenes are not intelligent enough, while the latest supervised imitation learning (IL)-based require expensive sensor data and suffer from distribution mismatch issues caused by limited demonstrations. In this paper, we propose a novel Imitation Learning framework Laparoscope...

Simultaneous Localization and Mapping (SLAM) autonomous path planning, which is an important part of intelligent mobile robot research. In order to improve the robot's capability planning in dynamic environment, this paper studies based on laser slam scene. Firstly, sensor carried by used obtain change pose state robot, Cartographer algorithm graph optimization map indoor environment ROS robot. realize function global current methods are compared, Dijkstra $\mathrm{A}^{\star}$ algorithms...

With progressive advancements in modern technologies, image-guided automation, which plans and manipulates particular tasks by utilizing visual cues, is becoming an emerging sector medical robotics. Its prevalence nowadays requires a reliable hand-to-eye calibration owing to various configurations of surgical robots, especially for those with remote center motion (RCM) constraints, e.g., da Vinci Research Kit (dVRK). In this article, we proposed novel unified method, leverages the structure...

Estimating precise metric depth and scene reconstruction from monocular endoscopy is a fundamental task for surgical navigation in robotic surgery. However, traditional stereo matching adopts binocular images to perceive the information, which difficult transfer soft robotics-based systems due use of endoscopy. In this paper, we present novel framework that combines robot kinematics endoscope with deep unsupervised learning into single network estimation then achieve 3D complex anatomy....