Non-parametric face modeling aims to reconstruct 3D only from images without shape assumptions. While plausible facial details are predicted, the models tend over-depend on local color appearance and suffer ambiguous noise. To address such problem, this paper presents a novel Learning Aggregate Personalize (LAP) framework for unsupervised robust modeling. Instead of using controlled environment, proposed method implicitly disentangles ID-consistent scene-specific unconstrained photo set....

This work presents FaceX framework, a novel facial generalist model capable of handling diverse tasks simultaneously. To achieve this goal, we initially formulate unified representation for broad spectrum editing tasks, which macroscopically decomposes face into fundamental identity, intra-personal variation, and environmental factors. Based on this, introduce Facial Omni-Representation Decomposing (FORD) seamless manipulation various components, microscopically decomposing the core aspects...

Facial Appearance Editing (FAE) aims to modify physical attributes, such as pose, expression and lighting, of human facial images while preserving attributes like identity background, showing great importance in photograph. In spite the progress this area, current researches generally meet three challenges: low generation fidelity, poor attribute preservation, inefficient inference. To overcome above challenges, paper presents DiffFAE, a one-stage highly-efficient diffusion-based framework...

Current face reenactment and swapping methods mainly rely on GAN frameworks, but recent focus has shifted to pre-trained diffusion models for their superior generation capabilities. However, training these is resource-intensive, the results have not yet achieved satisfactory performance levels. To address this issue, we introduce Face-Adapter, an efficient effective adapter designed high-precision high-fidelity editing models. We observe that both reenactment/swapping tasks essentially...

3D-aware GANs have shown their impressive power on 3D controlling for synthesized portraits. While the plausible facial reality is achieved, inherent properties of generated results actually not been well analyzed. One reasons that wildly-used metrics, such as Inception Score (IS) or Fréchet Distance (FID), focus more perceptual features rather than explicit clues. In this article, we propose two novel which measure face consistency and diversity a level, to compensate IS FID GAN evaluation....

We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability generalize arbitrary identity or fail preserve attributes like facial expression gaze direction, our framework is capable of transferring source into target while preserving face. overcome above defects in following two ways. First, we present ID Injection Module (IIM) which transfers information at feature...

This paper presents a novel Physically-guided Disentangled Implicit Rendering (PhyDIR) framework for highfidelity 3D face modeling. The motivation comes from two observations: Widely-used graphics renderers yield excessive approximations against photo-realistic imaging, while neural rendering methods produce superior appearances but are highly entangled to perceive 3D-aware operations. Hence, we learn disentangle the implicit via explicit physical guidance, guaranteeing properties of: (1)...

In this work, we propose a novel framework named Region-Aware Network (RANet), which learns the ability of anti-confusing in case heavy occlusion, nearby person and symmetric appearance, for human pose estimation. Specifically, proposed method addresses three key aspects, i.e., data augmentation, feature learning prediction fusion, respectively. First, Parsing-based Data Augmentation (PDA) to generate abundant that synthesizes confusing textures. Second, not only Feature Pyramid Stem (FPS)...

Stylized Text-to-Image Generation (STIG) aims to generate images based on text prompts and style reference images. We in this paper propose a novel framework dubbed as StyleMaster for task by leveraging pretrained Stable Diffusion (SD), which tries solve the previous problems such insufficient inconsistent semantics. The enhancement lies two module, namely multi-source embedder dynamic attention adapter. In order provide SD with better embeddings, we considers both global local level visual...

In the field of image editing, three core challenges persist: controllability, background preservation, and efficiency. Inversion-based methods rely on time-consuming optimization to preserve features initial images, which results in low efficiency due requirement for extensive network inference. Conversely, inversion-free lack theoretical support similarity, as they circumvent issue maintaining achieve As a consequence, none these can both high consistency. To tackle aforementioned...

Face swapping aims to generate results that combine the identity from source with attributes target. Existing methods primarily focus on image-based face swapping. When processing videos, each frame is handled independently, making it difficult ensure temporal stability. From a model perspective, gradually shifting generative adversarial networks (GANs) diffusion models (DMs), as DMs have been shown possess stronger capabilities. Current diffusion-based approaches often employ inpainting...

Practical and efficient face alignment has been highly required widely focused in recent years, especially under the trend of edge computation real-time operation. And it is a critical need to deal with masked faces context COVID-19 epidemic. In this paper, we propose novel cascaded facial landmark detector towards alignment, which call QCN (Quantized Cascaded Network). consists three stages: estimation refinement. The stage help pre-align alleviate extreme poses. next two stages localize...

Deepfake aims to swap a face of an image with someone else’s likeness in reasonable manner. Existing methods usually perform deepfake frame by frame, thus ignoring video consistency and producing incoherent results. To address such problem, we propose novel framework Neural Identity Carrier (NICe), which learns identity transformation from arbitrary face-swapping proxy via U-Net. By modeling the incoherence between frames as noise, NICe naturally suppresses its disturbance preserves primary...

Human pose estimation is the task of localizing body keypoints from still images. The state-of-the-art methods suffer insufficient examples challenging cases such as symmetric appearance, heavy occlusion and nearby person. To enlarge amounts cases, previous augmented images by cropping pasting image patches with weak semantics, which leads to unrealistic appearance limited diversity. We instead propose Semantic Data Augmentation (SDA), a method that augments segmented parts various semantic...

In-the-wild 3D face modelling is a challenging problem as the predicted facial geometry and texture suffer from lack of reliable clues or priors, when input images are degraded. To address such problem, in this paper we propose novel Learning to Restore (L2R) framework for unsupervised high-quality reconstruction low-resolution images. Rather than directly refining 2D image appearance, L2R learns recover fine-grained details on proxy against degradation via extracting generative priors....

Pedestrian attribute recognition, a multi-task problem, is popular task in computer vision. Generally, deep learning end-to-end networks to predict attributes are the basic method solve this problem. To fully use neural network, paper proposes novel network structure called Raft Block. Block designed not only extract task-specific features, but also share features of different tasks. Using Block, we build an Raftnet for pedestrian recognition. We implement experiments on three public...

Human motion prediction aims to predict future 3D skeletal sequences by giving a limited human as inputs. Two popular methods, recurrent neural networks and feed-forward deep networks, are able rough trend, but details such limb movement may be lost. To more accurate motion, we propose an Adversarial Refinement Network (ARNet) following simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation. Specifically, take both the historical coarse input of our cascaded...

Non-parametric face modeling aims to reconstruct 3D only from images without shape assumptions. While plausible facial details are predicted, the models tend over-depend on local color appearance and suffer ambiguous noise. To address such problem, this paper presents a novel Learning Aggregate Personalize (LAP) framework for unsupervised robust modeling. Instead of using controlled environment, proposed method implicitly disentangles ID-consistent scene-specific unconstrained photo set....