Recent research work has developed powerful generative models (e.g., StyleGAN2) that can synthesize complete human head images with impressive photorealism, enabling applications such as photorealistically editing real photographs. While these be trained on large collections of unposed images, their lack explicit 3D knowledge makes it difficult to achieve even basic control over viewpoint without unintentionally altering identity. On the other hand, recent Neural Radiance Field (NeRF)...

Facial landmark detection is a fundamental task for many consumer and high-end applications almost entirely solved by machine learning methods today. Existing datasets used to train such algorithms are primarily made up of only low resolution images, current limited inputs comparable quality as the training dataset. On other hand, high imagery becoming increasingly more common cameras improve in every year. Therefore, there need that can leverage rich information available imagery. Naively...

Style transfer between images is an artistic application of CNNs, where the ‘style’ one image transferred onto another while preserving latter’s content. The state art in neural style based on Adaptive Instance Normalization (AdaIN), a technique that transfers statistical properties features to content image, and can large number styles real time. However, AdaIN global operation; thus local geometric structures are often ignored during transfer. We propose Convolutions (AdaConv), generic...

2D portrait animation has experienced significant advancements in recent years. Much research utilized the prior knowledge embedded large generative diffusion models to enhance high-quality image manipulation. However, most methods only focus on generating RGB images as output, and co-generation of consistent visual plus 3D output remains largely under-explored. In our work, we propose jointly learn appearance depth simultaneously a diffusion-based generator. Our method embraces end-to-end...

Face models built from 3D face databases are often used in computer vision and graphics tasks such as reconstruction, replacement, tracking manipulation. For tasks, commonly multi-linear morphable models, which provide semantic control over facial identity expression, lack quality expressivity due to their linear nature. Deep neural networks offer the possibility of non-linear modeling, where so far most research has focused on generating realistic images with less focus geometry, methods...

Generating realistic facial animation for CG characters and digital doubles is one of the hardest tasks in animation. A typical production workflow involves capturing performance a real actor using mo-cap technology, transferring captured motion to target character. This process, known as retargeting , has been used over decade, typically relies on either large blendshape rigs that are expensive create, or direct deformation transfer algorithms operate individual geometric elements prone...

Neural networks for facial landmark detection are notoriously limited to a fixed set of landmarks in dedicated layout, which must be specified at training time. Dedicated datasets also hand-annotated with the corresponding configuration training. We propose first network that can predict continuous, unlimited landmarks, allowing specify number and location desired inference Our method combines simple image feature extractor queried predictor, user any continuous query points relative 3D...

Recent work on radiance fields and volumetric inverse rendering (e.g., NeRFs) has provided excellent results in building data-driven models of real scenes for novel view synthesis with high photorealism. While full control over viewpoint is achieved, scene lighting typically "baked" into the model cannot be changed; other methods only capture limited variation or make restrictive assumptions about captured scene. These limitations prevent application arbitrary materials 3D environments...

For several decades, researchers have been advancing techniques for creating and rendering 3D digital faces, where a lot of the effort has gone into geometry appearance capture, modeling techniques. This body research work largely focused on facial skin, with much less attention devoted to peripheral components like hair, eyes interior mouth. As result, even best technology capture rendering, in most high-end productions artist time is still spent missing fine-tuning parameters combine...

Abstract Parametric 3D shape models are heavily utilized in computer graphics and vision applications to provide priors on the observed variability of an object's geometry ( e.g ., for faces). Original were linear operated entire at once. They later enhanced localized control different parts separately. In deep models, nonlinearity was introduced via a sequence fully‐connected layers activation functions, locality recent that use mesh convolution networks. As common limitations, these often...

Photorealistic digital re-aging of faces in video is becoming increasingly common entertainment and advertising. But the predominant 2D painting workflow often requires frame-by-frame manual work that can take days to accomplish, even by skilled artists. Although research on facial image has attempted automate solve this problem, current techniques are little practical use as they typically suffer from identity loss, poor resolution, unstable results across subsequent frames. In paper, we...

Facial hair is a largely overlooked topic in facial performance capture. Most production pipelines the entertainment industry do not have way to automatically capture or track skin underneath it. Thus, actors are asked shave clean before face capture, which very often undesirable. Capturing geometry of individual hairs challenging, and their presence makes it harder deforming shape underlying surface. Some attempts already been made at automating this task, but only for static faces with...

Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond external forces and perform realistic anatomy edits. Today's methods data-driven, where actuations finite elements inferred from captured skin geometry. Unfortunately, these approaches have not been widely adopted due complexity of initializing material space learning deformation model each character...

3D facial animation is often produced by manipulating deformation models (or rigs), that are traditionally parameterized expression controls. A key component usually overlooked 'style', as in, how a particular performed. Although it common to define semantic basis of expressions characters can perform, most perform each in their own style. To date, style entangled with the expression, and not possible transfer one character another when considering animation. We present new face model, based...

Shaped-offset quadrature phase shift keying (SOQPSK) is a highly bandwidth efficient modulation technique used widely in military and aeronautical telemetry standards. It can be classified as form of continuous (CPM), but its major distinction from other CPMs that it has constrained (correlated) ternary data alphabet. CPM-based detection models for SOQPSK have been developed only recently. While these detectors offer an appreciable performance gain over current schemes, one roadblock...

Abstract We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non‐rigid motion during performance. Our work extends neural morphable models by learning manifold using transformer architecture. More specifically, we derive novel transformer‐based autoencoder that can model and synthesize geometry arbitrary length. This naturally determines frame‐to‐frame correlations required to represent the manifold, via internal self‐attention mechanism....

We consider symbol timing recovery for continuous phase modulations (CPMs) with correlated data symbols. A popular example of such a scheme is shaped offset quadrature phase-shift keying (SOQPSK). propose an extension to existing non-data-aided (blind) error detector (TED) make it compatible modulation schemes. The merits the modified TED are demonstrated by comparing its performance and without taking correlation into account. As further modification, we show that quantization can be used...

Abstract We present a novel graph‐based simulation approach for generating micro wrinkle geometry on human skin, which can easily scale up to the micro‐meter range and millions of wrinkles. The first samples pores skin treats them as nodes in graph. These are then connected resulting edges become candidate An iterative optimization inspired by pedestrian trail formation is used assign weights those edges, i.e., carve out Finally, we convert graph detailed displacement map using shape...

Abstract Monocular 3D face reconstruction is a wide‐spread topic, and existing approaches tackle the problem either through fast neural network inference or offline iterative of geometry. In case carefully‐designed energy functions are minimized, commonly including loss terms like photometric loss, landmark reprojection others. this work we propose new function for monocular capture, inspired by how humans would perceive quality given particular image. It widely known that shading provides...

Road segmentation and tracking is of prime importance in Advanced Driver Assistance Systems (ADAS) to either assist autonomous navigation or provide useful information drivers operating semi-autonomous vehicles. The work reported herein describes a novel algorithm based on particle filters for segmenting the edges roads real world scenarios. This accomplished with help video camera mounted vehicle. measurement prediction functions filtering are modified suitably measure track road time. One...

Abstract Face swapping is the process of applying a source actor's appearance to target performance in video. This challenging visual effect that has seen increasing demand film and television production. Recent work shown data‐driven methods based on deep learning can produce compelling effects at production quality fraction time required for traditional 3D pipeline. However, dominant approach operates only 2D imagery without reference underlying facial geometry or texture, resulting poor...

Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond external forces and perform realistic anatomy edits. Today's methods data-driven, where actuations finite elements inferred from captured skin geometry. Unfortunately, these approaches have not been widely adopted due complexity of initializing material space learning deformation model each character...