Hyperspectral images provide higher spectral resolution than typical RGB by including per-pixel irradiance measurements in a number of narrow bands wavelength the visible spectrum. The additional may be useful for many visual tasks, segmentation, recognition, and relighting. Vision systems that seek to capture exploit hyperspectral data should benefit from statistical models natural images, but at present, relatively little is known about their structure. Using new collection fifty indoor...

Neural Radiance Fields (NeRF) is a popular view synthesis technique that represents scene as continuous volumetric function, parameterized by multilayer perceptrons provide the volume density and view-dependent emitted radiance at each location. While NeRF-based techniques excel representing fine geometric structures with smoothly varying appearance, they often fail to accurately capture reproduce appearance of glossy surfaces. We address this limitation introducing Ref-NeRF, which replaces...

Significance Nature provides diverse solutions to passive visual depth sensing. Evolution has produced vision systems that are highly specialized and efficient, delivering depth-perception capabilities often surpass those of existing artificial sensors. Here, we learn from the eyes jumping spiders demonstrate a metalens sensor shares compactness high computational efficiency its biological counterpart. Our device combines multifunctional metalenses, ultrathin nanophotonic components control...

We present a method for simultaneously recovering shape and spatially varying reflectance of surface from photometric stereo images. The distinguishing feature our approach is its generality; it does not rely on specific parametric model therefore purely ldquodata-drivenrdquo. This achieved by employing novel bi-variate approximations isotropic functions. By combining this new approximation with recent developments in stereo, we are able to estimate an independent normal at each point,...

Blur is caused by a pixel receiving light from multiple scene points, and in many cases, such as object motion, the induced blur varies spatially across image plane. However, seemingly straight-forward task of estimating spatially-varying single has proved hard to accomplish reliably. This work considers makes two contributions: local cue that measures likelihood small neighborhood being blurred candidate kernel; an algorithm that, given image, simultaneously selects motion kernel segments...

We introduce an efficient maximum likelihood approach for one part of the color constancy problem: removing from image cast caused by spectral distribution dominating scene illuminant. do this developing a statistical model spatial colors in white balanced images (i.e., those that have no cast), and then using to infer illumination parameters as being most likely under our model. The key observation is applying band-pass filters unveils distributions are unimodal, symmetric, well represented...

Most personal photos that are shared online embedded in some form of social network, and these networks a potent source contextual information can be leveraged for automatic image understanding. In this paper, we investigate the utility network context task face recognition photographs. We combine scores with conditional random field (CRF) model apply to label faces from popular Facebook, which is now top photo-sharing site on Web billions total. demonstrate our simple method enhancing...

We propose an approach for cross-view action recognition by way of `virtual views' that connect the descriptors extracted from one (source) view to those another (target) view. Each virtual is associated with a linear transformation descriptor, and sequence transformations arising views aims at bridging source target while preserving discrimination among categories. Our capable operating without access labeled samples in corresponding instances two views, it also naturally incorporate...

Translucent materials are ubiquitous, and simulating their appearance requires accurate physical parameters. However, physically-accurate parameters for scattering difficult to acquire. We introduce an optimization framework measuring bulk properties of homogeneous (phase function, coefficient, absorption coefficient) that is more accurate, applicable a broad range materials. The combines stochastic gradient descent with Monte Carlo rendering material dictionary invert the radiative transfer...

Biological visual systems are currently unrivaled by artificial in their ability to recognize faces and objects highly variable cluttered real-world environments. Biologically-inspired computer vision seek capture key aspects of the computational architecture brain, such approaches have proven successful across a range standard object face recognition tasks (e.g. [23, 8, 9, 18]). Here, we explore effectiveness these algorithms on large-scale unconstrained problem based images taken from...

We present a photometric stereo method for non-diffuse materials that does not require an explicit reflectance model or reference object. By computing data-dependent rotation of RGB color space, we show the specular reflection effects can be separated from much simpler, diffuse (approximately Lambertian) surfaces modeled with dichromatic reflectance. Images in this transformed space are used to obtain reconstructions independent In contrast other methods highlight removal based on separation...

We present a novel optical setup and processing pipeline for measuring the 3D geometry spatially-varying surface reflectance of physical objects. Central to our design is digital camera high frequency spatially-modulated light source aligned share common focal point axis. Pairs such devices allow capturing sequence images from which precise measurements can be recovered. Our approach enabled by two technical contributions: new active multiview stereo algorithm an analysis descattering that...

Multiple scattering contributes critically to the characteristic translucent appearance of food, liquids, skin, and crystals; but little is known about how it perceived by human observers. This article explores perception translucency studying image effects variations in one factor multiple scattering: phase function. We consider an expanded space functions created linear combinations Henyey-Greenstein von Mises-Fisher lobes, we study this physical parameter using computational data analysis...

We develop a framework for extracting concise representation of the shape information available from diffuse shading in small image patch. This produces mid-level scene descriptor, comprised local distributions that are inferred separately at every patch across multiple scales. The is based on quadratic that, absence noise, has guarantees recovering accurate and lighting. And when noise present, provide useful without over-committing to any particular explanation. These naturally encode fact...

Abstract Translucency is an important aspect of material appearance. To some extent, humans are able to estimate translucency in a consistent way across different shapes and lighting conditions, i.e., exhibit constancy. However, Fleming Bülthoff (2005) have shown that there can be large failures constancy, with direction playing role. In this paper, we explore the interaction shape, illumination, degree constancy more deeply by including our analysis variations translucent appearance induced...

We present a computational imaging system, inspired by the optical coherence tomography (OCT) framework, that uses interferometry to produce decompositions of light transport in small scenes or volumes. The system decomposes according various attributes paths photons travel through scene, including where on source originate, their pathlengths from camera wavelength, and polarization. Since it interference, can achieve high pathlength resolutions, with ability distinguish whose lengths differ...

Algorithms for scene understanding and realistic image synthesis require accurate models of the way real-world materials scatter light. This study describes recent work in graphics community to measure spatially-and directionally-varying reflectance subsurface scattering complex materials, develop efficient representations analysis tools these datasets. We describe design acquisition devices capture strategies functions such as BRDFs BSSRDFs, factored representations, a case capturing...

Images harvested from the Web are proving to be useful for many visual tasks, including recognition, geo-location, and three-dimensional reconstruction.These images captured under a variety of lighting conditions by consumer-level digital cameras, these cameras have color processing pipelines that diverse, complex, scenedependent.As result, information contained in is difficult exploit.In this paper, we analyze factors contribute output typical camera, explore use parametric models relating...

This paper presents a technique for acquiring the shape of real-world objects with complex isotropic and anisotropic reflectance. Our method estimates local normal tangent vectors at each pixel in reference view from sequence images taken under varying point lighting. We show that many materials restricted set light positions, 2D slice BRDF obtained by fixing direction is symmetric reflections halfway vector across normal-tangent normal-binormal planes. Based on this analysis, we develop an...