We develop an efficient computational solution to train deep neural networks (DNN) with free-form activation functions. To make the problem well-posed, we augment cost functional of DNN by adding appropriate shape regularization: sum second-order total-variations trainable nonlinearities. The representer theorem for DNNs tells us that optimal functions are adaptive piecewise-linear splines, which allows recast as a parametric optimization. challenging point is corresponding basis (ReLUs)...

We introduce a variational framework to learn the activation functions of deep neural networks. Our aim is increase capacity network while controlling an upper-bound actual Lipschitz constant input-output relation. To that end, we first establish global bound for Based on obtained bound, then formulate problem learning functions. infinite-dimensional and not computationally tractable. However, prove there always exists solution has continuous piecewise-linear (linear-spline) activations....

We study one-dimensional continuous-domain inverse problems with multiple generalized total-variation regularization, which involves the joint use of several regularization operators. Our starting point is a new representer theorem that states such have hybrid-spline solutions total sparsity bounded by number measurements. show can be discretized in an exact way using union B-spline dictionary bases matched to then propose multiresolution algorithm selects appropriate grid size depends on...

Abstract In this paper, we characterize the class of extremal points unit ball Hessian–Schatten total variation (HTV) functional. The underlying motivation for our work stems from a general representer theorem that characterizes solution set regularized linear inverse problems in terms regularization ball. Our analysis is mainly based on studying continuous and piecewise (CPWL) functions. particular, show dimension $$d=2$$ <mml:math...

We characterize the solution of a broad class convex optimization problems that address reconstruction function from finite number linear measurements. The underlying hypothesis is decomposable as sum components, where each component belongs to its own prescribed Banach space; moreover, problem regularized by penalizing some composite norm solution. establish general conditions for existence and derive generic parametric representation components. These representations fall into three...

We characterize the local smoothness and asymptotic growth rate of Lévy white noise. do so by characterizing weighted Besov spaces in which it is located. extend known results two ways. First, we obtain new bounds for via Blumenthal-Getoor indices also deduce critical when coincide, true symmetric-alpha-stable, compound Poisson, symmetric-gamma noises to name a few. Second, express terms moment properties Previous analyses only provided lower both rate. Showing sharpness these requires us...

In this paper, we formally investigate two mathematical aspects of Hermite splines that are relevant to practical applications. We first demonstrate maximally localized, in the sense size their support is minimal among pairs functions with identical reproduction properties. Then, precisely quantify approximation power for reconstruction and derivatives. It known B-spline schemes have same order. More precisely, error vanishes as O(T4) when step T goes zero. work, show they actually...

We characterize the local smoothness and asymptotic growth rate of Lévy white noise. do so by characterizing weighted Besov spaces in which it is located. extend known results two ways. First, we obtain new bounds for via Blumenthal-Getoor indices also deduce critical when coincide, true symmetric-$\alpha $-stable, compound Poisson, symmetric-gamma noises to name a few. Second, express terms moment properties Previous analyses only provided lower both rate. Showing sharpness these requires...

In this paper, we provide a Banach-space formulation of supervised learning with generalized total- variation (gTV) regularization. We identify the class kernel functions that are admissible in framework. Then, propose continuous-domain hybrid search space gTV show solution admits multikernel expansion adaptive positions. representation, number active kernels is upper-bounded by data points while regularization imposes an $\ell_1$ penalty on coefficients. Finally, illustrate numerically...

In this paper, we introduce the Hessian-Schatten total variation (HTV)---a novel seminorm that quantifies ``rugosity"" of multivariate functions.Our motivation for defining HTV is to assess complexity supervised-learning schemes.We start by specifying adequate matrixvalued Banach spaces are equipped with suitable classes mixed norms.We then show invariant rotations, scalings, and translations.Additionally, its minimum value achieved linear mappings, which supports common intuition regression...

We formulate as an inverse problem the construction of sparse parametric continuous curve models that fit a sequence contour points. Our prior is incorporated regularization term encourages rotation invariance and sparsity. prove optimal solution to closed with spline components. then show how efficiently solve task using B-splines basis functions. extend our formulation curves made two distinct components complementary smoothness properties it hybrid splines. illustrate performance model on...

We develop a novel 2D functional learning framework that employs sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of regularizer, we restrict search space to span piecewise-linear box splines shifted lattice. Our formulation infinite-dimensional problem this allows us recast it exactly as finite-dimensional one can be solved using standard methods in convex optimization. Since our is composed continuous and functions, work presents itself an...

In this paper, we fully characterize the duality mapping over space of matrices that are equipped with Schatten norms. Our approach is based on analysis saturation Hölder inequality for We prove in our main result that, p∈(1,∞), real-valued Schatten-p norm a continuous and single-valued function provide an explicit form its computation. For special case p = 1, set-valued; by adding rank constraint, show it can be reduced to Borel-measurable which also closed-form expression.

We provide an algorithm to generate trajectories of sparse stochastic processes that are solutions linear ordinary differential equations driven by Lévy white noises. A recent paper showed these limits in law generalized compound-Poisson processes. Based on this result, we derive off-the-grid generates arbitrarily close approximations the target process. Our method relies a B-spline representation illustrate numerically validity our approach.

The motivation for this work is to improve the performance of deep neural networks through optimization individual activation functions. Since latter results in an infinite- dimensional problem, we resolve ambiguity by searching sparsest and most regular solution sense Lipschitz. To that end, first introduce a bound relates properties pointwise nonlinearities global Lipschitz constant network. By using proposed as regularizer, then derive representer theorem shows optimum configuration...

We investigate the problem of index coding, where a sender transmits distinct packets over shared link to multiple users with side information. The aim is find an encoding scheme (linear combinations) minimize number transmitted packets, while providing each user sufficient amount data for recovery desired parts. It has been shown that finding optimal linear code equivalent matrix completion problem, observed elements indicate information available users. This modeling results in incomplete...

The goal of derivative sampling is to reconstruct a signal from the samples function and its first-order derivative. In this paper, we consider problem over shift-invariant reconstruction subspace generated by two compact-support functions. We assume that reproduces polynomials up certain degree. then derive lower bound on sum supports generators. Finally, illustrate tightness our with some examples.

Beside the minimizationof prediction error, two of most desirable properties a regression scheme are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stability</i> and xmlns:xlink="http://www.w3.org/1999/xlink">interpretability</i> . Driven by these principles, we propose continuous-domain formulations for one-dimensional problems. In our first approach, use Lipschitz constant as regularizer, which results in an implicit tuning overall...

ABSTRACT We propose a novel method for the clustering of point-cloud data that originate from single-molecule localization microscopy (SMLM). Our scheme has ability to infer hierarchical structure data. It takes particular relevance when quantitatively analyzing biological particles interest at different scales. assumes prior neither on shape nor background noise. multiscale pipeline is built upon graph theory. At each scale, we first construct weighted represents SMLM Next, find clusters...

In this paper, we precisely quantify the wavelet compressibility of compound Poisson processes.To that end, expand given random process over Haar basis and analyse its asymptotic approximation properties.By only considering nonzero coefficients up to a scale, what call greedy approximation, exploit extreme sparsity expansion derives from piecewise-constant nature processes.More precisely, provide lower upper bounds for mean squared error processes.We are then able deduce has sub-exponential...

In this paper, we provide a Banach-space formulation of supervised learning with generalized total-variation (gTV) regularization. We identify the class kernel functions that are admissible in framework. Then, propose variation continuous-domain hybrid search space gTV show solution admits multi-kernel expansion adaptive positions. representation, number active kernels is upper-bounded by data points while regularization imposes an $\ell_1$ penalty on coefficients. Finally, illustrate...