Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper wider architectures. In this work, we interpret as ordinary differential equations (ODEs), which long studied mathematics physics rich theoretical empirical success. From interpretation, develop a framework on stability reversibility of neural networks, derive three reversible network architectures that can go...

Inspired by the observation that humans are able to process videos efficiently only paying attention where and when it is needed, we propose an interpretable easy plug-in spatial-temporal mechanism for video action recognition. For spatial attention, learn a saliency mask allow model focus on most salient parts of feature maps. temporal employ convolutional LSTM based identify relevant frames from input video. Further, set regularizers ensure our attends coherent regions in space time. Our...

Recurrent neural networks have gained widespread use in modeling sequential data. Learning long-term dependencies using these models remains difficult though, due to exploding or vanishing gradients. In this paper, we draw connections between recurrent and ordinary differential equations. A special form of called the AntisymmetricRNN is proposed under theoretical framework, which able capture thanks stability property its underlying equation. Existing approaches improving RNN trainability...

Deep residual networks (ResNets) and their variants are widely used in many computer vision applications natural language processing tasks. However, the theoretical principles for designing training ResNets still not fully understood. Recently, several points of view have emerged to try interpret ResNet theoretically, such as unraveled view, unrolled iterative estimation dynamical systems view. In this paper, we adopt point analyze lesioning properties both theoretically experimentally....

Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper wider architectures. In this work, we interpret as ordinary differential equations (ODEs), which long studied mathematics physics rich theoretical empirical success. From interpretation, develop a framework on stability reversibility of neural networks, derive three reversible network architectures that can go...

Handwriting of Chinese has long been an important skill in East Asia. However, automatic generation handwritten characters poses a great challenge due to the large number characters. Various machine learning techniques have used recognize characters, but few works studied character problem, especially with unpaired training data. In this work, we formulate as problem that learns mapping from existing printed font personalized style. We further propose DenseNet CycleGAN generate Our method is...

Reinforcement Learning (RL) techniques have been sought after as the next-generation tools to further advance field of recommendation research. Different from classic applications RL, recommender agents, especially those deployed on commercial platforms, operate in extremely large state and action spaces, serving a dynamic user base order billions, long-tail item corpus millions or billions. The (positive) feedback available train such agents is scarce retrospect. Improving sample efficiency...

A convolutional neural network can be constructed using numerical methods for solving dynamical systems, since the forward pass of regarded as a trajectory system. However, existing models based on solvers cannot avoid iterations implicit methods, which makes inefficient at inference time. In this paper, we reinterpret pre-activation Residual Networks (ResNets) and their variants from systems view. We consider that Runge-Kutta are fused into training these models. Moreover, propose novel...

Training recurrent neural networks (RNNs) on long sequence tasks is plagued with difficulties arising from the exponential explosion or vanishing of signals as they propagate forward backward through network. Many techniques have been proposed to ameliorate these issues, including various algorithmic and architectural modifications. Two most successful RNN architectures, LSTM GRU, do exhibit modest improvements over vanilla cells, but still suffer instabilities when trained very sequences....

Sequential recommender models are essential components of modern industrial systems. These learn to predict the next items a user is likely interact with based on his/her interaction history platform. Most sequential recommenders however lack higher-level understanding intents, which often drive behaviors online. Intent modeling thus critical for users and optimizing long-term experience. We propose probabilistic approach formulate intent as latent variables, inferred behavior signals using...

Recommender systems are essential for finding personalized content users on online platforms. These often trained historical user interaction data, which collects feedback system recommendations. This creates a loop leading to popularity bias; popular is over-represented in the better learned, and thus recommended even more. Less struggles reach its potential audiences. Popularity bias limits diversity of that exposed to, makes it harder new creators gain traction. Existing methods alleviate...

By using the concepts of contingent epiderivative, radial Clarke tangent epiderivative and Y-epiderivative, we present necessary sufficient conditions for weakly efficient solution, Henig globally proper respectively, to vector equilibrium problems with constraints.

Normalizing flows transform a simple base distribution into complex target and have proved to be powerful models for data generation density estimation. In this work, we propose novel type of normalizing flow driven by differential deformation the Wiener process. As result, obtain rich time series model whose observable process inherits many appealing properties its process, such as efficient computation likelihoods marginals. Furthermore, our continuous treatment provides natural framework...

Existing methods for multi-domain image-to-image translation (or generation) attempt to directly map an input image a random vector) in one of the output domains. However, most existing have limited scalability and robustness, since they require building independent models each pair domains question. This leads two significant shortcomings: (1) need train exponential number pairwise models, (2) inability leverage data from other when training particular mapping. Inspired by recent work on...

Regular vines or vine copula provide a rich class of multivariate densities with arbitrary one dimensional margins and Gaussian non-Gaussian dependence structures. The density enables calculation all conditional distributions, in particular, regression functions for any subset variables on disjoint set can be computed, either analytically by simulation. used to fit smooth non-discrete data. epicycles — including/excluding covariates, interactions, higher order terms, multi collinearity,...

Users who come to recommendation platforms are heterogeneous in activity levels. There usually exists a group of core users visit the platform regularly and consume large body content upon each visit, while others casual tend occasionally less time. As result, consumption activities from often dominate training data used for learning. can exhibit different patterns users, recommender systems trained on historical user achieve much worse performance than users. To bridge gap, we propose...