A5012826227- Neural Networks and Applications
- Stochastic Gradient Optimization Techniques
- Machine Learning and ELM
- Sparse and Compressive Sensing Techniques
- Blind Source Separation Techniques
- Algebraic and Geometric Analysis
- Mathematical Analysis and Transform Methods
- Face and Expression Recognition
- Matrix Theory and Algorithms
- Advanced Adaptive Filtering Techniques
- Digital Filter Design and Implementation
- Advanced Neural Network Applications
- Image and Signal Denoising Methods
- Indoor and Outdoor Localization Technologies
- Neural Networks Stability and Synchronization
- Statistical and numerical algorithms
- Control Systems and Identification
- Speech and Audio Processing
- Gaussian Processes and Bayesian Inference
- Advanced Algorithms and Applications
- Inertial Sensor and Navigation
- Advanced Bandit Algorithms Research
- Optimization and Variational Analysis
- Distributed Control Multi-Agent Systems
- Markov Chains and Monte Carlo Methods
Northeast Normal University
2015-2025
Harbin Engineering University
2010-2015
Imperial College London
2014-2015
Harbin University
2010
Dalian University of Technology
2007-2009
University of Pennsylvania
2002
The optimization of real scalar functions quaternion variables, such as the mean square error or array output power, underpins many practical applications. Solutions typically require calculation gradient and Hessian. However, variables are essentially nonanalytic, which prohibitive to development quaternion-valued learning systems. To address this issue, we propose new definitions Hessian, based on novel generalized Hamilton-real (GHR) calculus, thus making a possible efficient derivation...
A systematic theory is introduced for calculating the derivatives of quaternion matrix function with respect to variables. The proposed methodology equipped product rule and chain it able handle both analytic nonanalytic functions. This corrects a flaw in existing methods, that is, incorrect use traditional rule. In framework introduced, functions can be calculated directly without differential this function. Key results are summarized tables. Several examples show how used as an important...
Quaternion derivatives exist only for a very restricted class of analytic (regular) functions; however, in many applications, functions interest are real-valued and hence not analytic, typical case being the standard real mean square error objective function. The recent HR calculus is step forward provides way to calculate gradients both non-analytic quaternion variables; can become cumbersome complex optimization problems due lack rigorous product chain rules, consequence non-commutativity...
In this article, we investigate the boundedness and convergence of online gradient method with smoothing group <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1/2}$</tex-math> </inline-formula> regularization for sigma-pi-sigma neural network (SPSNN). This enhances sparseness improves its generalization ability. For original regularization, error function is nonconvex nonsmooth, which can cause...
This letter presents a unified convergence analysis of the split-complex nonlinear gradient descent (SCNGD) learning algorithms for complex-valued recurrent neural networks, covering three classes SCNGD algorithms: standard SCNGD, normalized and adaptive SCNGD. We prove that if activation functions are type some conditions satisfied, error function is monotonically decreasing during training iteration process, gradients with respect to real imaginary parts weights converge zero. A strong...
Adam-type algorithms have become a preferred choice for optimization in the deep learning setting; however, despite their success, convergence is still not well understood. To this end, we introduce unified framework algorithms, termed UAdam. It equipped with general form of second-order moment, which makes it possible to include Adam and its existing future variants as special cases, such NAdam, AMSGrad, AdaBound, AdaFom, Adan. The approach supported by rigorous analysis UAdam nonconvex...