A5101655527- Advanced Neural Network Applications
- CCD and CMOS Imaging Sensors
- Advanced Memory and Neural Computing
- Video Analysis and Summarization
- Advanced Image and Video Retrieval Techniques
- Generative Adversarial Networks and Image Synthesis
- Artificial Intelligence in Games
- Brain Tumor Detection and Classification
- Ferroelectric and Negative Capacitance Devices
- Video Surveillance and Tracking Methods
- Advancements in Photolithography Techniques
- Machine Learning and ELM
- Natural Language Processing Techniques
- Anomaly Detection Techniques and Applications
- Domain Adaptation and Few-Shot Learning
- Advanced Image Fusion Techniques
- Human Motion and Animation
- Data Management and Algorithms
- Advanced Measurement and Detection Methods
- Human Pose and Action Recognition
- Advanced Algorithms and Applications
- Topic Modeling
- Image Enhancement Techniques
- Structural Health Monitoring Techniques
- Gambling Behavior and Treatments
Northeastern University
2021-2025
Boston University
2025
Northwest University
2025
Universidad del Noreste
2021-2024
Yanshan University
2023
China University of Petroleum, Beijing
2021
Jilin University
2019
Southeast University
2017
Semiconductor Manufacturing International (China)
2016
Nanjing University of Posts and Telecommunications
2014
The rapid advances in Vision Transformer (ViT) refresh the state-of-the-art performances various vision tasks, overshadowing conventional CNN-based models. This ignites a few recent striking-back research CNN world showing that pure models can achieve as good performance ViT when carefully tuned. While encouraging, designing such high-performance is challenging, requiring non-trivial prior knowledge of network design. To this end, novel framework termed Mathematical Architecture Design for...
Large Language Models (LLMs) stand out for their impressive performance in intricate language modeling tasks. However, demanding computational and memory needs pose obstacles broad use on edge devices. Quantization is then introduced to boost LLMs' on-device efficiency. Recent works show that 8-bit or lower weight quantization feasible with minimal impact end-to-end task performance, while the activation still not quantized. On other hand, mainstream commodity devices struggle execute these...
Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication-the intensive and key computation in deep neural networks (DNNs). However, hardware failure, such stuck-at-fault defects, is one main concerns that impedes ReRAM devices be a feasible solution for real implementations. The existing solutions address this issue usually require optimization conducted...
Coordinated gene transcription in plastid and nucleus is essential for the photosynthetic apparatus assembly during chloroplast biogenesis. Despite identification of several factors regulating nuclear-encoded genes,no factor has been discovered. Here we report that BAI1 ("albino" Chinese), a nucleus-plastid dual-targeted C2H2-type zinc finger Arabidopsis, positively regulates orchestrates nuclear genes. The knockout leads to blockage formation, albino seedling, lethality. In plastid, newly...
Structured pruning for large language models (LLMs) has garnered significant academic interest due to its ability efficiently compress and accelerate LLMs by eliminating redundant weight groups at a coarse-grained granularity. Current structured methods typically depend on singular granularity assessing importance, resulting in notable performance degradation downstream tasks. Intriguingly, our empirical investigations reveal that utilizing unstructured pruning, which achieves better...
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important reduce unnecessary computation and increase execution speed. Prior methods towards this goal, including model compression network architecture search (NAS), are largely performed independently, do not fully consider compiler-level optimizations which is a must-do for acceleration. In work, we first propose (i) general category of fine-grained structured pruning applicable various DNN...
Fine-tuning helps large language models (LLM) recover degraded information and enhance task performance.Although Low-Rank Adaptation (LoRA) is widely used effective for fine-tuning, we have observed that its scaling factor can limit or even reduce performance as the rank size increases. To address this issue, propose RoRA (Rank-adaptive Reliability Optimization), a simple yet method optimizing LoRA's factor. By replacing $\alpha/r$ with $\alpha/\sqrt{r}$, ensures improved Moreover, enhances...
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training inference time limit generalization. Previous compression algorithms usually start from the pre-trained dense models only focus on efficient inference, while time-consuming is still unavoidable. In contrast, this paper points out that million-scale data redundant, which fundamental reason for tedious training. To address issue, aims to...
There have been long-standing controversies and inconsistencies over the experiment setup criteria for identifying "winning ticket" in literature. To reconcile such, we revisit definition of lottery ticket hypothesis, with comprehensive more rigorous conditions. Under our new definition, show concrete evidence to clarify whether winning exists across major DNN architectures and/or applications. Through extensive experiments, perform quantitative analysis on correlations between tickets...
Despite the remarkable strides of Large Language Models (LLMs) in various fields, wide applications LLMs on edge devices are limited due to their massive parameters and computations. To address this, quantization is commonly adopted generate lightweight with efficient computations fast inference. However, Post-Training Quantization (PTQ) methods dramatically degrade quality when quantizing weights, activations, KV cache together below 8 bits. Besides, many Quantization-Aware Training (QAT)...
The conventional lottery ticket hypothesis (LTH) claims that there exists a sparse subnetwork within dense neural network and proper random initialization method, called the winning ticket, such it can be trained from scratch to almost as good counterpart. Meanwhile, research of LTH in vision transformers (ViTs) is scarcely evaluated. In this paper, we first show hard find at weight level ViTs by existing methods. Then, generalize for input data consisting image patches inspired dependence...
In the production of automotive gauge, one most important step is to test meters on dashboard some which are needle meters. Based computer vision technology, a dynamic automatic reading value method presented in paper. With hardware loop (HIL) platform simulating actual vehicle running state and providing sensor signals or data via CAN(controller area network) bus for tested dashboard, by comparing input indication obtained through image process method. As core technology method, how read...
The rapid development of autonomous driving, abnormal behavior detection, and recognition makes an increasing demand for multi-person pose estimation-based applications, especially on mobile platforms. However, to achieve high accuracy, state-of-the-art methods tend have a large model size complex post-processing algorithm, which costs intense computation long end-to-end latency. To solve this problem, we propose architecture optimization weight pruning framework accelerate inference...
As data privacy is gradually valued by people, federated learning(FL) has emerged because of its potential to protect data. FL uses homomorphic encryption and differential on the promise ensuring security realize distributed machine learning exchanging encrypted information between different providers. However, there are still many problems in FL, such as communication efficiency client server non-iid. In order solve two mentioned above, we propose a novel vertical framework based DFP...
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., properly pruned sub-network together with original weight initialization) can achieve competitive performance than dense network. However, it is not easy to observe such property in many scenarios, where for example, relatively large learning rate used even if benefits training model. this work, we investigate underlying condition...
Despite the superior performance, it is challenging to deploy foundation models or large language (LLMs) due their massive parameters and computations. While pruning a promising technique reduce model size accelerate inference, traditional techniques can hardly be applied for LLMs as they need finetune on full dataset with multiple epochs consuming data hardware resources. To deal this problem, post-training methods are proposed prune in one-shot without retraining. However, accuracy after...