Weight and activation binarization is an effective approach to deep neural network compression can accelerate the inference by leveraging bitwise operations. Although many methods have improved accuracy of model minimizing quantization error in forward propagation, there remains a noticeable performance gap between binarized full-precision one. Our empirical study indicates that brings information loss both backward which bottleneck training accurate binary networks. To address these issues,...

Recently, perception task based on Bird's-Eye View (BEV) representation has drawn more and attention, BEV is promising as the foundation for next-generation Autonomous Vehicle (AV) perception. However, most existing solutions either require considerable resources to execute on-vehicle inference or suffer from modest performance. This paper proposes a simple yet effective framework, termed Fast-BEV, which capable of performing faster chips. Towards this goal, we first empirically find that...

Binary neural networks have attracted tremendous attention due to the efficiency for deploying them on mobile devices. Since weak expression ability of binary weights and features, their accuracy is usually much lower than that full-precision (i.e. 32-bit) models. Here we present a new frame work automatically searching compact but accurate networks. In practice, number channels in each layer will be encoded into search space optimized using evolutionary algorithm. Experiments conducted...

Biochar is a kind of organic matter that can be added into soil to improve quality. To study the effect biochar combined with and inorganic fertilizers on rapeseed growth purple fertility microbial community, completely randomized block design was designed three levels (B0: no biochar, B1: low-rate B2: high-rate biochar); two (F1: fertilizer; F2: fertilizer); (M1: M2: fertilizer). All combinations were repeated times. The application could pH, community richness: pH B1F2M1 increased 0.41...

Binary neural networks have attracted numerous attention in recent years. However, mainly due to the information loss stemming from biased binarization, how preserve accuracy of still remains a critical issue. In this paper, we attempt maintain propagated forward process and propose Balanced Neural Networks with Gated Residual (BBG for short). First, weight balanced binarization is introduced thus informative binary weights can capture more contained activations. Second, activations, gated...

Quantization Neural Networks (QNN) have attracted a lot of attention due to their high efficiency. To enhance the quantization accuracy, prior works mainly focus on designing advanced algorithms but still fail achieve satisfactory results under extremely low-bit case. In this work, we take an architecture perspective investigate potential high-performance QNN. Therefore, propose combine Network Architecture Search methods with enjoy merits two sides. However, naive combination inevitably...

Low-Rank Adaptation (LoRA) has emerged as a widely adopted technique in text-to-image models, enabling precise rendering of multiple distinct elements, such characters and styles, multi-concept image generation. However, current approaches face significant challenges when composing these LoRAs for generation, resulting diminished generated quality. In this paper, we initially investigate the role denoising process through lens Fourier frequency domain. Based on hypothesis that applying could...

User data confidentiality protection is becoming a rising challenge in the present deep learning research. Without access to data, conventional data-driven model compression faces higher risk of performance degradation. Recently, some works propose generate images from specific pretrained serve as training data. However, inversion process only utilizes biased feature statistics stored one and low-dimension high-dimension. As consequence, it inevitably encounters difficulties generalizability...

Model quantization has emerged as an indispensable technique to accelerate deep learning inference. While researchers continue push the frontier of algorithms, existing work is often unreproducible and undeployable. This because do not choose consistent training pipelines ignore requirements for hardware deployments. In this work, we propose Quantization Benchmark (MQBench), a first attempt evaluate, analyze, benchmark reproducibility deployability model algorithms. We multiple different...

Recently, the pure camera-based Bird's-Eye-View (BEV) perception removes expensive Lidar sensors, making it a feasible solution for economical autonomous driving. However, most existing BEV solutions either suffer from modest performance or require considerable resources to execute on-vehicle inference. This paper proposes simple yet effective framework, termed Fast-BEV, which is capable of performing real-time on chips. Towards this goal, we first empirically find that representation can be...

Recently, perception task based on Bird's-Eye View (BEV) representation has drawn more and attention, BEV is promising as the foundation for next-generation Autonomous Vehicle (AV) perception. However, most existing solutions either require considerable resources to execute on-vehicle inference or suffer from modest performance. This paper proposes a simple yet effective framework, termed Fast-BEV , which capable of performing faster chips. Towards this goal, we first empirically find that...

Deep neural networks (DNNs) are widely used in various applications. The accurate and latency feedback is essential for model design deployment. In this work, we attempt to alleviate the cost of acquisition from two aspects: query prediction. To ease difficulty acquiring on multi-platform, our system can automatically convert DNN into corresponding executable format, measure target hardware. Powered by this, queries be fulfilled with a simple interface calling. For efficient utilization...

Diffusion models are widely recognized for generating high-quality and diverse images, but their poor real-time performance has led to numerous acceleration works, primarily focusing on UNet-based structures. With the more successful results achieved by diffusion transformers (DiT), there is still a lack of exploration regarding impact DiT structure generation, as well absence an framework tailored architecture. To tackle these challenges, we conduct investigation into correlation between...

Binary neural networks have attracted tremendous attention due to the efficiency for deploying them on mobile devices. Since weak expression ability of binary weights and features, their accuracy is usually much lower than that full-precision (i.e. 32-bit) models. Here we present a new frame work automatically searching compact but accurate networks. In practice, number channels in each layer will be encoded into search space optimized using evolutionary algorithm. Experiments conducted...

Binary neural networks have attracted numerous attention in recent years. However, mainly due to the information loss stemming from biased binarization, how preserve accuracy of still remains a critical issue. In this paper, we attempt maintain propagated forward process and propose Balanced Neural Networks with Gated Residual (BBG for short). First, weight balanced binarization is introduced maximize entropy binary weights, thus informative weights can capture more contained activations....

Weight and activation binarization is an effective approach to deep neural network compression can accelerate the inference by leveraging bitwise operations. Although many methods have improved accuracy of model minimizing quantization error in forward propagation, there remains a noticeable performance gap between binarized full-precision one. Our empirical study indicates that brings information loss both backward which bottleneck training accurate binary networks. To address these issues,...

Abstract Background Biochar is one kind of organic matter that can be added into soil as a amendment to improve its quality. To study the effect biochar addition combined with and inorganic fertilizers on growth fertility microbial community in purple soil, completely randomized block design was designed three levels [B0: no biochar, B1: low-rate (35 t/ha) , B2: high-rate (50 t/ha)]; two [F1: fertilizer (30 kg/ha N, 87.5 P 2 O 5 60 K O); F2: (60 175 120 O)]; [M1: fertilizer; M2: (4.5 t/ha)]....

Graphs are widely used to encapsulate a variety of data formats, but real-world networks often involve complex node relations beyond only being pairwise. While hypergraphs and hierarchical graphs have been developed employed account for the relations, they cannot fully represent these complexities in practice. Additionally, though many Graph Neural Networks (GNNs) proposed representation learning on higher-order graphs, usually evaluated simple graph datasets. Therefore, there is need...

This work targets to merge various Vision Transformers (ViTs) trained on different tasks (i.e., datasets with object categories) or domains the same categories but environments) into one unified model, yielding still good performance each task domain. Previous model merging works focus either CNNs NLP models, leaving ViTs research untouched. To fill this gap, we first explore and find that existing methods cannot well handle of whole ViT models have improvement space. enable ViT, propose a...

User data confidentiality protection is becoming a rising challenge in the present deep learning research. Without access to data, conventional data-driven model compression faces higher risk of performance degradation. Recently, some works propose generate images from specific pretrained serve as training data. However, inversion process only utilizes biased feature statistics stored one and low-dimension high-dimension. As consequence, it inevitably encounters difficulties generalizability...