Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. However, existing work of federated mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular many applications. Graph (GCN) been proposed as one the most promising techniques for learning, but setting seldom explored. In this article, we propose FedGraph among multiple computing clients, each holds a subgraph....

Transformers have dominated the field of natural language processing, attributed to their capability handle sequential input data. There is a surge work on computational and networking optimizations, aimed at improving training efficiency Transformers. However, transformer inference, cornerstone myriad AI services, remains relatively underexplored. With challenge variable-length inputs, conventional methods adopt padding schemes, resulting in waste. Moreover, works inference often overlook...

As a rising star of social apps, short video e.g., TikTok, have attracted large number mobile users by providing fresh and contents that highly match their watching preferences. Meanwhile, the booming growth apps imposes new technical challenges on existing computation communication infrastructure. Traditional solutions maintain all videos cloud stream them to via contend delivery networks or Internet. However, they incur huge network traffic long delay seriously affects users’ experiences....

Transformer has dominated the field of natural language processing because its strong capability in learning from sequential input data. In recent years, various computing and networking optimizations have been proposed for improving transformer training efficiency. However, inference, as core many AI services, seldom studied. A key challenge inference is variable-length input. order to align these input, existing work batching schemes by padding zeros, which unfortunately introduces...

Cross-silo federated learning becomes popular in various fields due to its great promises protecting training data. By carefully examining the interaction among distributed nodes, we find that existing still suffers from security weakness and network bottleneck during model synchronization. It has no protection on models, which also contain significant private information. In addition, many evidences have shown synchronization over wide-area is slow, bottlenecking whole process. To fill this...

Distributed machine learning (DML) has shown great promise in accelerating model training on multiple GPUs. To increase GPU utilization, a common practice is to let jobs share clusters, where the most fundamental and critical challenge how efficiently schedule these However, existing works about DML job scheduling are constrained settings with homogeneous heterogeneity practice, but its influence been seldom studied. Moreover, have internal structures that contain parallelism potentials,...

Dynamic Graph Neural Network (DGNN) has shown a strong capability of learning dynamic graphs by exploiting both spatial and temporal features. Although DGNN recently received considerable attention AI community various models have been proposed, building distributed system for efficient training is still challenging. It well recognized that how to partition the graph assign workloads multiple GPUs plays critical role in acceleration. Existing works into snapshots or sequences, which only...

Federated Learning (FL) has emerged as a promising learning approch for data distributed across edge devices. Existing research mainly focuses on single-job FL systems. However, in practical scenarios, multiple jobs are often submitted simultaneously. Simply applying optimizations to multi-job systems results sub-optimal system performance. Specifically, we find considerably low resource utilization the client side due device heterogeneity. In this paper, exploit opportunities improve by...

Giant models, characterized by their billions or even trillions of parameters, has demonstrated unprecedented capabilities in handling complex tasks on Artificial intelligence (AI)-driven UAVs, such as disaster relief, aerial navigation, and manipulation. However, there is an open challenge about the mismatching between massive computation memory requirements giant models limited resources UAVs. Existing works either pose privacy concerns with offloading methods compromise model accuracy...

Mixture-of-Experts (MoE) is an emerging technique for scaling large models with sparse activation. MoE are typically trained in a distributed manner expert parallelism scheme, where experts each layer across multiple GPUs. However, the default suffers from heavy network burden due to all-to-all intermediate data exchange among GPUs before and after run. Some existing works have proposed reduce exchanges by transferring loads, however, which would decrease level of execution make computation...

Dynamic Graph Neural Network (DGNN) has shown a strong capability of learning dynamic graphs by exploiting both spatial and temporal features. Although DGNN recently received considerable attention AI community various models have been proposed, building distributed system for efficient training is still challenging. It well recognized that how to partition the graph assign workloads multiple GPUs plays critical role in acceleration. Existing works into snapshots or sequences, which only...

Connected autonomous vehicles (CAVs) are promising to improve road safety, thanks various on-board sensors, such as LiDAR, radars, and stereo cameras. However, perception view could be significantly limited due occlusions, extreme weather, far objects. To address these challenges, in this paper, we propose an efficient edge-assisted sharing scheme, which enables exchange the information about their sensed environment safety. We formulate online optimization problem, with objective of...

The advent of deep learning applications for data collection has elicited concerns pertaining to privacy, particularly regarding the potential exposure through various vulnerabilities, such as membership inference attacks. In response these concerns, several machine unlearning techniques have been proposed, which can effectively eliminate specific from a trained model. However, it is important note that existing methods primarily concentrate on Euclidean data, leaving non-Euclidean...

Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. However, existing work of federated mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular many applications. Graph (GCN) been proposed as one the most promising techniques for learning, but setting seldom explored. In this paper, we propose FedGraph among multiple computing clients, each holds a subgraph. provides...

In the past several years, pervasive surveillance cameras have generated massive video records continually, and can be used for applications (e.g., tracking object detection). Machine Learning (ML), especially Deep Learning, is a powerful method extensively analytics. Typically, do not enough computational power to analyze videos locally. A commonly strategy gathering from processing them in cloud or edge server. However, limited bandwidth between server results inefficiency of existing...