A5012467907- Advanced Neural Network Applications
- Age of Information Optimization
- IoT and Edge/Fog Computing
- Air Quality Monitoring and Forecasting
- Insurance, Mortality, Demography, Risk Management
- Stochastic Gradient Optimization Techniques
- Explainable Artificial Intelligence (XAI)
- Machine Learning and ELM
- Fire effects on ecosystems
- Ferroelectric and Negative Capacitance Devices
- Adversarial Robustness in Machine Learning
- Topic Modeling
- Computational Physics and Python Applications
- Smart Parking Systems Research
- Machine Learning and Data Classification
- Parallel Computing and Optimization Techniques
- Fire dynamics and safety research
- Fire Detection and Safety Systems
- Natural Language Processing Techniques
University of Nevada, Reno
2019-2022
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such Mixtral 8x7B GPT-3.5 (e.g., phi-3-mini achieves 69% MMLU 8.38 MT-bench), despite being small enough to be deployed phone. The innovation lies entirely in our dataset for training, scaled-up version the one used phi-2, composed heavily filtered web data synthetic data. is also further...
Deep-learning (DL)-based object detection algorithms can greatly benefit the community at large in fighting fires, advancing climate intelligence, and reducing health complications caused by hazardous smoke particles. Existing DL-based techniques, which are mostly based on convolutional networks, have proven to be effective wildfire detection. However, there is still room for improvement. First, existing methods tend some commercial aspects, with limited publicly available data models. In...
Zero Redundancy Optimizer (ZeRO) has been used to train a wide range of large language models on massive GPUs clusters due its ease use, efficiency, and good scalability. However, when training low-bandwidth clusters, or at scale which forces batch size per GPU be small, ZeRO's effective throughput is limited because high communication volume from gathering weights in forward pass, backward averaging gradients. This paper introduces three reduction techniques, we collectively refer as...
ChatGPT-like models have revolutionized various applications in artificial intelligence, from summarization and coding to translation, matching or even surpassing human performance. However, the current landscape lacks an accessible, efficient, cost-effective end-to-end RLHF (Reinforcement Learning with Human Feedback) training pipeline for these powerful models, particularly when at scale of billions parameters. This paper introduces DeepSpeed-Chat, a novel system that democratizes...
The deployment and scaling of large language models (LLMs) have become critical as they permeate various applications, demanding high-throughput low-latency serving systems. Existing frameworks struggle to balance these requirements, especially for workloads with long prompts. This paper introduces DeepSpeed-FastGen, a system that employs Dynamic SplitFuse, novel prompt generation composition strategy, deliver up 2.3x higher effective throughput, 2x lower latency on average, 3.7x...
The success of machine learning has prospered Machine-Learning-as-a-Service (MLaaS) - deploying trained (ML) models in cloud to provide low latency inference services at scale. To meet Service-Level-Objective (SLO), judicious parallelization both request and operation levels is utterly important. However, existing ML systems (e.g., Tensorflow) serving platforms SageMaker) are SLO-agnostic rely on users manually configure the parallelism. serving, this paper proposes a swift scheduling...
In this paper, we analyze the impact of information freshness on supervised learning based forecasting. these applications, a neural network is trained to predict time-varying target (e.g., solar power), multiple correlated features temperature, humidity, and cloud coverage). The are collected from different data sources subject heterogeneous ages. By using an information-theoretic approach, prove that minimum training loss function ages features, where not always monotonic. However, if...
Machine learning (ML) has been embedded in many Internet of Things (IoT) applications (e.g., smart home and autonomous driving). Yet it is often infeasible to deploy ML models on IoT devices due resource limitation. Thus, deploying trained the cloud providing inference services becomes a plausible solution. To provide low-latency serving massive devices, natural promising approach use parallelism computation. However, existing systems Tensorflow) ML-serving platforms SageMaker) are...
In this paper, we analyze the impact of information freshness on supervised learning based forecasting. these applications, a neural network is trained to predict time-varying target (e.g., solar power), multiple correlated features temperature, humidity, and cloud coverage). The are collected from different data sources subject heterogeneous ages. By using an information-theoretic approach, prove that minimum training loss function ages features, where not always monotonic. However, if...