Advances in multicore processors and accelerators have opened the flood gates to greater exploration application of machine learning techniques a variety applications. These advances, along with breakdowns several trends including Moore's Law, prompted an explosion that promise even computational capabilities. are coming many forms, from CPUs GPUs ASICs, FPGAs, dataflow accelerators. This paper surveys current state these been publicly announced performance power consumption numbers. The...

Abstract Massive scale, in terms of both data availability and computation, enables important breakthroughs key application areas deep learning such as natural language processing computer vision. There is emerging evidence that scale may be a ingredient scientific learning, but the importance physical priors domains makes strategies benefits scaling uncertain. Here we investigate neural-scaling behaviour large chemical models by varying model dataset sizes over many orders magnitude,...

Interactive massively parallel computations are critical for machine learning and data analysis. These a staple of the MIT Lincoln Laboratory Supercomputing Center (LLSC) has required LLSC to develop unique interactive supercomputing capabilities. Scaling frameworks, such as TensorFlow, analysis environments, MATLAB/Octave, tens thousands cores presents many technical challenges - in particular, rapidly dispatching tasks through scheduler, Slurm, starting instances applications with...

We present a framework for the estimation of driver behavior at intersections, with applications to autonomous driving and vehicle safety. The is based on modeling dynamics as hybrid-state system (HSS), decisions being modeled discrete-state continuous-state system. proposed method uses observable parameters track instantaneous continuous state estimates most likely given these observations. This paper describes that encompasses hybrid structure vehicle-driver coupling hidden Markov models...

This paper presents BigDAWG, a reference implementation of new architecture for "Big Data" applications. Such applications not only call large-scale analytics, but also real-time streaming support, smaller analytics at interactive speeds, data visualization, and cross-storage-system queries. Guided by the principle that "one size does fit all", we build on top variety storage engines, each designed specialized use case. To illustrate promise this approach, demonstrate its effectiveness...

Protected database search systems cryptographically isolate the roles of reading from, writing to, and administering database. This separation limits unnecessary administrator access protects data in case system breaches. Since protected was introduced 2000, area has grown rapidly, are offered by academia, start-ups, established companies. However, there is no best or set techniques. Design such a balancing act between security, functionality, performance, usability. challenge made more...

Modern applications often need to manage and analyze widely diverse datasets that span multiple data models [1], [2], [3], [4], [5]. Warehousing the through Extract-Transform-Load (ETL) processes can be expensive in such scenarios. Transforming disparate into a single model may degrade performance. Further, curating maintaining pipeline prove labor intensive. As result, an emerging trend is shift focus federating specialized stores enabling query processing across heterogeneous [6]. This...

This paper updates the survey of AI accelerators and processors from past three years. collects summarizes current commercial that have been publicly announced with peak performance power consumption numbers. The values are plotted on a scatter graph, number dimensions observations trends this plot again discussed analyzed. Two new plots based accelerator release dates included in year's paper, along additional some neuromorphic, photonic, memristor-based inference accelerators.

Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly being leveraged various domains such as law, finance, and medicine. However, these carry significant computational challenges, especially the compute energy costs required for inference. Inference already receive less attention than of training LLMs-despite how often large called on conduct inference reality (e.g.,...

The growing gap between data and users calls for innovative tools that address the challenges faced by big volume, velocity variety. Along with these standard three V's of data, an emerging fourth "V" is veracity, which addresses confidentiality, integrity, availability data. Traditional cryptographic techniques ensure veracity can have overheads are too large to apply This work introduces a new technique called Computing on Masked Data (CMD), improves allowing computations be performed...

The rise of graph analytic systems has created a need for ways to measure and compare the capabilities these systems. Graph analytics present unique scalability difficulties. machine learning, high performance computing, visual communities have wrestled with difficulties decades developed methodologies creating challenges move forward. proposed Subgraph Isomorphism Challenge draws upon prior from create challenge that is reflective many real-world processing holistic specification multiple...

Organizations are often faced with the challenge of providing data management solutions for large, heterogenous datasets that may have different underlying and programming models. For example, a medical dataset unstructured text, relational data, time series waveforms imagery. Trying to fit such in single system can adverse performance efficiency effects. As part Intel Science Technology Center on Big Data, we developing polystore designed problems. BigDAWG (short Data Analytics Working...

The ability to collect and analyze large amounts of data is a growing problem within the scientific community. gap between users calls for innovative tools that address challenges faced by big volume, velocity variety. Numerous exist allow store, query index these massive quantities data. Each storage or database engine comes with promise dealing complex Scientists engineers who wish use systems often quickly find there no single technology offers panacea complexity information. When using...

The energy requirements of current natural language processing models continue to grow at a rapid, unsustainable pace. Recent works highlighting this problem conclude there is an urgent need for methods that reduce the needs NLP and machine learning more broadly. In article, we investigate techniques can be used consumption common applications. particular, focus on measure usage different hardware datacenter-oriented settings tuned training inference models. We characterize impact these...

This work introduces Mashup, a novel strategy to leverage serverless computing model for executing scientific workflows in hybrid fashion by taking advantage of both the traditional VM-based cloud platform and emerging platform. Mashup outperforms state-of-the-art workflow execution engines an average 34% 43% terms time reduction cost reduction, respectively, widely-used HPC on Amazon Cloud (EC2 Lambda).

The rapid growth in demand for HPC systems has led to a rise carbon footprint, which requires urgent intervention. In this work, we present comprehensive analysis of the footprint high-performance computing (HPC) systems, considering during both hardware manufacturing and system operational stages. Our work employs component modeling, regional intensity analysis, experimental characterization life cycle highlight importance quantifying systems.

The growing demand for cloud computing motivates the need to study security of data received, stored, processed, and transmitted by a cloud. In this paper, we present framework such study. We introduce model that captures rich class big-data use-cases allows reasoning about relevant threats goals. then survey three cryptographic techniques - homomorphic encryption, verifiable computation, multi-party computation can be used achieve these describe in context our highlight differences...

The Apache Accumulo database is an open source relaxed consistency that widely used for government applications. designed to deliver high performance on unstructured data such as graphs of network data. This paper tests the using from Graph500 benchmark. Dynamic Distributed Dimensional Data Model (D4M) software implement benchmark a 216-node cluster running MIT SuperCloud stack. A peak over 100,000,000 inserts per second was achieved which 100x larger than highest previously published value...

For decades, the growth and volume of digital data collection has made it challenging to digest large volumes information extract underlying structure. Coined `Big Data', massive amounts quite often been gathered inconsistently (e.g from many sources, various forms, at different rates, etc.). These factors impede practices not only processing data, but also analyzing displaying in an efficient manner user. Many efforts have completed mining visual analytics community create effective ways...

The success of SQL, NoSQL, and NewSQL databases is a reflection their ability to provide significant functionality performance benefits for specific domains, such as financial transactions, internet search, data analysis. BigDAWG polystore seeks mechanism allow applications transparently achieve the diverse while insulating from details these databases. Associative arrays common approach mathematics found in different databases: sets (SQL), graphs (NoSQL), matrices (NewSQL). This work...

This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML is critical modern technology products, but it also significant contributor footprint. We introduce Clover, carbon-friendly service runtime system that balances performance, accuracy, and through mixed-quality models GPU resource partitioning. Our experimental results demonstrate Clover effective in substantially reducing while maintaining high...

The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud and high performance computing (HPC) infrastructure. This paper presents Sprout, an innovative framework designed to address these concerns by reducing footprint generative Large Language Model (LLM) inference services. Sprout leverages concept "generation directives" guide autoregressive generation process,...

An important objective for analyzing real-world graphs is to achieve scalable performance on large, streaming graphs. A challenging and relevant example the graph partition problem. As a combinatorial problem, NP-hard, but existing relaxation methods provide reasonable approximate solutions that can be scaled large Competitive benchmarks challenges have proven an effective means advance state-of-the-art foster community collaboration. This paper describes challenge with baseline algorithm of...

The MIT/IEEE/Amazon GraphChallenge.org encourages community approaches to developing new solutions for analyzing graphs and sparse data. Sparse AI analytics present unique scalability difficulties. proposed Deep Neural Network (DNN) Challenge draws upon prior challenges from machine learning, high performance computing, visual create a challenge that is reflective of emerging systems. DNN based on mathematically well-defined inference computation can be implemented in any programming...