Data centers are often under-utilized due to over-provisioning as well time-varying resource demands of typical enterprise applications. One approach increase utilization is consolidate applications in a shared infrastructure using virtualization. Meeting application-level quality service (QoS) goals becomes challenge consolidated environment application needs differ. Furthermore, for multi-tier applications, the amount resources needed achieve their QoS might be different at each tier and...

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a control system that automatically adapts dynamic workload changes achieve SLOs. AutoControl combination an online model estimator novel multi-input, multi-output (MIMO) controller. The...

We propose a new paradigm for building scalable distributed systems. Our approach does not require dealing with message-passing protocols -- major complication in existing Instead, developers just design and manipulate data structures within our service called Sinfonia. Sinfonia keeps applications on set of memory nodes, each exporting linear address space. At the core is novel minitransaction primitive that enables efficient consistent access to data, while hiding complexities arise from...

Enterprise-scale storage systems, which can contain hundreds of host computers and devices up to tens thousands disks logical volumes, are difficult design. The volume choices that need be made is massive, many have unforeseen interactions. Storage system design tedious complicated do by hand, usually leading solutions grossly over-provisioned, substantially under-performing or, in the worst case, both.To solve configuration nightmare, we present minerva: a suite tools for designing systems...

This paper describes the design, implementation, and evaluation of a Federated Array Bricks (FAB), distributed disk array that provides reliability traditional enterprise arrays with lower cost better scalability. FAB is built from collection bricks, small storage appliances containing commodity disks, CPU, NVRAM, network interface cards. deploys new majority-voting-based algorithm to replicate or erasure-code logical blocks across bricks reconfiguration move data in background when are...

Virtualized servers run a diverse set of virtual machines (VMs), ranging from interactive desktops to test and development environments even batch workloads. Hypervisors are responsible for multiplexing the underlying hardware resources among VMs while providing them desired degree isolation using resource management controls. Existing methods provide many knobs allocating CPU memory VMs, but support control IO allocation has been quite limited. in hypervisor introduces significant new...

Considers a problem of network design personal communication services (PCS). The is to assign cells the switches PCS in an optimum manner. authors consider two types costs. One cost handoffs between cells. other cabling (or trunking) cell site and its associated switch. constrained by call volume that each switch can handle. formulate exactly as integer programming problem. They also propose heuristic solution for this show it performs extremely well.< <ETX...

Modern disk drives read-ahead data and reorder incoming requests in a workload-dependent fashion. This improves their performance, but makes simple analytical models of them inadequate for performance prediction, capacity planning, workload balancing, so on. To address this problem we have developed new analytic model that do readahead request reordering. We did by developing the drive components (queues, caches, mechanism) transformation technique composing them. Our includes effects...

Data centers are often under-utilized due to over-provisioning as well time-varying resource demands of typical enterprise applications. One approach increase utilization is consolidate applications in a shared infrastructure using virtualization. Meeting application-level quality service (QoS) goals becomes challenge consolidated environment application needs differ. Furthermore, for multi-tier applications, the amount resources needed achieve their QoS might be different at each tier and...

Storage consolidation is becoming an attractive paradigm for data organization because of the economies sharing and ease centralized management. However, resources viable only if applications can be isolated from each other. This work targets problem providing performance guarantees to application irrespective behavior other workloads. Application requirements are represented in terms average throughput, latency maximum burst size. Most earlier schemes do weighted bandwidth allocation; that...

Feedback mechanisms can help today's increasingly complex computer systems adapt to changes in workloads or operating conditions. Control theory offers a principled way for designing feedback loops deal with unpredictable changes, uncertainties, and disturbances systems. We provide an overview of the joint research at HP Labs University Michigan past few years, where control was applied automated resource service level management data centers. highlight key benefits control-theoretic...

The performance modeling and analysis of disk arrays is challenging due to the presence multiple disks, large array caches, sophisticated controllers. Moreover, storage manufacturers may not reveal internal algorithms implemented in their devices, so real are effectively black-boxes. We use standard techniques develop an integrated model that incorporates some complexities arrays. show how measurement data baseline models can be used extract information about various features a array. In...

Enterprise storage systems depend on disk arrays for their capacity and availability needs. To design maintain that efficiently satisfy evolving requirements, it is critical to be able evaluate configuration alternatives without having physically implement them. In this paper, we describe an analytical model predict array throughput, based a hierarchical decomposition of the internal architecture. We validate against state-of-the-art variety synthetic workloads configurations. our knowledge,...

Most of the data in large-scale storage clusters is erasure coded. At exascale, optimizing codes for low overhead, efficient reconstruction, and easy deployment critical importance. Locally recoverable (LRCs) have deservedly gained central importance this field, because they can balance many these requirements. In our work, we study wide LRCs; LRCs with large number blocks per stripe overhead. These are a natural next step practitioners to unlock higher savings, but come their own...

We propose a new paradigm for building scalable distributed systems. Our approach does not require dealing with message-passing protocols, major complication in existing Instead, developers just design and manipulate data structures within our service called Sinfonia. Sinfonia keeps applications on set of memory nodes, each exporting linear address space. At the core is minitransaction primitive that enables efficient consistent access to data, while hiding complexities arise from...

Solid-state drives (SSDs) based on NAND flash are making deep inroads into data centers as well the consumer market. In 2016, manufacturers shipped more than 130 million units totaling around 50 Exabytes of storage capacity. As amount stored solid state keeps increasing, it is important to understand reliability characteristics these devices. For a long time, our knowledge about was derived from controlled experiments in lab environments under synthetic workloads, often using methods for...

Sharding is a fundamental building block of large-scale applications, but most have their own custom, ad-hoc implementations. Our goal to make sharding as easily reusable filesystem or lock manager. Slicer Google's general purpose service. It monitors signals such load hotspots and server health dynamically shard work over set servers. Its goals are maintain high availability reduce imbalance while minimizing churn from moved work.In this paper, we describe Slicer's design implementation....

Storage systems account for a major portion of the total cost ownership (TCO) warehouse-scale computers, and thus have impact on overall system's efficiency. Machine learning (ML)-based methods solving key problems in storage system efficiency, such as data placement, shown significant promise. However, there are few known practical deployments methods. Studying this problem context real-world hyperscale center at Google, we identify number challenges that believe cause lack adoption....

A Redundant Array of Independent Disks (RAID) G disks provides protection against single disk failures by adding one parity block for each G-1 data blocks. In a clustered RAID, the data/parity blocks are distributed over cluster C (C>G), thus reducing additional load on due to failure. However, most methods proposed implementing such mapping do not work general and values. this paper, we describe fast algorithm based almost-random permutations. An analytical model is constructed, queue with...