Admission Control with Response Time Objectives for Low-latency Online Data Systems

• DOI: 10.48550/arxiv.2312.15123
• Publication Date: 2023-01-01
• Authors: Hao Xu; Juan A. Colmenares
• Keywords: Response time; Service-level agreement; Guard (computer science)

To provide quick responses to users, Internet companies rely on online data systems able answer queries in milliseconds. These employ complementary overload management techniques ensure they a continued, acceptable service through-out traffic surges, where 'acceptable' partly means that serviced meet or track closely their response time objectives. Thus, this paper we present Bouncer, an admission control policy aimed keep admitted under near level objectives (SLOs) percentile times. It computes inexpensive estimates of times for every incoming query and compares the against objective values decide whether accept reject query. Bouncer allows assigning separate SLOs different classes workload, implements early rejections let clients react promptly help avoid doing useless work, complements other load shedding policies guard from exceeding capacity. Moreover, propose two starvation avoidance strategies supplement Bouncer's basic formulation prevent types receiving no (starving). We evaluate its starvation-avoiding variants simulation production-grade in-memory distributed graph database. Our results show allow stay close when do not. They also report fewer overall rejections, small overhead, with given latency SLOs, system reach high utilization. In addition, observe proposed can stop starvation, but at expense modest increase causing SLO violations requests.