Strange Loop

Measuring and Optimizing Tail Latency

Data centers that service interactive user requests require careful engineering to optimize the tail (99th percentile or higher) response times or users abandon the service. Optimizing these systems is challenging because tail latency is hard to measure and root causes of latency span networking, software, and hardware. We describe a new tool and methodology to measure performance at 1000 cycle granularities with 1% overheads, orders-of-magnitude more detail than interrupt based sampling approaches. Our tool teases apart root causes of tail latency inspiring optimizations. For queuing delay, we describe scaling approaches and for requests with lots of work, we introduce a novel dynamic adaptive parallelization approach that reduces tail latency and surprisingly improves throughput. Optimizing the tail while substantially improving server efficiency is good for users, profitability, and the environment.

Kathryn McKinley

Google and University of Texas at Austin

Kathryn S. McKinley is a Research Scientist at Google and an Adjunct Professor at the University of Texas at Austin. She was previously a Principal Researcher at Microsoft and held an Endowed Professorship at The University of Texas. Her research interests span programming languages, compilers, runtime systems, architecture, performance, and energy. She and her collaborators have produced several widely used tools: the DaCapo Java Benchmarks (30,000+ downloads), the TRIPS Compiler, Hoard memory manager, MMTk memory management toolkit, and the Immix garbage collector. Her awards include the ACM SIGPLAN Programming Languages Software Award and the ACM SIGPLAN Distinguished Service Award. Her research has received test-of-time awards from OOPSLA (2), ICS, and SIGMETRICS, and best paper awards from ASPLOS, IEEE Micro Top Picks (4), SIGPLAN Research Highlights, and CACM Research Highlights (2). She served as program chair for ASPLOS, PACT, PLDI, ISMM, and CGO. She currently serves on the CRA Board, CRA-W Board member, and SIGPLAN EC. Dr. McKinley was honored to testify to the House Science Committee (Feb. 14, 2013). She is an IEEE and ACM Fellow, has graduated 22 PhD students, and is the parent of three sons.