JAWS: Job-aware workload scheduling for the exploration of turbulence simulations

Xiaodan Wang, Eric Perlman, Randal Burns, Tanu Malik, Tamas Budavári, Charles Meneveau, Alexander Szalay

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present JAWS, a job-aware, data-driven batch scheduler that improves query throughput for data-intensive scientific database clusters. As datasets reach petabyte-scale, workloads that scan through vast amounts of data to extract features are gaining importance in the sciences. However, acute performance bottlenecks result when multiple queries execute simultaneously and compete for I/O resources. Our solution, JAWS, divides queries into I/O-friendly sub-queries for scheduling. It then identifies overlapping data requirements within the workload and executes sub-queries in batches to maximize data sharing and reduce redundant I/O. JAWS extends our previous work [1] by supporting workflows in which queries exhibit data dependencies, exploiting workload knowledge to coordinate caching decisions, and combating starvation through adaptive and incremental trade-offs between query throughput and response time. Instrumenting JAWS in the Turbulence Database Cluster [2] yields nearly three-fold improvement in query throughput when contention in the workload is high.

Original languageEnglish (US)
Title of host publication2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2010
DOIs
StatePublished - 2010
Event2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2010 - New Orleans, LA, United States
Duration: Nov 13 2010Nov 19 2010

Publication series

Name2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2010

Conference

Conference2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2010
Country/TerritoryUnited States
CityNew Orleans, LA
Period11/13/1011/19/10

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'JAWS: Job-aware workload scheduling for the exploration of turbulence simulations'. Together they form a unique fingerprint.

Cite this