Abstract
Large systems supporting mixed workloads incorporate workload management software allocating and controlling usage of resources and focusing on meeting Service Level Goals (SLG) of each workload [8,9]. When the response time of a workload exceeds SLG, we call it an anomaly. When the increase in response time by a workload happens periodically at the same time, we call it a seasonal peak [3]. Systems administrators are creating Workload Management rules to prevent anomalies and allocate enough resources during seasonal peaks to satisfy SLGs. When usage of resources increases, they allocate more resources, but they are still concerned with the risk of performance surprises. In this paper we will focus on Big Data environment and will discuss how to apply modeling to find appropriate YARN Scheduler Queue settings to meet SLGs for Data Lakes, ad hoc and batch workloads and how to determine when additional hardware resources will be required.
| Original language | English (US) |
|---|---|
| State | Published - 2017 |
| Externally published | Yes |
| Event | Internet, Mobile, Performance and Capacity, Cloud and Technology Conference, imPACt 2017 - New Orleans, United States Duration: Nov 6 2017 → Nov 9 2017 |
Conference
| Conference | Internet, Mobile, Performance and Capacity, Cloud and Technology Conference, imPACt 2017 |
|---|---|
| Country/Territory | United States |
| City | New Orleans |
| Period | 11/6/17 → 11/9/17 |
Keywords
- Anomaly detection
- Autonomic computing
- Big data dynamic performance management
- Capacity planning
- Prescriptive analytics
- Problem prediction
- Root cause analysis
- Self-aware computing
- Workload management
ASJC Scopus subject areas
- Computer Vision and Pattern Recognition
- Software
- Computer Networks and Communications