Enhancing Performance Prediction Robustness by Combining Analytical Modeling and Machine Learning
- 31 January 2015
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
Abstract
Classical approaches to performance prediction rely on two, typically antithetic, techniques: Machine Learning (ML) and Analytical Modeling (AM). ML takes a black box approach, whose accuracy strongly depends on the representativeness of the dataset used during the initial training phase. Specifically, it can achieve very good accuracy in areas of the features' space that have been sufficiently explored during the training process. Conversely, AM techniques require no or minimal training, hence exhibiting the potential for supporting prompt instantiation of the performance model of the target system. However, in order to ensure their tractability, they typically rely on a set of simplifying assumptions. Consequently, AM's accuracy can be seriously challenged in scenarios (e.g., workload conditions) in which such assumptions are not matched. In this paper we explore several hybrid/gray box techniques that exploit AM and ML in synergy in order to get the best of the two worlds. We evaluate the proposed techniques in case studies targeting two complex and widely adopted middleware systems: a NoSQL distributed key-value store and a Total Order Broadcast (TOB) service.Keywords
Funding Information
- Fundação para a Ciência e a Tecnologia (specSTM (PTDC/EIA-EIA/122785/2010), GreenTM (EXPL/EEI-ESS/0361/2013), PEst-OE/EEI/LA0021/2013 Program Funds (INESC-ID multiannual funding), Action IC1001)
- Fundação para a Ciência e a Tecnologia (specSTM (PTDC/EIA-EIA/122785/2010), GreenTM (EXPL/EEI-ESS/0361/2013), PEst-OE/EEI/LA0021/2013 Program Funds (INESC-ID multiannual funding), Action IC1001)
This publication has 36 references indexed in Scilit:
- Identifying the optimal level of parallelism in transactional memory applicationsComputing, 2013
- Regulating Concurrency in Software Transactional Memory: An Effective Model-based ApproachPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- A Framework for High Performance Simulation of Transactional Data Grid PlatformsPublished by European Alliance for Innovation n.o. ,2013
- On the analytical modeling of concurrency control algorithms for Software Transactional Memories: The case of Commit-Time-LockingPerformance Evaluation, 2012
- ModellusACM Transactions on the Web, 2012
- On deciding between conservative and optimistic approaches on massively parallel platformsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- A Machine Learning Approach to Performance Prediction of Total Order Broadcast ProtocolsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Benchmarking cloud serving systems with YCSBPublished by Association for Computing Machinery (ACM) ,2010
- D2STM: Dependable Distributed Software Transactional MemoryPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Ensemble selection from libraries of modelsPublished by Association for Computing Machinery (ACM) ,2004