Multi-Job Intelligent Scheduling With Cross-Device Federated Learning
- 28 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 34 (2), 535-551
- https://doi.org/10.1109/tpds.2022.3224941
Abstract
Recent years have witnessed a large amount of decentralized data in various (edge) devices of end-users, while the decentralized data aggregation remains complicated for machine learning jobs because of regulations and laws. As a practical approach to handling decentralized data, Federated Learning (FL) enables collaborative global machine learning model training without sharing sensitive raw data. The servers schedule devices to jobs within the training process of FL. In contrast, device scheduling with multiple jobs in FL remains a critical and open problem. In this paper, we propose a novel multi-job FL framework, which enables the training process of multiple jobs in parallel. The multi-job FL framework is composed of a system model and a scheduling method. The system model enables a parallel training process of multiple jobs, with a cost model based on the data fairness and the training time of diverse devices during the parallel training process. We propose a novel intelligent scheduling approach based on multiple scheduling methods, including an original reinforcement learning-based scheduling method and an original Bayesian optimization-based scheduling method, which corresponds to a small cost while scheduling devices to multiple jobs. We conduct extensive experimentation with diverse jobs and datasets. The experimental results reveal that our proposed approaches significantly outperform baseline approaches in terms of training time (up to 12.73 times faster) and accuracy (up to 46.4% higher).Keywords
Funding Information
- Collaborative Innovation Center of Novel Software Technology and Industrialization
This publication has 55 references indexed in Scilit:
- Celebrating diversity: a mixture of experts approach for runtime mapping in dynamic environmentsPublished by Association for Computing Machinery (ACM) ,2015
- The Singapore Personal Data Protection Act and an assessment of future trends in data privacy reformComputer Law & Security Review, 2013
- An architecture for parallel topic modelsProceedings of the VLDB Endowment, 2010
- Differential Privacy: A Survey of ResultsPublished by Springer Science and Business Media LLC ,2008
- Optimization engineering techniques for the exact solution of NP-hard combinatorial optimization problemsEuropean Journal of Operational Research, 2000
- Public-Key Cryptosystems Based on Composite Degree Residuosity ClassesPublished by Springer Science and Business Media LLC ,1999
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- Complexity of Scheduling Parallel Task SystemsSIAM Journal on Discrete Mathematics, 1989
- Simulated annealingPublished by Springer Science and Business Media LLC ,1987