Distributed hierarchical deep optimization for federated learning in mobile edge computing

Zheng, Xiao ORCID: https://orcid.org/0000-0003-4740-7695, Shah, Syed Bilal Hussain, Bashir, Ali Kashif ORCID: https://orcid.org/0000-0001-7595-2522, Nawaz, Raheel ORCID: https://orcid.org/0000-0001-9588-0052 and Rana, Umer (2022) Distributed hierarchical deep optimization for federated learning in mobile edge computing. Computer Communications, 194. pp. 321-328. ISSN 0140-3664

Preview

Accepted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview

Abstract

Deep learning has recently attracted great attention in many application fields, especially for big data analysis in the field of edge computing. Federated learning, as a promising machine learning technology, applies training data on distributed edge nodes to design shared learning systems to protect data privacy. Due to the system update in federated learning is at the expense of parameter exchange between edge nodes, it is extremely bandwidth consuming. A novel distributed hierarchical tensor depth optimization algorithm is proposed, which compresses the model parameters from the high-dimensional tensor space to a union of low-dimensional subspaces to reduce bandwidth consumption and storage demands of federated learning. In addition, an update method based on hierarchical tensor back propagation is developed by directly calculating the gradient of low-dimensional parameters to reduce the memory requirement and improve the training efficiency caused by edge node training. Finally, a large number of simulation experiments were performed to evaluate performance based on classical data sets with different local data distributions. Experimental results show that the proposed algorithm reduces the burden of communication bandwidth and the energy consumption of edge nodes.