A Case Study Report on Optimization Techniques for Federated Learning in Machine Learning
Keywords:
CIFAR-10, CNN, Decentralized training, Federated learning, Flower framework, Optimization techniques, Privacy preservationAbstract
Federated Learning (FL) has recently emerged as a transformative paradigm in machine learning, enabling multiple decentralized clients to collaboratively train models without exchanging raw data. This approach addresses critical privacy concerns while still leveraging diverse data distributions from different sources. The present case study investigates optimization techniques for FL using the Flower framework, focusing on the CIFAR-10 image classification dataset. A Convolutional Neural Network (CNN) architecture was implemented and trained under both centralized and federated settings to provide a direct performance comparison. Several optimization strategies, including Federated Averaging (FedAvg), Local SGD, and Adam optimizer, were examined to assess their efficiency, accuracy, and robustness. The experimental findings reveal that FL can achieve performance levels comparable to centralized training while significantly reducing risks associated with data sharing. The analysis further highlights the trade-offs between communication efficiency, convergence speed, and resilience to non-IID data distributions. The study demonstrates Flower’s practicality as a research and deployment framework for distributed learning, offering valuable insights for extending FL applications in privacy-sensitive domains such as healthcare, finance, and edge computing.
References
D. J. Beutel, T. Topal, A. Mathur, X. Qiu, “Flower: A Friendly Federated Learning Research Framework,” arXiv:2007.14390, Apr. 2021, Available: https://arxiv.org/abs/2007.14390
K. Bonawitz, H. Eichner, W. Grieskamp, “Towards Federated Learning at Scale: System Design,” arXiv.org, 2019. https://arxiv.org/abs/1902.01046
M. H. Brendan, E. Moore, D. Ramage, and S. Hampson, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” arXiv.org, 2016. https://arxiv.org/abs/1602.05629
L. Lyu, H. Yu, and Q. Yang, “Threats to Federated Learning: A Survey,” arXiv:2003.02133 , Mar. 2020, Available: https://arxiv.org/abs/2003.02133
X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the Convergence of FedAvg on Non-IID Data,” 2020 International Conference on Learning Representations. June. 2020, Available: https://arxiv.org/abs/1907.02189
T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated Learning: Challenges, Methods, and Future Directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, May 2020, doi: https://doi.org/10.1109/msp.2020.2975749
P. Kairouz and H. B. McMahan, “Advances and Open Problems in Federated Learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1, 2021, doi: https://doi.org/10.1561/2200000083
S. Bharati, M. R. H. Mondal, P. Podder, and V. B. S. Prasath, “Federated learning: Applications, challenges and future scopes,” International Journal of Hybrid Intelligent Systems, pp. 1–17, Apr. 2022, doi: https://doi.org/10.3233/his-220006
B. Liu, L. Wu, L. Chen, “Communication Efficient Distributed Training with Distributed Lion,” arXiv.org, 2024. https://arxiv.org/abs/2404.00438
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated Learning with Non-IID Data,” arXiv.org, 2018. https://arxiv.org/abs/1806.00582
T. Lin, S. U. Stich, K. K. Patel, and M. Jaggi, “Don’t Use Large Mini-Batches, Use Local SGD,” arXiv.org, 2018. https://arxiv.org/abs/1808.07217
