Introduction
At Apple, we believe that privacy is a fundamental human right. It’s also one of our core values, influencing both our research and the design of Apple’s products and services. In this article, we’ll discuss the importance of privacy in machine learning, and how we’re working to ensure that our technology is designed with privacy in mind.
Federated Learning and Statistics
Workshop presenters explored advances in federated learning (FL), the associated differentially private algorithms for ML optimization, and its widespread impact via practical deployment.
- Training data remains private on user devices, commonly referred to as edge devices.
- Aggregated gradients enable data scientists to learn from a population.
- Differential privacy (DP) guarantees that the model protects the privacy of the data on edge devices.
Simulations are key to quantifying data requirements and the impact of federated partitioning, as well as for bootstrapping the gradient and hyper-parameter search space. In their talk, presenters Mona Chitnis and Filip Granqvist of Apple discussed their work developing a PFL simulation framework. This framework has enabled the training of models with PFL for various use cases, including large vocabulary language models on resource-constrained devices, a tokenizer, and a recommender system improved with embeddings.
In conjunction with the workshop, we open sourced the pfl-research Python framework for PFL simulations, with key benchmarks showing much faster speeds than existing open source tools. We envision that pfl-research will accelerate research by equipping the community with vetted tools for prototyping algorithms and promoting the reproducibility of realistic FL benchmarks. This follows the open sourcing of the FLAIR dataset, which was shared at the Workshop on Privacy Preserving Machine Learning in 2022.
Benchmarks Dashboard
Image 1: CIFAR10 iid no DP benchmark results.
Image 2: StackOverflow no DP benchmark results.
Image 3: FLAIR non-iid DP benchmark results.
Conclusion
In conclusion, our workshop on privacy-preserving machine learning explored the latest advances in federated learning and statistics, and how we’re working to ensure that our technology is designed with privacy in mind. We believe that privacy is a fundamental human right, and we’re committed to protecting the privacy of our users.
Frequently Asked Questions
Q1: What is federated learning?
Federated learning is a machine learning approach that enables the training of models on decentralized data without requiring the sharing of the data itself. This approach ensures that the data remains private, and the model is trained on aggregated gradients.
Q2: What is differential privacy?
Differential privacy is a mathematical framework that ensures that the outcome of a computation or data analysis cannot be distinguished from the outcome of a slightly different computation or data analysis, without knowing the specifics of the computation or data analysis.
Q3: How does Apple ensure privacy in its products and services?
