​DNN pruning is a popular way to reduce the size of a model, improve the inference latency, and minimize the power consumption on DNN accelerators. However, existing approaches might be too complex, expensive or ineffective to apply to a variety of vision/language tasks, DNN architectures… Read More »PDP: Parameter-free Differentiable Pruning is All You Need Apple Machine Learning Research

​Given a sequence of observable variables , the conformal prediction method estimates a confidence set for given that is valid for any finite sample size by merely assuming that the joint distribution of the data is permutation invariant. Although attractive, computing such a set is… Read More »Conformalization of Sparse Generalized Linear Models Apple Machine Learning Research

​Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and… Read More »DUET: 2D Structured and Equivariant Representations Apple Machine Learning Research

​We study practical heuristics to improve the performance of prefix-tree based algorithms for differentially private heavy hitter detection. Our model assumes each user has multiple data points and the goal is to learn as many of the most frequent data points as possible across all… Read More »Differentially Private Heavy Hitter Detection using Federated Analytics Apple Machine Learning Research

​Federated learning (FL) combined with differential privacy (DP) offers machine learning (ML) training with distributed devices and with a formal privacy guarantee. With a large population of devices, FL with DP produces a performant model in a timely manner. However, for applications with a smaller… Read More »Population Expansion for Training Language Models with Private Federated Learning Apple Machine Learning Research

​The mechanisms behind the success of multi-view self-supervised learning (MVSSL) are not yet fully understood. Contrastive MVSSL methods have been studied though the lens of InfoNCE, a lower bound of the Mutual Information (MI). However, the relation between other MVSSL methods and MI remains unclear.… Read More »The Role of Entropy and Reconstruction for Multi-View Self-Supervised Learning Apple Machine Learning Research