PrivBoost: A federated learning framework for differentially private tree boosting

Federated learning (FL) provides an intriguing paradigm for privacy-friendly collaborative model training. While many efforts focus on training deep learning models under FL, a recent trend is to train gradient-boosted tree models, due to their high efficiency, good interpretability, and promising accuracy, particularly on tabular data. For strong privacy protection, it is imperative to protect not only individual clients’ private training data and gradient updates during the training process, but also the trained gradient-boosted trees. This is because the trained models, if not protected appropriately, could be exploited to infer private information about the training data. This paper proposes PrivBoost, a new FL service framework for training differentially private gradient-boosted decision trees. PrivBoost builds on a delicate synergy of lightweight secure aggregation techniques with differential privacy (DP), enabling the coordinating server to obliviously aggregate individual gradient updates and produce differentially private gradient-boosted tree models. PrivBoost mainly departs from prior work by distributing noise generation for DP among the clients, with custom constructions enabling clients to generate effective inputs for our differentially private secure aggregation protocol. Formal analysis is provided to prove the DP guarantees. Extensive experiments on both real-world and synthetic datasets demonstrate that PrivBoost can achieve model utility comparable to the central DP setting, with promising performance.