Attentive capsule network for click-through rate and conversion rate prediction in online advertising

Estimating Click-through Rate (CTR) and Conversion Rate (CVR) are two essential user response prediction tasks in computing advertising and recommendation systems. The mainstream methods map sparse, high-dimensional categorical features (e.g., user id, item id) into low-dimensional representations with neural networks. Although they have achieved significant advancement in recent years, how to capture user's diverse interests effectively from past behaviors is still challenging. Recently some works try using attention-based methods to learn the representation from user behavior history adaptively. However, it is insufficient to capture the diversity of user's interests. As a step forward to improve this goal, we propose a method named Attentive Capsule Network (ACN). It uses Transformers for feature interaction and leverages capsule networks to capture multiple interests from user behavior history. To precisely obtain sequence representation related to the current advertisement, we further design a modified dynamic routing algorithm integrating with an attention mechanism. Experimental results on real-world datasets demonstrate the effectiveness of our proposed ACN with significant improvement over state-of-the-art approaches. Moreover, it also offers good explainability when extracting diverse interest points of users from behavior history.