Search for deep graph neural networks

Although graph neural networks (GNNs) have shown their effectiveness with non-Euclidean data, robust GNNs require elaborate designs. Recent works have attempted to use Neural Network Architecture Search (NAS) to automatically design GNN model architectures. However, existing GNN-oriented NAS methods mainly focus on combining different layer aggregate components for shallow and simple architectures that are limited by the “over-smooth” problem. To further explore the benefits of structural diversity and depth of GNN architectures, we propose a GNN generation pipeline with a novel two-stage search space, which aims at automatically generating a high-performance while transferable deep GNN models in a block-wise manner. To alleviate the “over-smooth” problem, we incorporate flexible residual connections in our search space and apply identity mapping in the GNN layers. We use deep-Q-learning with an epsilon-greedy strategy and reward reshaping to guide the network search. Extensive experiments on real-world datasets show that our generated GNN models outperform both existing manually designed and NAS-based ones, and surpass all other methods by a large margin in the fully-supervised scenario.