Adaptive Skewness Kurtosis Neural Network: Enabling Communication Between Neural Nodes Within a Layer

The statistical properties of neural networks are closely associated with their performance. From this perspective, the training process of deep learning models can be divided into two stages corresponding to the procedures of feature extraction and integration. In the feature extraction stage, the mean and variance of the hidden layer changes; during the feature integration stage, the mean and variance remain relatively stable, while the skewness and kurtosis change considerably. Meanwhile, constructing intra-layer connections may improve the performance of neural networks. Consequently, a novel Adaptive Skewness Kurtosis (ASK) structure is proposed, which enables deep learning networks to connect within a layer. On the basis of stabilizing the mean and variance of the layer, the ASK structure adaptively adjusts the skewness and kurtosis of the layer by communicating the connections between neuron nodes in the layer to improve the feature integration ability of the model ultimately. Based on the ASK structure, we propose an ASK neural network (ASKNN) where the ASK structure designed to a standard BP neural network (BPNN) to adjust the high order moments. Compared with the standard BPNN, ASKNN performs better especially when dealing with the data contaminated with noise.