Local contexts are effective for neural aspect extraction

Recently, long short-term memory based recurrent neural network (LSTM-RNN), which is capable of capturing long dependencies over sequence, obtained state-of-the-art performance on aspect extraction. In this work, we would like to investigate to which extent could we achieve if we only take into account of the local dependencies. To this end, we develop a simple feed-forward neural network which takes a window of context words surrounding the aspect to be processed. Surprisingly, we find that a purely window-based neural network obtain comparable performance with a LSTM-RNN approach, which reveals the importance of local contexts for aspect extraction. Furthermore, we introduce a simple and natural way to leverage local contexts and global contexts together, which is not only computationally cheaper than existing LSTM-RNN approach, but also gets higher classification accuracy.