A Risk-sensitive Automatic Stock Trading Strategy Based on Deep Reinforcement Learning and Transformer

Navigating the world of stock trading can be a daunting task, with its high risk, high reward potential, and intricate market dynamics. For investors and financial researchers, making informed investment decisions is essential. To our knowledge, limited by insufficient feature representation in highly dynamic trading scenarios, existing rule-based and machine learning-based methods struggle to extract complex intrinsic information from the stock market. Thus, with the huge success of reinforcement learning (RL) in sequential decision-making tasks and the powerful feature extraction capabilities of Transformer, this paper proposes an automatic stock trading method based on RL and Transformer. First, data from multiple stocks are collected into Transformer to extract the correlations among stocks. Then, the output of Transformer is used as the state input for deep RL, and the trading action is obtained. Furthermore, we propose a risk-sensitive reward function by adding the turbulence index to avoid risk. Experimental results on Dow Jones 30 constituent stocks show that the proposed method outperforms baselines regarding profit, risk, and comprehensive indicators.