Labeling-free RAG-enhanced LLM for intelligent fault diagnosis via reinforcement learning

Large language models (LLMs)-based fault diagnosis methods leverage extensive general-domain knowledge to provide valuable insights for fault analysis and maintenance. However, due to the low frequency of faults occurrences in complex systems, LLMs often exhibit insufficient coverage of fault diagnosis-related knowledge. This limitation reduces their effectiveness in such applications. In this context, retrieval-augmented generation (RAG) has emerged as a promising solution by enriching LLMs with relevant expert knowledge. Most existing RAG frameworks rely on supervised learning, training the retrieval network with fault corpora, query sets, and corresponding annotated labels. However, the inherent randomness of LLMs leads to instability in generated responses. As a result, it becomes challenging to consistently determine the relevance of different retrieved documents. To mitigate this instability, multiple LLM calls are often required, which further complicates the labeling process. To address these challenges, this paper proposes a reinforcement learning-based RAG method (TG-RL-RAG) that avoids supervised labeling. The method employs graph-structured fault corpora, optimizes the retrieval strategy via proximal policy optimization (PPO). Furthermore, considering the steadily increasing of fault-related queries over time, the method introduces a progressively diminishing teacher-guidance strategy, employing the previously trained agent as a dynamic teacher to guide a new processing agent, enabling continual learning with new queries. Finally, the paper uses fault logs from a specific set of Type-1 and Type-2 heavy-duty trains over one year, and conducts comparative experiments to validate the effectiveness and necessity of the proposed method.