On-device large language model-powered intelligent copilot for on-site construction tutorials

On-site technical support is essential for low-skilled construction workers, but persistent challenges including inefficient communication and disorganized construction documents hinder knowledge transmission. The remarkable advancements of large language models (LLMs) in natural language processing make it possible to address this issue, but existing chatbots like ChatGPT rely on cloud services, limiting their application at construction sites. Therefore, this paper combines a lightweight LLM with edge computing to propose an offline AI copilot framework for providing construction tutorials, which can be deployed on a smartphone. This framework is divided into a web UI and a backend consisting of four layers: interaction layer, agent layer, engine layer, and OS layer. A three-stage workflow is designed to guide the LLM in proper tool use. Multiple optimization strategies including stream processing, model quantization, prompt cache, multithreading tuning and so on are employed to improve system efficiency. A laboratory experiment and a field experiment were conducted to verify its workability. The experimental results demonstrate that the AI copilot provides accurate responses to diverse queries across different construction scenarios and maintains acceptable interaction latency. This research advances the application of LLMs in on-site worker support, offering an intelligent approach to improve the management of construction progress, quality and safety.