A Two-Stage Framework for Real-Time Action Recognition in Human-Robot Confrontation

The balance between high accuracy and low latency in multi-target action recognition for Human-Robot confrontation has been a key research focus. To address the challenge, a novel two-stage Top-Down approach is proposed for perceptual feature extraction and behavioral understanding separately. At the target perception stage, YOLOv8 is utilized to extract multi-target bounding boxes and feature maps. Hue-Saturation-Value (HSV) based color tracking is employed to achieve real-time positioning and identity discrimination of antagonistic individuals. For behavior understanding, Real-Time Multi-person Coordinate Classification (RTMCC) is introduced as the detection head for multi-target pose estimation, enabling the extraction of high-quality topological information of key skeleton points. Hierarchical Multi-Scale Graph Convolutional Network (HMS-GCN) is designed to enhance action recognition. Inference accuracy of the proposed integrated algorithmic framework is enhanced while maintaining the speed of 30+ fps, significantly improving robustness in confrontation scenarios.