The acceleration of YOLOv5s through NVDLA on RISC-V SoC

Neural network hardware accelerators have become essential for deploying convolutional neural networks (CNNs) on edge devices. However, due to hardware and software constraints, these accelerators are often optimized for specific network architectures, limiting their ability to accelerate general-purpose neural network models such as YOLO (You Only Look Once). In contrast, in this paper, the integration of the NVIDIA Deep Learning Accelerator (NVDLA) into a RISC-V architecture-based System on Chip (SoC) to accelerate the YOLOv5s network model is presented. While simple network acceleration can be achieved through register-level configurations of NVDLA, like LeNet-5, optimizing YOLOv5s performance requires a more advanced approach. We interfaced the Tengine edge computing framework with the NVDLA software, enabling the registration of specific operators to leverage NVDLA for intensive computations while delegating unsupported tasks to the CPU. This hybrid acceleration strategy ensures efficient workload distribution, maximizing the performance benefits of both the accelerator and the processor. The proposed system was validated on a 250MHz Field Programmable Gate Array (FPGA) platform, demonstrating significant improvements in inference speed and energy efficiency for the YOLOv5s model compared to traditional CPU core implementations. Experimental results indicate that the integrated RISC-V SoC with NVDLA achieves processing efficiency 17x and 25x greater than the standard ARM Cortex-A53 and Intel Core i7-12700H. This work establishes a foundation for deploying more complex deep learning models on resource-constrained edge devices, highlighting its potential for real-time object detection applications across various domains.