Pivot-FSOF: A Full-Stack Optimization Framework for real-time DiT-based TTS on edge devices

Compute-intensive models have become the paradigm for high-fidelity generative tasks; however, severe memory bandwidth and I/O bottlenecks hinder their deployment on edge devices. To address this issue, we propose an algorithm-hardware co-design optimization framework, utilizing Text-to-Speech (TTS) as a case study. Algorithmically, we identify a “semantic pivot” during timestep accumulation and design a pivot-aware structured pruning strategy, coupled with sensitivity-aware INT8 quantization and mixed-supervision distillation. Architecturally, we implement targeted edge GPU accelerations, including a fused Rotary Positional Embedding (RoPE) CUDA kernel with vectorized memory access, a pointer-based zero-copy memory handoff mechanism, and CUDA Graph recording to eliminate redundant memory transactions and kernel launch overheads. We integrated this optimization framework on the NVIDIA Jetson AGX Orin and conducted extensive experiments. Experimental results demonstrate that the proposed framework reduces the network layers by 27% while maintaining an objective speaker similarity score of 80.45% under the SIM-T evaluation protocol. Furthermore, the framework achieves a 9.13× end-to-end inference speedup over the ONNX Runtime FP32 baseline and a 2.23× acceleration over the TensorRT FP16 industrial baseline.