An efficient heterogeneous parallel algorithm of the 3D MOC for multizone heterogeneous systems

Recent innovations in powerful high-performance computing (HPC) architecture have enabled high-fidelity whole-core neutron transport simulations in a reasonable amount of time. Among high-fidelity numerical methods, the one-step 3D Method of Characteristic (MOC) is becoming increasingly popular in the reactor design community due to its numerical accuracy, geometrical flexibility and high degrees of parallelism. In this work, we propose a sophisticated heterogeneous parallel algorithm of the 3D MOC based on a supercomputer prototype consisting of brand-new multizone heterogeneous processors called the MT-3000, which has a peak double precision performance of 11.6 TFLOPS when operating at 1.2 GHz. In the 3D MOC calculation procedure, the total run time is dominated by the so-called MOC transport sweep where large amounts of tracks traverse the mesh grids along different directions. We significantly improve the performance of the MOC transport sweep through a three-level parallel algorithm with multiple hiding latency techniques. We have also implemented heterogeneous level-1 basic linear algebra subprograms (BLAS) that's the second computational hot spot. The 3D C5G7 benchmark problem is employed to verify the correctness and efficiency of the heterogeneous parallel 3D MOC algorithm. Numerical results show that it achieves very high performance on the MT-3000-based supercomputer prototype. Specifically, the three-layer parallel algorithm has proven to be highly scalable for each layer, and the Quarter-MT-3000 performance of the MOC transport sweep has achieved 23.6% of the peak performance, which is close to the theoretical upper limit.