Load balance heuristics for synchronous iterative applications on heterogeneous cluster systems

Heterogeneous computing systems are emerging as a computing infrastructure that will enable the use of distributed heterogeneous clusters for a variety of challenging applications. The actual challenge is the load balance for tightly-coupled applications. In this paper, we focus on the important subclass of tightly-coupled applications, synchronous iterative applications and formally define their load balance problem. Two novel static meta heuristic algorithms are proposed for the load distribution: a genetic tabu hybrid search (GTHS) algorithm and a host clustering based iterative search (HCIS) algorithm, when different communication computation ratios are considered. To this end, the analysis and experiment results demonstrate the effectiveness of heuristic algorithms.