Evaluation and comparison of ten data race detection techniques

Many techniques for dynamically detecting data races in multithreaded programs have been proposed. However, it is unclear how these techniques compare in terms of precision, overhead and scalability. This paper presents an experiment to evaluate ten data-race detection techniques on 100 small-scale or middle-scale C/C++ programs. The selected ten techniques, implemented in the same Maple framework, cover not only the classical but also the state-of-the-art in dynamical data-race detection. We compare the ten techniques and try to give reasonable explanations for why some techniques are weaker or stronger than other ones. Evaluation results show that no one technique performs perfectly for all programs according to the three criteria. Based on the evaluation and comparison, we give suggestions of which technique is the most suitable one to use when the target program exhibits particular characteristics. Later researchers can also benefit from our results to construct a better detection technique.