LBP-TOP: A tensor unfolding revisit

Local Binary Pattern histograms from Three Orthogonal Planes (LBP-TOP) has shown its promising performance on facial expression recognition as well as human activity analysis, as it extracts features from spatial-temporal information. Originally, as the calculation of LBP-TOP has to traverse all the pixels in the three dimensional space to compute the LBP operation along XY, YT and XT planes respectively, the frequent use of loops in implementation shapely increases the computational costs. In this work, we aim to fasten the computational efficiency of LBP-TOP on spatial-temporal information and introduce the concept of tensor unfolding to accelerate the implementation process from three-dimensional space to two-dimensional space. The spatial-temporal information is interpreted as a 3-order tensor, and we use tensor unfolding method to compute three concatenated big matrices in two-dimensional space. LBP operation is then performed on the three unfolded matrices. As the demand for loops in implementation is largely down, the computational cost is substantially reduced. We compared the computational time of the original LBP-TOP implementation to that of our fast LBP-TOP implementation on both synthetic and real data, the results show that the fast LBP-TOP implementation is much more time-saving than the original one. The implementation code of the proposed fast LBP-TOP is now publicly available (The implementation code of the proposed fast LBP-TOP can be downloaded at http://www.ee.oulu.fi/ research/imag/cmvs/files/code/Fast LBPTOP Code.zip).