Multiple-candidate hierachical block matching with inherent smoothness properties

In this paper, we present a multiple candidate hierarchical block matching (MCHBM) estimation approach of the apparent motion vector field (MVF) in image sequences. In contrast to the standard hierarchical block matching (HBM) approach, which considers only the best solution at each level (single candidate case), MCHBM considers the H best candidate solution vectors and the associated matching error at that level. Then, the H selected candidate solution vectors are projected to the next higher resolution level in order to serve as initial estimates for the search process at that level, which improves all H estimates. Thus, at the highest resolution level, the final vector is selected by taking into account the global suitability of the vector and not just the local error. The resulting MVF approximates the true motion by avoiding local minima which lead to solutions that differ from the true MVF. The multiple candidate approach is considered in combination with the overlapped multi-grid and multi-resolution HBM estimation methods. The final algorithm has very good smoothness properties regardless of the application of any additional magnitude and/or smoothness constraints. Experimental results on video-conference image sequences demonstrate the improved performance of the proposed methods.