This paper presents an algorithm for tracking the articulate hand motion in monocular video sequences. The task is challenging due to the high degrees of freedom involved in the hand motion. The complexity can be reduced by considering the natural motion constraints. To take advantage of the constraints, we propose to use a nonparametric representation of the feasible configuration space and employ a Monte Carlo Nelder-Mead simplex search algorithm. The tracker combines the strengths of both sequential Monte Carlo and direct search algorithms. First, its multiple hypotheses nature increases the chance of the simplex method to identify the global maximum. Second, the direct search algorithm produces a set of more representative particles. Experiment results show that this hybrid approach is robust for tracking the hand motion.