An algorithm is presented for generating trajectories for efficient exploration that takes into account a probabilistic representation of information density over a sampling region. The problem is cast as a continuous-time trajectory optimization problem, where the objective function directly involves the relationship between the probability density functions representing the spatial distribution and the statistical representation of the time-averaged trajectory. The difference is expressed using ergodicity. It is shown that the trajectory optimization problem can be solved using descent directions that are solutions to linear quadratic optimal control problems. The proposed method generates continuous-time optimal feedback controllers, demonstrated in simulation for a nonlinear sensor model.