We study on-line strategies for solving problems with hybrid algorithms in the following situation: There is a problem Q and w basic algorithms for solving Q. We have a computer with λ≤w disjoint memory areas, each of which can be used to run a basic algorithm and store its intermediate results. In the worst case, only one basic algorithm can solve Q in finite time, and all the other basic algorithms will run forever without solving Q. A natural way to solve Q is to use a hybrid algorithm constructed from the basic algorithms. That is, we run a basic algorithm for some time, then switch to another, and continue this process until Q is solved. The goal is to solve Q in the least amount of time by optimizing the time spent running a given basic algorithm before switching to one of the alternatives. We use the notion of a competitive ratio to measure the efficiency of a hybrid algorithm. We construct optimal deterministic hybrid algorithms and efficient randomized hybrid algorithms. This resolves an open question posed by Baeza-Yates, Culberson, and Rawlins in the context of searching with multiple robots. We also prove that our randomized hybrid algorithms are optimal for λ = 1. This settles a conjecture of Kao, Reif, and Tate.