TY - GEN

T1 - Discrete-time local dynamic programming

AU - Berniker, Max

AU - Kording, Konrad

PY - 2011

Y1 - 2011

N2 - Optimal control theory is a powerful analytical tool useful for many diverse fields, including biological motor control, where the theory is used to predict characteristics of motor control problems under optimal conditions. However, finding solutions to these control problems can be very difficult when examining biological systems, where nonlinearity and stochasticity are typical. In an effort to overcome this dilemma and analyze more realistic problems, we present an algorithm that approximates the solution to the discrete-time Hamilton-Jacobi-Bellman equations. As with similar local dynamic programming algorithms, the algorithm approximates a local solution around a nominal trajectory and progressively improves the trajectory and the value function's local estimate. Using this algorithm, we obtain optimal solutions for a single joint musculo-skeletal system. In particular, we take advantage of this new algorithm to examine solutions with fast and discontinuous dynamics and non-Gaussian noise. These solutions are examined for some of the stereotypical responses of biological systems, such as the tri-phasic muscle activations and bell-shaped velocity profiles. The results are also compared with their deterministic counterparts, emphasizing the need for stochastic solutions.

AB - Optimal control theory is a powerful analytical tool useful for many diverse fields, including biological motor control, where the theory is used to predict characteristics of motor control problems under optimal conditions. However, finding solutions to these control problems can be very difficult when examining biological systems, where nonlinearity and stochasticity are typical. In an effort to overcome this dilemma and analyze more realistic problems, we present an algorithm that approximates the solution to the discrete-time Hamilton-Jacobi-Bellman equations. As with similar local dynamic programming algorithms, the algorithm approximates a local solution around a nominal trajectory and progressively improves the trajectory and the value function's local estimate. Using this algorithm, we obtain optimal solutions for a single joint musculo-skeletal system. In particular, we take advantage of this new algorithm to examine solutions with fast and discontinuous dynamics and non-Gaussian noise. These solutions are examined for some of the stereotypical responses of biological systems, such as the tri-phasic muscle activations and bell-shaped velocity profiles. The results are also compared with their deterministic counterparts, emphasizing the need for stochastic solutions.

UR - http://www.scopus.com/inward/record.url?scp=80053165182&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053165182&partnerID=8YFLogxK

U2 - 10.1109/acc.2011.5990808

DO - 10.1109/acc.2011.5990808

M3 - Conference contribution

AN - SCOPUS:80053165182

SN - 9781457700804

T3 - Proceedings of the American Control Conference

SP - 618

EP - 625

BT - Proceedings of the 2011 American Control Conference, ACC 2011

PB - Institute of Electrical and Electronics Engineers Inc.

ER -