Sensor Fusion of Vision, Kinetics, and Kinematics for Forward Prediction During Walking With a Transfemoral Prosthesis

Powered lower limb prostheses utilize activity specific controllers, where predicting desired activity and transitioning in a timely manner is essential for seamless locomotion without trips. Previously research considered EMG and mechanical sensors for these predictions; however, predictability must be improved to ensure safe usage. Previously we showed that combining features from mechanical sensors, EMG and Vision yielded greater repeatability, greater separability and lower variability. Here we compare performance of offline forward prediction systems combining these different sensor modalities. We trained and tested subject-specific classifiers for steady-state and transition steps, with data from 8 able-bodied subjects, 4 able-bodied subjects walking with a powered knee-ankle prosthesis using a bypass socket, and a single transfemoral amputee walking with a knee-ankle prosthesis. Fusing Mechanical, EMG, and Vision features produced the best classification for all subjects, with transition error rates in the range of 1% and steady-state error rates close to 0%. Though generalizability was good across able-bodied subjects, it was poor when training with able-bodied or bypass subjects and testing with our amputee subject, regardless of sensor modality, particularly for transition steps. Therefore, we believe a general classifier will require inclusion of amputee training data. Future work will test more subjects and continue development of a general control system.