A Formalism for Customizing and Training Intelligent Assistive Devices

Highly capable assistive robotic arms are well-poised to dramatically increase the independence of those with severe motor impairments, by reducing their dependence on caregivers to perform select activities of daily living. However, the more sophisticated a robotic arm, the more complicated its control. Intuitive operation remains a challenge that only increases with task complexity, and is exasperated by limited or low-dimensional control interfaces. Our solution is to introduce machine automation and intelligence. We propose a formalism for customizable shared control that enables users to customize the way they share control with intelligent assistive devices based on the user's abilities and preferences. In our formalism, the system arbitrates between user input and the autonomous policy prediction, based on the confidence it has in the policy’s prediction and in the user’s ability to perform the task. Moreover, the system is invisible and able to augment minimal teleoperation interfaces (e.g. Sip-N-Puff). This point is critical for users whose own control signals are limited, and are only able to operate minimal interfaces as a result. While dexterous manipulation can be difficult for a user to achieve using minimal control interfaces, full robot autonomy is often lacking in robustness or unsatisfactory for users who wish to retain some control authority. Assistive teleoperation offers a customized and robust alternative. We propose methods for customizing system components based on user data that can generalize to new situations. In particular, we aim to address the questions of (i) how can the system adapt its arbitration function to a new user or task, (ii) how can a user achieve the optimal arbitration function for a given task, (iii) how can the robot learn good policies from user demonstration and interaction, and (iv) whether there are user-centric and/or task-centric measures of confidence. We test the proposed methods for customizing system components in user studies with high Spinal Cord Injury patients as well as uninjured subjects. The result will be a system that can learn from its user and improve over time. The larger goal is assistive device—here specifically, robotic arm—operation that is accessible to, and intuitive for, persons with extremely limited or no upper limb motor control.