TY - GEN
T1 - Using DeepLabCut to Predict Locations of Subdermal Landmarks from Video
AU - Basrai, Diya
AU - Andrada, Emanuel
AU - Weber, Janina
AU - Fischer, Martin S.
AU - Tresch, Matthew
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Recent developments in markerless tracking software such as DeepLabCut (DLC) allow estimation of skin landmark positions during behavioral studies. However, studies that require highly accurate skeletal kinematics require estimation of 3D positions of subdermal landmarks such as joint centers of rotation or skeletal features. In many animals, significant slippage between the skin and underlying skeleton makes accurate tracking of skeletal configuration from skin landmarks difficult. While biplanar, high-speed X-ray acquisition cameras offer a way to measure accurate skeletal configuration using tantalum markers and XROMM, this technology is expensive, not widely available, and the manual annotation required is time-consuming. Here, we present an approach that utilizes DLC to estimate subdermal landmarks in a rat from video collected from two standard cameras. By simultaneously recording X-ray and live video of an animal, we train a DLC model to predict the skin locations representing the projected positions of subdermal landmarks obtained from X-ray data. Predicted skin locations from multiple camera views were triangulated to reconstruct depth-accurate positions of subdermal landmarks. We found that DLC was able to estimate skeletal landmarks with good 3D accuracy, suggesting that this might be an approach to provide accurate estimates of skeletal configuration using standard live video.
AB - Recent developments in markerless tracking software such as DeepLabCut (DLC) allow estimation of skin landmark positions during behavioral studies. However, studies that require highly accurate skeletal kinematics require estimation of 3D positions of subdermal landmarks such as joint centers of rotation or skeletal features. In many animals, significant slippage between the skin and underlying skeleton makes accurate tracking of skeletal configuration from skin landmarks difficult. While biplanar, high-speed X-ray acquisition cameras offer a way to measure accurate skeletal configuration using tantalum markers and XROMM, this technology is expensive, not widely available, and the manual annotation required is time-consuming. Here, we present an approach that utilizes DLC to estimate subdermal landmarks in a rat from video collected from two standard cameras. By simultaneously recording X-ray and live video of an animal, we train a DLC model to predict the skin locations representing the projected positions of subdermal landmarks obtained from X-ray data. Predicted skin locations from multiple camera views were triangulated to reconstruct depth-accurate positions of subdermal landmarks. We found that DLC was able to estimate skeletal landmarks with good 3D accuracy, suggesting that this might be an approach to provide accurate estimates of skeletal configuration using standard live video.
UR - http://www.scopus.com/inward/record.url?scp=85144285052&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85144285052&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-20470-8_29
DO - 10.1007/978-3-031-20470-8_29
M3 - Conference contribution
AN - SCOPUS:85144285052
SN - 9783031204692
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 292
EP - 296
BT - Biomimetic and Biohybrid Systems - 11th International Conference, Living Machines 2022, Proceedings
A2 - Hunt, Alexander
A2 - Vouloutsi, Vasiliki
A2 - Moses, Kenneth
A2 - Quinn, Roger
A2 - Mura, Anna
A2 - Prescott, Tony
A2 - Verschure, Paul F.
PB - Springer Science and Business Media Deutschland GmbH
T2 - 11th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2022
Y2 - 19 July 2022 through 22 July 2022
ER -