Using DeepLabCut to Predict Locations of Subdermal Landmarks from Video

Diya Basrai*, Emanuel Andrada, Janina Weber, Martin S. Fischer, Matthew Tresch

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent developments in markerless tracking software such as DeepLabCut (DLC) allow estimation of skin landmark positions during behavioral studies. However, studies that require highly accurate skeletal kinematics require estimation of 3D positions of subdermal landmarks such as joint centers of rotation or skeletal features. In many animals, significant slippage between the skin and underlying skeleton makes accurate tracking of skeletal configuration from skin landmarks difficult. While biplanar, high-speed X-ray acquisition cameras offer a way to measure accurate skeletal configuration using tantalum markers and XROMM, this technology is expensive, not widely available, and the manual annotation required is time-consuming. Here, we present an approach that utilizes DLC to estimate subdermal landmarks in a rat from video collected from two standard cameras. By simultaneously recording X-ray and live video of an animal, we train a DLC model to predict the skin locations representing the projected positions of subdermal landmarks obtained from X-ray data. Predicted skin locations from multiple camera views were triangulated to reconstruct depth-accurate positions of subdermal landmarks. We found that DLC was able to estimate skeletal landmarks with good 3D accuracy, suggesting that this might be an approach to provide accurate estimates of skeletal configuration using standard live video.

Original languageEnglish (US)
Title of host publicationBiomimetic and Biohybrid Systems - 11th International Conference, Living Machines 2022, Proceedings
EditorsAlexander Hunt, Vasiliki Vouloutsi, Kenneth Moses, Roger Quinn, Anna Mura, Tony Prescott, Paul F. Verschure
PublisherSpringer Science and Business Media Deutschland GmbH
Pages292-296
Number of pages5
ISBN (Print)9783031204692
DOIs
StatePublished - 2022
Event11th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2022 - Virtual, Online
Duration: Jul 19 2022Jul 22 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13548 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference11th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2022
CityVirtual, Online
Period7/19/227/22/22

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Using DeepLabCut to Predict Locations of Subdermal Landmarks from Video'. Together they form a unique fingerprint.

Cite this