Implementing Tensor Flow for Multidimensional Markerless Pose Estimation

Session Number

Project ID: CMPS 3

Advisor(s)

Dr. Craig Weiss; Northwestern University Feinberg School of Medicine, John Disterhoft Laboratory

Discipline

Computer Science

Start Date

22-4-2020 9:10 AM

End Date

22-4-2020 9:25 AM

Abstract

Quantifying behavior is critical to neuroscientific activities to draw correlations between brain activity and specific activations of the body. While videography enables swift recording of animal behavior, it remains an elusive challenge to isolate raw movement pattern data suitable for neurological correlation. While sensors and reflective markers aid systems in recording movement patterns, these physical flags impose an external stress upon the animal which can alter the physiological state of the animal outside of the parameters of the experiment. Therefore, a need emerges for automated processing of video data to obtain markerless pose estimation for maximizing behavior tracking efficiency across multiple species. Our lab is currently implementing a trained machine learning package written in Python, DeepLabCut, to test the efficacy of this process. By building upon tensor flow, the DeepLabCUt package enables human-like accuracy (within 0.05%) with minimal frame training (<200 >frames) by allowing the researcher to select and label specific Regions of Interest (ROIs) such as the paws, tongue, etc. The motion of these ROIs can then be analyzed in response to a specific stimulus during free ambulation and can also be correlated to EEG data or neurological stimulation using MatPlotLib to visually present the data.

Share

COinS
 
Apr 22nd, 9:10 AM Apr 22nd, 9:25 AM

Implementing Tensor Flow for Multidimensional Markerless Pose Estimation

Quantifying behavior is critical to neuroscientific activities to draw correlations between brain activity and specific activations of the body. While videography enables swift recording of animal behavior, it remains an elusive challenge to isolate raw movement pattern data suitable for neurological correlation. While sensors and reflective markers aid systems in recording movement patterns, these physical flags impose an external stress upon the animal which can alter the physiological state of the animal outside of the parameters of the experiment. Therefore, a need emerges for automated processing of video data to obtain markerless pose estimation for maximizing behavior tracking efficiency across multiple species. Our lab is currently implementing a trained machine learning package written in Python, DeepLabCut, to test the efficacy of this process. By building upon tensor flow, the DeepLabCUt package enables human-like accuracy (within 0.05%) with minimal frame training (<200>frames) by allowing the researcher to select and label specific Regions of Interest (ROIs) such as the paws, tongue, etc. The motion of these ROIs can then be analyzed in response to a specific stimulus during free ambulation and can also be correlated to EEG data or neurological stimulation using MatPlotLib to visually present the data.