Implementing Tensor Flow for Multidimensional Markerless Pose Estimation
Session Number
Project ID: CMPS 3
Advisor(s)
Dr. Craig Weiss; Northwestern University Feinberg School of Medicine, John Disterhoft Laboratory
Discipline
Computer Science
Start Date
22-4-2020 9:10 AM
End Date
22-4-2020 9:25 AM
Abstract
Quantifying behavior is critical to neuroscientific activities to draw correlations between brain activity and specific activations of the body. While videography enables swift recording of animal behavior, it remains an elusive challenge to isolate raw movement pattern data suitable for neurological correlation. While sensors and reflective markers aid systems in recording movement patterns, these physical flags impose an external stress upon the animal which can alter the physiological state of the animal outside of the parameters of the experiment. Therefore, a need emerges for automated processing of video data to obtain markerless pose estimation for maximizing behavior tracking efficiency across multiple species. Our lab is currently implementing a trained machine learning package written in Python, DeepLabCut, to test the efficacy of this process. By building upon tensor flow, the DeepLabCUt package enables human-like accuracy (within 0.05%) with minimal frame training (<200 >frames) by allowing the researcher to select and label specific Regions of Interest (ROIs) such as the paws, tongue, etc. The motion of these ROIs can then be analyzed in response to a specific stimulus during free ambulation and can also be correlated to EEG data or neurological stimulation using MatPlotLib to visually present the data.
Implementing Tensor Flow for Multidimensional Markerless Pose Estimation
Quantifying behavior is critical to neuroscientific activities to draw correlations between brain activity and specific activations of the body. While videography enables swift recording of animal behavior, it remains an elusive challenge to isolate raw movement pattern data suitable for neurological correlation. While sensors and reflective markers aid systems in recording movement patterns, these physical flags impose an external stress upon the animal which can alter the physiological state of the animal outside of the parameters of the experiment. Therefore, a need emerges for automated processing of video data to obtain markerless pose estimation for maximizing behavior tracking efficiency across multiple species. Our lab is currently implementing a trained machine learning package written in Python, DeepLabCut, to test the efficacy of this process. By building upon tensor flow, the DeepLabCUt package enables human-like accuracy (within 0.05%) with minimal frame training (<200>frames) by allowing the researcher to select and label specific Regions of Interest (ROIs) such as the paws, tongue, etc. The motion of these ROIs can then be analyzed in response to a specific stimulus during free ambulation and can also be correlated to EEG data or neurological stimulation using MatPlotLib to visually present the data.