October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
Deep Convolutional Neural Networks for Predicting Head Pose During Brain MRI Acquisition
Author Affiliations
  • Yijun Zhao
    Fordham University
  • Hui Yuan
    Fordham University
  • Jingjie Zhou
    Fordham University
  • Samantha Martin
    NYU Langone School of Medicine
  • Heath Pardoe
    NYU Langone School of Medicine
Journal of Vision October 2020, Vol.20, 817. doi:https://doi.org/10.1167/jov.20.11.817
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Yijun Zhao, Hui Yuan, Jingjie Zhou, Samantha Martin, Heath Pardoe; Deep Convolutional Neural Networks for Predicting Head Pose During Brain MRI Acquisition. Journal of Vision 2020;20(11):817. https://doi.org/10.1167/jov.20.11.817.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

In-scanner head motion is a major source of error for brain MRI. There are currently no widely available methods for directly assessing in-scanner motion during the acquisition of neuroanatomical sequences. In this study, we developed a convolutional neural network-based method to measure changes in head pose over the duration of a scan via analysis of video obtained from an in-scanner eye tracker. Our method will allow for direct measurement of head motion during scanning, which can then be used to statistically control for head motion in quantitative neuroanatomy studies. Video data was obtained at a sampling rate of 30Hz. Thirteen healthy adults were imaged on a Siemens Prisma MRI scanner with an in-bore eye-tracker system. Ground truth head pose estimates were obtained using fMRI acquisitions that were acquired simultaneously with in-scanner video using a repetition time=1.3s (total scan time=7min). Participants carried out a series of deliberate head motions during the acquisition. Head pose over the duration of the scan was parameterized by coregistering each image volume to the first volume of the fMRI scan using rigid body registration. We employed deep learning methodologies to build 1) a classification model that classifies each video frame as “motion” vs. “no-motion” in reference to the starting position, and 2) a regression model that predicts the magnitude of subject motion for each frame. Our model is a deep convolutional neural network with nine convolutional layers, four max-pooling layers, and seven dense layers. Our classification model achieved an overall accuracy of 89% with 96% and 84% performance for the “no-motion" and “motion" frames respectively. Our regression model obtained an overall generalization loss (MSE)<7.8E-3. Sample performance plots are provided in the Supplementary Material page. Our study provides convincing data supporting the utility in-scanner video and deep learning methodologies for detecting subject motion during MRI scans.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.