Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell University Graduate School
  3. Cornell Theses and Dissertations
  4. Synthesis of Brain Images using Deep Learning

Synthesis of Brain Images using Deep Learning

File(s)
Ngo_cornellgrad_0058F_13160.pdf (26.54 MB)
Permanent Link(s)
https://doi.org/10.7298/r555-4b68
https://hdl.handle.net/1813/112017
Collections
Cornell Theses and Dissertations
Author
Ngo, Gia
Abstract

Deep learning and neuroscience research have significantly progressed in the past few decades. While deep learning's conception and several key developments have been inspired by neuroscientific knowledge, the reverse osmosis of artificial neural network application in neuroscience has been limited. In this thesis, we tackle two important, but relatively under-explored tasks of brain images synthesis using deep learning. The first task bridges the gap between task-based and resting-state functional magnetic resonance imaging (fMRI), which represent the two most common experimental paradigms of functional neuroimaging. While resting-state offers a flexible and scalable approach for characterizing brain function, task-based techniques provide superior localization. We build on recent deep learning methods to create a model that predicts task-based contrast maps from resting-state fMRI scans. Specifically, we propose BrainSurfCNN, a surface-based fully-convolutional neural network model that works with a representation of the brain's cortical sheet. BrainSurfCNN achieves exceptional predictive accuracy on independent test data from the Human Connectome Project, which is on par with the repeat reliability of the measured subject-level contrast maps. Conversely, our analyses reveal that a previously published benchmark is no better than group-average contrast maps. We demonstrate that BrainSurfCNN can generalize remarkably well to novel domains with limited training data. In the second task, we apply deep learning methods in the field of neuroimaging coordinate-based meta-analysis. Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific studies have collectively accumulated an extensive wealth of results. Digesting this growing literature and obtaining novel insights remains to be a major challenge, since existing meta-analytic tools are constrained to keyword queries. In this paper, we present \textit{Text2Brain}, an easy to use tool for synthesizing brain activation maps from open-ended text queries. Text2Brain was built on a transformer-based neural network language model and a coordinate-based meta-analysis of neuroimaging studies. Text2Brain combines a transformer-based text encoder and a 3D image generator, and was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published studies. In our experiments, we demonstrate that Text2Brain can synthesize meaningful neural activation patterns from various free-form textual descriptions. The methodological advancements and results presented in this thesis are only the very first steps in their respective research directions but demonstrate the massive potential of deep learning in advancing neuroimaging applications and research.

Description
142 pages
Date Issued
2022-08
Keywords
coordinate-based meta-analysis
•
deep learning
•
resting-state fMRI
•
task-based fMRI
•
transformer
Committee Chair
Sabuncu, Mert
Committee Member
Kuceyeski, Amy Frances
Weinberger, Kilian Quirin
Degree Discipline
Electrical and Computer Engineering
Degree Name
Ph. D., Electrical and Computer Engineering
Degree Level
Doctor of Philosophy
Rights
Attribution 4.0 International
Rights URI
https://creativecommons.org/licenses/by/4.0/
Type
dissertation or thesis
Link(s) to Catalog Record
https://newcatalog.library.cornell.edu/catalog/15578831

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance