Daniela de Albuquerque

Daniela de Albuquerque

MD-PhD candidate developing deep generative models for neuroscience and biomedical data.

Deep Generative Analysis for Task-Based Functional MRI Experiments


Schematic of the GAM-VAE fMRI model . Each fMRI volume is compressed to a lower-dimensional representation using a probabilistic encoder network. Latent space vectors are then sampled and fed to a probabilistic decoder, which yields a base map (blue) and a set of covariate-specific maps (dark green). The gain of each covariate map (light green) is the sum of a linear effect term and a potential non-linearity modeled by a Gaussian process. Finally, the full mean reconstruction of the input is obtained by adding the base and the scaled covariate maps. Variance is modeled on a per-voxel basis, yielding a separate variance map (purple).

In this project, we develped a 3D convolutional Variational Autoencoder (VAE) and embedded it in a Generalized Additive Modeling (GAM) framework, so as to capture sources of spatial variability across entire brain volumes, while also yielding separable and interpretable regressor effect maps. These regressor maps are constructed as the product of a spatial map (learned by the VAE) and a (potentially non-linear) gain, modelled by a covariate-specific Gaussian Process (GP) and represent the effect of a given task regressor on brain-wide activity.


Average effect maps for visual data set experiments. Group average maps generated by the proposed GAM-VAE model (top row) vs. the GLM (bottom row). Note that GAM-VAE model not only captures V1 effect appropriately, but also produces smoother and more contiguous clusters, with fewer spurious activations outside of V1.

This model has the key advange of correctly accounting for spatial autocorrelations typically encountered in fMRI data, and which are often ignored in standard fMRI analyses. Additionally, similarly to classic fMRI analysis methods, it generates separate volumetric regressor effect maps, which are highly interpretable and useful for clinicians and researchers interested in conducting fMRI experiments.

For more details, check our JMLR publication. Code for this project is available at this repository.