8.1 Experiments on Image Benchmark datasets
To verify that inflationary flows successfully compress high-dimensional data, we compressed two benchmark datasets (CIFAR-10 Krizhevsky et al., 2009 AFHQv2 Choi et al., 2020) to 20%, 10%, and 2% of their nominal dimensionality and examined their round-trip and pure generative performance. For estimating score, we used DBM networks trained using the scheme of Karras et al., 2022 combined with the novel scaling, noise, and preconditioning proposed above (see Appendices B.1, B.4.2 of our paper for details).
In the table below, we show computed Frechet Inception Distance (FID) scores Heusel et al., 2017 over three different sets of 50,000 randomly generated images for each of these schedules and for both datasets. Note that preserving intrinsic dimension (PRP schedules) leads to better (smaller) FID scores, as expected. Perhaps surprisingly, increasing the number of preserved dimensions in the PR-Reducing regimes leads to worse (i.e., higher) FID scores. This is because retaining more dimensions in our PR-Reducing schedules leads to larger scale gaps between our preserved and compressed dimensions (i.e., larger \( e^{\rho(g_* - g_i)T} \)), thus increasing the required noise range over which networks must optimally estimate scores. That is, PR-Reducing schedules with higher numbers of preserved-dimensions pose a more challenging learning problem (Appendix B.2 of our paper).
In the same table, we also include mean squared errors (MSEs) for round-trip integration experiments (i.e., from data to compressed space and back) across all schedules and datasets, computed over 3 randomly sampled sets, each with 10K images. Trends observed here are similar to the ones seen for FID experiments. PR-Preserving networks produced lower MSEs overall, whereas PR-Reducing networks with higher percentage of preserved dimensions (i.e., 20%) yielded higher MSEs than PR-Reducing networks with smaller number of preserved dimensions (i.e., 2%). Together, these results suggest that dimension reduction with inflationary flows may necessitate trade-offs between effective compression and the difficulty of score estimation, as noted above.
| FID | CIFAR-10 | 17.01 ± 0.10 | 22.23 ± 0.16 | 23.63 ±0.13 | 25.93 ± 0.40 |
| Round-Trip MSE | CIFAR-10 | 0.23 ± 0.03 | 2.06 ± 0.04 | 2.25 ± 0.01 | 4.16 ± 0.39 |
| FID | AFHQv2 | 11.89 ± 0.08 | 13.07 ± 0.07 | 13.67 ± 0.09 | 16.77 ± 0.14 |
| Round-Trip MSE | AFHQv2 | 0.38 ± 0.04 | 5.57 ± 0.20 | 7.95 ± 0.31 | 8.17 ± 0.08 |
Finally, we include below animations showcasing roundtrip-integration and generation experiments for some sample small batches of the AFHQv2 dataset (Choi et al., 2020). For the round-trip experiments, reconstructions are qualitatively good across all conditions. Of note, networks trained with proposed dimension-reducing schedules produce inflated representations that resemble true low-rank Gaussian samples, consistent with a reduced-dimensional latent space.