3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images. (arXiv:2107.09700v1 [eess.IV])

Image synthesis via Generative Adversarial Networks (GANs) of
three-dimensional (3D) medical images has great potential that can be extended
to many medical applications, such as, image enhancement and disease
progression modeling. However, current GAN technologies for 3D medical image
synthesis need to be significantly improved to be readily adapted to real-world
medical problems. In this paper, we extend the state-of-the-art StyleGAN2
model, which natively works with two-dimensional images, to enable 3D image
synthesis. In addition to the image synthesis, we investigate the
controllability and interpretability of the 3D-StyleGAN via style vectors
inherited form the original StyleGAN2 that are highly suitable for medical
applications: (i) the latent space projection and reconstruction of unseen real
images, and (ii) style mixing. We demonstrate the 3D-StyleGAN’s performance and
feasibility with ~12,000 three-dimensional full brain MR T1 images, although it
can be applied to any 3D volumetric images. Furthermore, we explore different
configurations of hyperparameters to investigate potential improvement of the
image synthesis with larger networks. The codes and pre-trained networks are
available online: https://github.com/sh4174/3DStyleGAN.

Source: https://arxiv.org/abs/2107.09700


Related post